An Improved Multi-Objective Memetic Algorithm with Q-Learning for Distributed Hybrid Flow Shop Considering Sequence-Dependent Setup Times

Abstract

Most multi-objective studies on distributed hybrid flow shops that include tardiness-related objectives focus solely on optimizing makespan alongside a single tardiness objective. However, in real-world scenarios with strict contractual deadlines or high penalty costs for delays, minimizing both total tardiness and the number of tardy jobs becomes critically important. This paper addresses this gap by prioritizing tardiness-related objectives while simultaneously optimizing makespan, total tardiness, and the number of tardy jobs. It investigates a distributed hybrid flow shop scheduling problem (DHFSP), which has some symmetries on machines. We propose an improved multi-objective memetic algorithm incorporating Q-learning (IMOMA-QL) to solve this problem, featuring (1) a hybrid initialization method that generates high-quality, diverse solutions by balancing all three objectives; (2) a multi-factory SB2OX crossover operator preserving high-performance job sequences across factories; (3) six problem-specific neighborhood structures for efficient solution space exploration; and (4) a Q-learning-guided variable neighborhood search that adaptively selects neighborhood structures. Based on extensive numerical experiments across 100 generated instances and a comprehensive comparison with four comparative algorithms, the proposed IMOMA demonstrates its effectiveness and proves to be a competitive method for solving the DHFSP.

1. Introduction

With the development of globalization, many enterprises setup production bases in different countries and regions. Distributed flow shop can coordinate production activities between different geographical areas, achieve load balancing between different factories, avoid overloading of some factories or waste of resources in some factories, and improve the overall production efficiency. The distributed flow shop scheduling problem (DFSP) has been thoroughly considered, yielding numerous results addressing a set of pragmatic constraints such as re-entrant [1], no idle [2], deteriorating job [3], sequence-dependent setup times (SDSTs) [4,5,6,7], and energy-conscious [8]. Various evolution algorithms have been widely used to solve real-world problems extensively, such as the memetic algorithm [9,10,11], artificial bee colony algorithm [12], distributional estimation algorithm (DEA) [13], discrete fruit fly optimization algorithm [14], hybrid meta-heuristic [15], variable neighborhood descent algorithm [8], and spherical evolution algorithm [16].

To address distributed scheduling problems (DSPs), researchers have developed diverse solution approaches. Meta-heuristic algorithms have gained widespread adoption for shop scheduling optimization due to their notable advantages: straightforward implementation, robust performance, rapid convergence characteristics, and seamless compatibility with other algorithmic frameworks.

Memetic algorithms (MAs) have gained significant attention for their effectiveness in tackling various NP-hard optimization problems, particularly in single-objective distributed scheduling problems (DSPs). For instance, Wang [17] developed an MA based on EDA to address DFSP and minimize makespan. To optimize the makespan in a two-stage DFSP, Zhang [18] integrated the social spider optimization method into an MA framework. Wang [19] further explored a cooperative bi-population MA that incorporated collaborative initialization and inter-population cooperation and intensified local search for minimizing makespan in distributed hybrid flow shop scheduling problems (DHFSPs). Zhang et al. [20] achieved makespan minimization by leveraging cooperation within MA.

Memetic algorithms (MAs) have also been widely applied to DFSPs. Deng and Wang [11] investigated a multi-objective DFSP aimed at minimizing both makespan and total tardiness, and developed a competitive MA employing two populations with distinct operators for each objective. Wang [19] examined an energy-focused variant targeting the reduction of energy consumption and makespan, introducing a collaborative MA guided by reinforcement learning policy agents. Shao [21] proposed a network-based MA to minimize total tardiness, overall production cost, and carbon emissions.

The rapid advancement of AI is fundamentally transforming operations across a multitude of fields. As a typical reinforcement learning algorithm originating from dynamic programming, Q-learning makes the best decision at each step to optimize the overall process. To address the uncertainty in assembly job shop scheduling and to enhance scheduling algorithms under various production environments, Q-learning has been widely integrated into different frameworks. For instance, in assembly job shop scheduling, a dual-loop framework based on Q-learning is proposed to cope with environmental uncertainty by self-learning [22]. In DFSP, Q-learning is combined with metaheuristics such as the fruit fly optimization algorithm to enhance neighborhood selection and improve solution quality [23]. For the studied DFSP variant with consistent sublots, the method employs a value-based RL method. It is coupled with the meta-heuristic to achieve adaptive operator selection [24].

In practical scenarios, decision-makers are often concerned not only with minimizing the makespan, but also with tardiness-related objectives. Tardiness-based objectives are particularly important in industries where late deliveries incur significant penalties or disrupt downstream processes. Cai [25] proposed two enhanced shuffled leapfrog algorithms (SLFA) for solving DHFSP in a multi-processor setting, aiming to minimize both total tardiness and the makespan simultaneously. Later, Li developed neighborhood-based heuristic to address a two-stage DHFSP with SDST, targeting reductions in total tardiness and makespan [12]. In addition, Lei [26] investigated a SFLA with memeplex partitioning in DHFSP. To address the dual objectives of makespan and maximum tardiness minimization, Lei [27] crafted a novel multi-class optimization approach based on the teaching–learning paradigm, which enhances search efficiency through inter-class interaction.

Few studies have treated tardiness-related objectives as the main focus in multi-objective optimization. Lei and Zheng [28] tackled HFSP with assembly operations and minimized total tardiness, maximum tardiness, and makespan with tardiness objectives regarded as key ones. In the DHFSP, which is more complex than the standard HFSP, it is therefore of great importance to develop effective algorithms that can simultaneously optimize total tardiness, the number of tardy jobs, and makespan.

In light of the above literature on DFSP and memetic algorithms, this study addresses the DHFSP with SDST, aiming to optimize makespan, total tardiness, and the number of tardy jobs prioritizing tardiness-related objectives. To tackle this problem, a multi-objective memetic algorithm combined with Q-learning is proposed (IMOMA-QL). The major contributions of this paper are as follows:

• Hybrid initialization method—A mixed initialization strategy is proposed to simultaneously optimize total tardiness, the number of tardy jobs, and makespan, generating a high-quality and diverse population.
• Multi-factory SB2OX crossover operator—The Similar Block 2-Point Order Crossover (SB2OX) is extended to a multi-factory context, leveraging structural similarity of job sequences to retain high-quality sub-sequences and enhance information exchange between factories.
• Problem-specific neighborhood structures are developed to guide the search process toward more promising regions. Considering the optimization objectives and problem characteristics, these neighborhood structures effectively explore the solution space.
• Q-learning-guided variable neighborhood search—A Q-learning strategy is introduced to adaptively choose the most effective neighborhood structure during the search process. The reward is designed based on the change in distance between the new and old solutions to their nearest Pareto front solution, encouraging moves that improve convergence toward the Pareto front while balancing intensification and diversification.

The paper is divided into the following sections. Section 2 formally describes DHFSP with SDST and presents its mathematical model. It also introduces the foundational framework of the memetic algorithm. Section 3 elaborates on the details of our proposed IMOMA, including its novel initialization, genetic operators, and the Q-learning-guided variable neighborhood search. Section 4 provides a comprehensive evaluation of IMOMA, including the experimental setup, sensitivity analysis, and comparisons with four other algorithms. Following this, Section 5 summarizes the main findings of this study, discusses its limitations, and suggests potential directions for future research.

2. Description of the Problem

2.1. Multi-Objective Optimization Problem

Many applied contexts involve simultaneously optimizing multiple conflicting objectives subject to a set of constraints. These are known as multi-objective optimization problems (MOPs). A mathematical model for an MOP is defined as follows:

$$\left\{\begin{array}{c}minF\left(\mathbf{x}\right)=\left(f_1\left(\mathbf{x}\right),f_2\left(\mathbf{x}\right),\dots , f_m\left(\mathbf{x}\right)\right),\hfill \\ \mathbf{x}\in {\Re }^D\hfill \end{array}\right.$$

(1)

where ${\Re }^D$ denotes the decision space of the multi-objective optimization problem, D denotes the dimension of the decision space, m denotes the number of objectives included in the multi-objective optimization problem, $\mathbf{x}=\left(X_1,X_2,\dots ,X_i,\dots ,X_D\right)$ denotes the decision vector, and $X_i$ denotes the i-th decision variable.

$$\left\{\begin{array}{c}{f}_i\left(\mathbf{a}\right)\le f_i\left(\mathbf{b}\right),\forall i=1,2,\dots ,m\hfill \\ f_j\left(\mathbf{a}\right)<f_j\left(\mathbf{b}\right),\exists j=1,2,\dots ,m\hfill \end{array}\right.$$

(2)

When solution a does not dominate solution b and solution b does not dominate solution a, then solution a and solution b are said to be non-dominated, and solution a and solution b are considered to have the same performance in MOPs.

2.2. Problem Definition

A DHFSP instance comprises a set of n jobs, denoted by $J_1,J_2,\dots ,J_n$, and they need to complete s stages of processing, each of which is assigned to F factories and each of which is a hybrid flow shop with m parallel machines. Each operation then selects an appropriate machine in the assigned factory. Symmetry exists both in the processing sequence of adjacent jobs on the same machine and in the machine distribution across different factories. The optimization objectives are makepspan, total tardiness, and the number of tardy jobs. The notation used in this model is defined in

Table 1. Sets, parameters, and variables.

Symbol	Description
Sets
N	Factories: $N=\{1,\dots ,f,\dots ,F\}$
I	Stages: $I=\{1,\dots ,i,\dots ,k\}$
J	Jobs: $J=\{1,\dots ,j,\dots ,n\}$
P	Positions: $P=\{1,\dots ,p,\dots ,n\}$
$M_f$	Machines at factory f. $M_f=\{1,\dots ,m,\dots ,l\}$
$M_{i,f}$	Machines at stage i in factory f. $M_f=\{1,\dots ,m,\dots ,F\times l\}$
Parameters
$p_{j,i}$	Processing time of job j at stage i
$d_j$	due date of job j
$S_{j,i}$	Start time of job j at stage i
$E_{j,i}$	Completion time of job j at stage i
$M{S}_{f,m,p}$	Start time of machine m at pth position in factory f
$M{E}_{f,m,p}$	Completion time of machine m at pth position in factory f
$s_{j,j^{\prime },i}$	Setup time at stage i from job $j^{\prime }$ to job j
Variables
$U_j$	Binary variable: 1 if job j is tardy, 0 otherwise
$X_{j,f}$	Binary variable: 1 if job j is assigned to factory f, 0 otherwise
$Y_{j,f,m,p,i}$	Binary variable: 1 if job j is processed at the pth position
	of machine m at stage i in factory f, 0 otherwise

.

The mathematical formulation is presented below.

$$Minimise\sum _{j\in J}{T}_j=\sum _{j\in J}max\{0,E_{j,k}-d_j\}$$

(3)

$$Minimise{C}_{max}\hfill$$

(4)

$$Minimise\sum _{j\in J}{U}_j\hfill$$

(5)

$$\sum _{f\in C}{X}_{j,f}=1,\forall j\in J$$

(6)

$$X_{j,f}=\sum _{m\in M_{i,f}}\sum _{p\in P}{Y}_{j,f,m,p,i},\forall i\in I,j\in J,f\in N$$

(7)

$$\sum _{j\in J}{Y}_{j,f,m,p,i}\le 1,\forall f\in N,m\in M_{i,f},p\in P$$

(8)

$$\sum _{j\in J}{Y}_{j,f,m,p,i}\ge \sum _{j^{\prime }\in J}{Y}_{j^{\prime },f,m,p+1,i},\forall f\in F,i\in I,m\in M_{i,f},p\in \{1,\dots ,n-1\}$$

(9)

$$M{E}_{f,m,p}=M{S}_{f,m,p}+\sum _{j\in J}\left(p_{j,i}{Y}_{j,f,m,p,i}\right),\forall f\in N,i\in I,m\in M_{i,c}$$

(10)

$$M{S}_{f,m,p+1}\ge M{E}_{f,m,p},\forall f\in N,m\in M_f,p\in \{1,\dots ,n-1\}$$

(11)

$$M{S}_{f,m,1}\ge s_{jj,i}-M\left(1-Y_{j,f,m,p,i}\right),\forall f\in N,i\in I,m\in M_f,p\in P$$

(12)

$$M{S}_{f,m,p}\le S_{j,i}+M\left(1-Y_{j,f,m,p,i}\right),\forall f\in N,i\in I,j\in J,m\in M_{i,f},p\in P$$

(13)

$$M{S}_{f,m,p}\ge S_{j,i}-M\left(1-Y_{j,f,m,p,i}\right),\forall f\in N,i\in I,j\in J,m\in M_{i,f},p\in P$$

(14)

$$E_{j,i}=S_{j,i}+p_{j,i},\forall j\in J,i\in \{1,\dots m-1\}$$

(15)

$$E_{j,i}\le S_{j,i+1},\forall j\in J,i\in \{1,\dots k-1\}$$

(16)

$$M{S}_{f,m,p}\ge 0,\forall f\in N,m\in M_f,p\in P$$

(17)

$$S_{j,i}\ge 0,\forall j\in J,i\in I$$

(18)

The objective functions (3)–(5) are to minimize the total tardiness, makespan, and the number of tardy jobs. Constraint (6) specifies that every job must be assigned to a single factory. Constraints (7)–(9) collectively govern machine–job assignment and processing sequence: each job is processed by exactly one machine per stage; no machine may process more than one job simultaneously; and job assignments must adhere to consecutive machine positions, ensuring previous slots are occupied. Constraints (10) and (11) define the start and finish times for each machine position. Constraint (12) incorporates setup time requirements. According to Constraint (13), the start time of the first job on any machine must account for the required setup time. Constraints (14) and (15) align machine positions with the corresponding job order. Constraint (16) defines a job’s completion time as the sum of its start time and processing duration. Sequential processing of jobs is enforced by Constraint (17), while Constraint (18) ensures that no job can start processing before time zero, which is equivalent to assuming all jobs have a release time of zero.

The layout of the distributed hybrid flow shop is shown in Figure 1.

2.3. Encoding and Decoding Methods

This paper utilizes a permutation-based coding scheme, where the solution is represented by factory vector $F_f=\left\{F_1,\dots ,F_c,\dots ,F_f\right\}$, with each vector corresponding to a specific factory, and $\alpha =\left\{{\alpha }_1,\dots ,{\alpha }_j,\dots ,{\alpha }_n\right\}$ representing the job processing sequence for the first stage in each factory. In the decoding mechanism, a combination of “FIFO” (first in, first out) and “FMA” (first machine available) is employed.

2.4. A Simple Memetic Algorithm

The MA fuses global evolutionary search with dedicated local search within a population-based paradigm to effectively navigate the exploration–exploitation trade-off [29]. A simple MA typically iterates through the following phases: initial population generation, fitness evaluation, selection, crossover, mutation, local refinement, and population update. The key feature distinguishing MA from a standard genetic algorithm is the embedded local search procedure (Line 12), which intensively refines individuals to reach local optima within promising regions identified by the evolutionary process.

3. Improved Multi-Objective Memetic Alogorithm

3.1. Algorithm Procedure

The proposed multi-objective memetic algorithm (IMOMA) is composed of four main components. First, a hybrid initialization procedure adopts a combination of random generation and problem-specific heuristics to yield a diverse set of well-performing initial solutions. Second, the algorithm incorporates genetic operators, to perform population-based global exploration of the search space. Third, a Q-learning-guided multi-neighborhood search is applied, in which a reinforcement learning mechanism adaptively selects among multiple neighborhood structures to intensify the search and enhance convergence toward the Pareto front. The algorithm flowchart is illustrated in Figure A1 and the pseudocode is shown in Algorithm 1. The proposed IMOMA algorithm consists of three main components per iteration, each contributing to the overall computational complexity. The time complexity per iteration is $O(N·n+N^2)$, where N denotes the population size and n represents the number of jobs. Given a maximum of T iterations, the overall time complexity of IMOMA is $O\left(T·(N·n+N^2)\right)$.

Algorithm 1 Algorithm of IMOMA

Input:  $instance$, $date$, $parameters$ Output:  $ApproximatParetofront$   1: Initialize population $PO{P}_0$;   2: Set t=0;   3: while $cputime<cputim{e}_{max}$ do   4:     for each $i\in [1,popSize/2]$ do   5:         selecting two parents $p_1$, $p_2$ from $PO{P}_t$ by using the binary tournament selection;   6:         Generate offspring $s_1$, $s_2$ by using Crossover Operator;   7:         Generate offspring $s_1^{\prime }$, $s_2^{\prime }$ by using Mutation Operator;   8:         Merge $s_1^{\prime }$, $s_2^{\prime }$ into $Po{p}_t$   9:     end for 10:     Apply VNS guided by Q-learning to $Po{p}_t$ to generate offspring population $Po{p}_t^{\prime }$ 11:     Combine $Po{p}_t$ and $Po{p}_t^{\prime }$ into $CombinedPop$ 12:     Perform non-dominated sorting on $CombinedPop$ 13:     Select N best individuals to form $Po{p}_{t+1}$ 14: end while 15: Output Pareto optimal solutions

3.2. Hybrid Initialization

High-quality initialization plays a pivotal role in multi-objective optimization algorithms for DHFSP by significantly influencing convergence speed, solution diversity, and computational efficiency. A well-designed initialization strategy reduces the algorithm’s exploration burden.

To address the multi-objective characteristics of the DHFSP—specifically the simultaneous minimization of makespan, total tardiness, and number of tardy jobs—this research develops a hybrid initialization approach. The proposed methodology combines targeted heuristic generation with stochastic exploration mechanisms, ensuring both high-quality initial solutions and adequate population diversity.

This paper introduces six initialization methods based on the LPT (Longest Processing Time) and EDD (Earliest Due Date) rules, combined with randomization, to enhance both the quality and diversity of the initial population.

Method 1: Generate a job vector in descending order of the total processing time (LPT), and then iteratively insert job into each factory in the order of the job vector to minimize the maximum completion time.

Method 2: Generate a job vector in ascending order of the due date (EDD), and then iteratively insert job into each factory in the order of the job vector to minimize the total tardiness.

Method 3: Generate a job vector in ascending order of the due date (EDD), and then iteratively insert job into each factory in the order of the job vector to minimize the number of tardy job.

Method 4: Randomly generate a job vector, and then iteratively insert job into each factory in the order of the job vector to minimize the maximum completion time.

Method 5: Randomly generate a job vector, and then iteratively insert job into each factory in the order of the job vector to minimize the total tardiness.

Method 6: Randomly generate a job vector, and then iteratively insert job into each factory in the order of the job vector to minimize the number of tardy job.

Methods 1–3 each generate one solution, while methods 4–6 generate the rest of the population in the ratio 2:2:1. Specifically, Method 4 and Method 5 each generate $\lfloor 2(N-3)/5\rfloor$ solutions, and Method 6 generates $(N-3)-2\times \lfloor 2(N-3)/5\rfloor$ solutions to ensure the total population size is N. The random seed for the initialization is set to 2020.

3.3. Selection

Use binary tournament selection to select two parents. The steps are as follows:

• Randomly select two individuals: Two candidate solutions are chosen at random from the existing population.
• Selection of better individuals into the next generation.

  (1)

  If a dominates b, choose a.

  (2)

  If b dominates a, choose b.

  If both individuals are non-dominated, one of the two individuals is chosen at random.
• Repeat until the next generation is filled.

The flowchart of the selection process is shown in Figure 2.

3.4. Genetic Operator

SB2OX solves flow shop problems that take into account SDST [30], but it is only used to solve the flow shop scheduling problem for a single factory.

This article proposes crossover operators based on SB2OX that can be adapted to distributed flow shops. Two vectors are used to represent the scheduling sequences of the two parents.

$\mathrm{parent}1=\left[x_1^1,x_2^1,\dots ,x_{n_1}^1;x_1^2,x_2^2,\dots ,x_{n_2}^2;\dots ;x_1^F,x_2^F,\dots ,x_{n_F}^F\right]$, $\mathrm{parent}2=\left[{x_1^{\prime }}^1,{x_2^{\prime }}^1,\dots ,\right.$$\left.{x_{n_1^{\prime }}^{\prime }}^1;{x_1^{\prime }}^2,{x_2^{\prime }}^2,\dots ;{x_{n_2^{\prime }}^{\prime }}^2;\dots ,{x_1^{\prime }}^F,{x_2^{\prime }}^F,\dots ,{x_{n_F^{\prime }}^{\prime }}^F\right]$. The superscript x denotes the factory, and the subscript indicates the position in the scheduling sequence of that factory.

Step 1: Both parents check on a position-by-position basis. This step exclusively transfer identical blocks containing at least two consecutive matching jobs from both parents directly to their offspring. $[x_p^{c_1},\dots ,x_q^{c_1}]=[x_p{’}^{c_2},\dots ,x_q^{\prime c_2}]$. In particular, these blocks that are retained in Parent 1 and Parent 2 are not necessarily in the same factory. An identical block in Parent 1 is processed in factory c1 and Parent 2 is processed in factory c2; Child 1 will retain the block in the same location in factory c1 and Child 2 will retain the block in the same location in factory c2.

Step 2: Two cut points are randomly selected in the scheduling sequence of each factory of the two parents, and the vector of each factory of Child 1 retains all the intercepts of cut point 1 and cut point 2 of Parent 1 and maintains their original positions. Child 2 is generated from Parent 2 using the same principle.

Step 3: In the final step, the missing element will be copied in the relative order of the other parent. In this step, the exchange of parental information is efficiently accomplished and the reassignment of jobs between factories is realized.

Nevertheless, relying exclusively on crossover operations proves inadequate. To enhance population diversity, a random swap mutation (RSM) mechanism is additionally implemented.

RSM: Randomly select two jobs in the sequence and swap their positions.

An illustration of the crossover process is shown in Figure 3.

3.5. Problem-Specific Neighborhood Structures

To address the DHFSP, the designed neighborhood structures generate alternative feasible solutions by modifying the current ones. The effectiveness of these structures has a significant influence on both the solution quality and the computational efficiency of the algorithm. Well-designed neighborhoods can guide the search process toward more promising regions. Hence, in order to improve the performance of IQLMA, six tailored neighborhood structures are proposed to produce superior solutions.

NS1: Randomly select a job from the factory with the largest maximum completion time and insert the jobs into another position of the same factory.

NS2: Randomly select a job from the factory with the largest maximum completion time and exchange the job with another job in the same factory.

NS3: Randomly select a tardy job that does not have the largest tardiness, and exchange that job with all other jobs that have a greater tardiness.

NS4: Randomly select a tardy job and insert it in the more forward position of all the factory positions.

NS5: Randomly select a job in a factory with the largest total tardiness and insert the job into other locations in the same factory.

NS6: Randomly select a tardy job in a factory with the largest number of tardy jobs and insert the job into other locations in the same factory.

3.6. Variable Neighborhood Search with Q-Learning

Q-learning operates by maintaining and updating a state–action value table (Q-table), which stores estimated cumulative rewards to derive an optimal policy [31]. As a model-free algorithm, it has been employed to tackle a range of scheduling problems [23,32]. Figure 4 illustrates this interaction process. The Q-table is updated according to the following formula:

$$Q(s,a)=Q(s,a)+\alpha \left(R+\gamma maxQ\left(s^{\prime },a^{\prime }\right)-Q(s,a)\right)$$

(19)

$\alpha$ and $\gamma$ are the learning rate and discount factor, respectively.

The agent selects actions according to Q-values stored in the Q-table. This work initializes the Q-table with zeros, as shown in Figure 5, to indicate that the agent co-initializes the Q-table with zeros. Actions are selected using an $ϵ$-greedy strategy that maximizes the expected reward while maintaining exploration. A random number p ∈ [0, 1] determines the selection: if p < $ϵ$, the action with the maximum Q value is selected (exploitation); otherwise, a random action is chosen (exploration). The Q-table is iteratively updated through this action–state–reward cycle. An example of a Q-table update is provided in Appendix A.3.

Q-learning, when integrated with neighborhood search, plays the role of adaptively selecting the most promising neighborhood structures based on feedback from the search performance, rather than relying on a fixed or predetermined local search method. This reinforcement learning mechanism allows the search process to dynamically focus on neighborhoods that are more likely to yield improvements, thereby enhancing both convergence speed and solution quality. In this study, the six neighborhood structures described earlier are defined as the action set in the Q-learning framework, where each action corresponds to applying one specific neighborhood to generate new solutions.

In this paper, the state is determined according to the three objective values of each solution. First, the current population is sorted in ascending order of makespan. The top 30% of solutions with the smallest makespan are assigned to State 1. The remaining solutions are then sorted in ascending order of total tardiness, and the top 30% of these solutions are assigned to State 2. The remaining solutions are assigned to State 3.

This study employs a delayed reward mechanism based on population cooperation to train the Q-learning strategy. In each generation, all individuals conduct local search following the current strategy. Upon completion of the local search for the entire population, non-dominated sorting is applied to obtain an updated Pareto front. After obtaining the updated Pareto front via non-dominated sorting, we compute the minimum Euclidean distance from every solution to this front both before and after its local search. This signal is utilized to update the shared Q-learning strategy. Let x be the original solution to be replaced, and y be the candidate solution generated by the local search. $A_{\mathrm{post}}$ is the non-dominated solution set obtained from the population after applying the variable neighborhood local search, inserting the new solution, and removing dominated solutions. The Euclidean distance from a solution z to $A_{\mathrm{post}}$ is defined as

$$d(z,A_{\mathrm{post}})=\underset{a\in A_{\mathrm{post}}}{min}\sqrt{\sum _{i=1}^m{\left(f_i\left(z\right)-f_i\left(a\right)\right)}^2},$$

(20)

$f_i\left(z\right)$ denotes the objective value of z on the i-th objective, with m representing the total number of objectives. The reward $r_t$ is then given by

$$r_t=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}d(y,A_{\mathrm{post}})<d(x,A_{\mathrm{post}}),\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(21)

This binary reward provides a positive signal whenever the candidate solution is closer to the updated Pareto front than the original solution, thereby encouraging the selection of neighborhood structures that improve convergence towards the Pareto front.

The Q-learning-guided local search is illustrated in Algorithm 2.

Algorithm 2 Variable Neighborhood Search using Q-learning

Input:  $PO{P}_t$ (current population), $PopSiz{e}_t$ (population size), Q-table, $s_i$ (state of the i-th individual), $a_i$ (action taken by the i-th individual) Output:  $Po{p}_t^{\prime }$, Q-table   1: $S_1\dots S_{PopSize}$← Ranking the three objective values for all individuals in the population yields the state of each individual.   2: for each i in $PopSiz{e}_t$ do   3:      $a_i$← select an action via the $ϵ$-greedy strategy and Q-table   4:      $Po{p}_t\left(i\right)$←Apply action $a_i$ to the i-th individual ($Po{p}_t\left(i\right)$) in the population   5: end for   6: $Po{p}_t^{\prime }$← Combine $Po{p}_t^{\prime }\left(1\right)toPo{p}_t^{\prime }\left(PopSize\right)$;   7: $A_{post}$← Non-dominated sorting $\left(Po{p}_t^{\prime }andPo{p}_t\right)$   8: for each i in $PopSiz{e}_t$ do   9:      For each individual $Po{p}_t^{\prime }\left(i\right)$ and $Po{p}_t\left(i\right)$: 10:      for $\mathrm{individual}\in \{Po{p}_t^{\prime }\left(i\right),Po{p}_t\left(i\right)\}$ do 11:          $\mathrm{dist}\leftarrow \infty$ 12:          for each solution $\mathbf{x}\in A_{post}$ do 13:              $\mathrm{curr}\_\mathrm{dist}\leftarrow {\parallel \mathrm{individual}-\mathbf{x}\parallel }_2$ {Euclidean distance} 14:              if $\mathrm{curr}\_\mathrm{dist}<\mathrm{dist}$ then 15:                  $\mathrm{dist}\leftarrow \mathrm{curr}\_\mathrm{dist}$ 16:              end if 17:          end for 18:      end for 19:      Record the minimum distance for this individual 20:      Check if $d_{\mathrm{after}}<d_{\mathrm{before}}$ to evaluate the improvement. 21:      Obtain reward r and next state $s_i^{\prime }$ 22:      Update Q-table: $Q(s_i,a_i)\leftarrow Q(s_i,a_i)+\alpha [r+\gamma {max}_{a^{\prime }}Q(s_i^{\prime },a^{\prime })-Q(s_i,a_i)]$ 23: end for

3.7. Non-Dominated Order and Elitism Strategy

For fast non-dominated ordering of the merged populations, low rank solutions are preferred to high rank solutions. If two solutions have the same rank, the one with the greater crowding distance is selected over the one with the smaller value. The population of the next generation of a given size is selected according to the above principle. In the iterative process, the population of the next generation is merged with the population of the parent. The merged population is then subjected to a fast non-dominated ordering.

4. Experimental Comparison and Analysis

4.1. Experiment Setting

All experiments were conducted in MATLAB R2022b (64-bit) with the Optimization Toolbox and Global Optimization Toolbox enabled. The operating system was Windows 11 (Version 22H2, 64-bit). All computations were performed on a desktop computer equipped with an Intel Core i7-12700K.

This article generates instances by following the design described by Sun [33]. (F) makes up each instance, in which N = 50, 100, 150, 200, F = 2, 3, 4, 5, 6, and S = 2, 4, 6, 8, 10. The processing time of each job at each stage and the sequence-dependent setup time are randomly generated within the range [1, 99]. The number of identical parallel machines are randomly generated within the range [1, 5]. The random seed is set to 2025. In total, there are 4 × 5 × 5 = 100 instances. The CPU time per instance run is set to 0.08 × FN × JN × S s.

Equations (22) and (23) [11] establish the due date for each job in each instance in the mathematical model provided in this study.

$$\begin{array}{cc}\hfill D_j& =P_j\times (1+3\times rand(0,1))\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill P_j& =\sum _{i=1}^S{p}_{i,j}\hfill \end{array}$$

(23)

4.2. Experimental Indicators

To evaluate the behavior of IMOMA, two performance metrics were used in the experiments.

The hypervolume (HV) represents the volume of the region bounded by the non-dominated solution set produced by the algorithm in the objective space and the reference points. The reference point is set to (1.2, 1.2, 1.2). A higher HV value indicates better overall performance. HV is calculated as follows.

$$HV=\mathrm{T}\left(\bigcup _{i=1}^{PF}{v}_i\right)$$

(24)

The Lebesgue measure is defined by $\mathrm{T}(•)$, while $v_i$ denotes the hypervolume bounded by the reference point and the non-dominated solution set.

$$IGD={\displaystyle \frac{{\sum }_{x\in P{F}^{\ast }}d(x,z)}{\left|P{F}^{\ast }\right|}}$$

(25)

$d(x,z)$ represents the minimum Euclidean distance from an individual x in $P{F}^{\ast }$ to an individual z in PF. The notation $|P{F}^{\ast }|$ denotes the the number of solutions.

4.3. Parameter Calibration

This section calibrates the main parameters of the algorithm, including population size, discount factor, greedy rate, crossover probability, and mutation probability. The parameters were calibrated using the design of experiments method. An exhaustive analytic factorization experiment is carried out on each parameter. The levels of each parameter were $PS$ = {60, 80, 100}, $pc$ = {0.7, 0.8, 0.9}, $pm$ = {0.2, 0.4, 0.6}, $\gamma$ = {0.7, 0.8, 0.9}, $ϵ$ = {0.7, 0.8, 0.9}. The five key parameters were analyzed using the Taguchi method of experiment (DOE) using orthogonal array $L_{18}\left(3^7\right)$, which consists of 18 different combinations of $PS$, $pc$, $pm$.

Figure 6 drafts the main effects diagram of three parameters of IMOMA, so the optimal parameter configuration is identified as follows: $PS$ = 80, $pc$ = 0.7, $pm$ = 0.2, $\gamma$ = 0.8, $ϵ$ = 0.8.

To further investigate the influence of key parameters on the performance of the proposed IMOMA algorithm, an extended sensitivity analysis was conducted. This analysis adopts a one-factor-at-a-time (OFAT) approach to observe the individual effect of each parameter clearly. During the test of each target parameter, the remaining parameters were fixed at their empirically determined optimal baseline values. The five target parameters were tested across three levels each. The analysis was performed on three representative instances of different scales selected from the benchmark set: a small-scale instance (F = 2, n = 50, s = 2), a medium-scale instance (F = 4, n = 150, s = 4), and a large-scale instance (F = 6, n = 150, s = 6).

The experimental results are shown in Figure 7. Based on the extended sensitivity analysis, the mutation probability ($p_m$) exhibits the most pronounced influence on algorithm performance, where increased values consistently lead to degradation across all tested instances, firmly validating the optimal baseline setting of $p_m=0.2$. The population size ($PS$) shows moderate sensitivity, with its optimal value shifting slightly with the problem scale, though the baseline $PS=80$ remains robust. The Q-learning discount factor ($\gamma$) demonstrates a moderate level of sensitivity, performing optimally near its baseline of $\gamma =0.8$. In contrast, the crossover probability ($p_c$) and the Q-learning exploration rate ($ϵ$) exhibit relatively low sensitivity within the tested ranges, confirming the stability of their baseline settings ($p_c=0.7$, $ϵ=0.8$). The overall low-to-moderate sensitivity of the Q-learning hyperparameters ($\gamma$ and $ϵ$) indicates that the proposed Q-learning-guided local search module possesses good robustness, as its performance does not critically depend on their precise tuning. Overall, the results strongly justify the selected parameter configuration as a reliable default, while indicating that fine-tuning efforts for future applications should primarily focus on $p_m$ and $PS$, particularly when addressing problems of substantially different scales.

4.4. Effectiveness of Each Improvement Component of IMOMA-QL

To examine the effectiveness of each component, IMOMA-QL was compared with four variant versions in which a specific component was removed. These variants are IMOMA-QL without the hybrid initialization strategy (denoted as IMOMA-QL1), IMOMA-QL without the genetic operators (denoted as IMOMA-QL2), IMOMA-QL without the multi-neighborhood search (denoted as IMOMA-QL3), and IMOMA-QL without the Q-learning selection mechanism (denoted as IMOMA-QL4). Specifically, IMOMA-QL1 adopts random population initialization instead of the hybrid method; IMOMA-QL2 and IMOMA-QL3, respectively, remove the genetic operators and the multi-neighborhood search while retaining the rest of the algorithm; and IMOMA-QL4 replaces the Q-learning mechanism with a random local search. To ensure a fair comparison, all algorithmic parameters for these variants remained consistent with those of the original IMOMA-QL. All algorithms were executed independently 10 times on the test instances. The reference Pareto front was constructed from the combined non-dominated solutions of all compared algorithms.

The experimental results systematically evaluate performance across different problem scales by grouping the 100 instances according to the number of factories F, jobs n, and stages s. The average HV and IGD values are presented in

Table 2. HV of the proposed algorithm and its four variants.

	Instances	IMOMA-QL	IMOMA-QL1	IMOMA-QL2	IMOMA-QL3	IMOMA-QL4
F	2	1.21	1.13	0.95	1.02	1.11
	3	1.19	1.12	0.98	0.99	1.08
	4	1.17	1.10	0.94	1.03	1.11
	5	1.23	1.15	0.97	1.05	1.13
	6	1.22	1.13	0.95	1.04	1.11
n	50	1.21	1.13	0.96	1.03	1.13
	100	1.18	1.10	0.93	1.01	1.10
	150	1.22	1.12	0.96	1.04	1.09
	200	1.20	1.13	0.97	1.03	1.12
s	2	1.24	1.13	0.97	1.03	1.12
	4	1.21	1.14	0.98	1.05	1.13
	6	1.19	1.10	0.92	1.00	1.08
	8	1.19	1.12	0.96	1.03	1.11
	10	1.18	1.13	0.95	1.02	1.10

 and 

Table 3. IGD of the proposed algorithm and and its four variants.

	Instances	IMOMA-QL	IMOMA-QL1	IMOMA-QL2	IMOMA-QL3	IMOMA-QL4
F	2	0.105	0.130	0.176	0.167	0.128
	3	0.083	0.115	0.191	0.173	0.120
	4	0.102	0.134	0.181	0.161	0.120
	5	0.098	0.124	0.166	0.151	0.117
	6	0.102	0.114	0.177	0.170	0.114
n	50	0.103	0.132	0.182	0.169	0.125
	100	0.085	0.108	0.162	0.147	0.102
	150	0.107	0.131	0.181	0.168	0.124
	200	0.108	0.122	0.188	0.173	0.128
s	2	0.120	0.143	0.204	0.188	0.144
	4	0.111	0.134	0.189	0.180	0.132
	6	0.082	0.108	0.158	0.145	0.103
	8	0.082	0.112	0.162	0.143	0.100
	10	0.096	0.120	0.180	0.166	0.120

. The complete experimental data for each instance are presented in Appendix A,

Table A1. HV of the proposed algorithm and its four variants (F = 2).

			F = 2
(n, s)	IMOMA-QL	IMOMA-QL1	IMOMA-QL2	IMOMA-QL3	IMOMA-QL4
(50, 2)	1.20	1.10	0.89	0.96	1.08
(50, 4)	1.24	1.08	1.00	1.05	1.15
(50, 6)	1.24	1.15	0.97	1.02	1.15
(50, 8)	1.16	1.13	0.92	1.03	1.08
(50, 10)	1.18	1.10	0.88	0.95	1.07
(100, 2)	1.26	1.16	1.00	1.01	1.15
(100, 4)	1.17	1.11	0.94	0.96	1.08
(100, 6)	1.18	1.11	0.85	0.94	1.07
(100, 8)	1.21	1.14	0.96	1.02	1.14
(100, 10)	1.16	1.07	0.88	0.98	1.06
(150, 2)	1.25	1.09	0.96	1.01	1.11
(150, 4)	1.27	1.19	1.03	1.09	1.14
(150, 6)	1.20	1.09	0.89	1.03	1.06
(150, 8)	1.22	1.13	0.95	1.09	1.10
(150, 10)	1.25	1.20	1.00	1.10	1.12
(200, 2)	1.26	1.17	0.98	1.03	1.16
(200, 4)	1.25	1.22	0.99	1.09	1.16
(200, 6)	1.17	1.10	0.93	0.99	1.08
(200, 8)	1.17	1.08	0.97	1.05	1.09
(200, 10)	1.18	1.13	0.92	1.02	1.11

,

Table A2. HV of the proposed algorithm and its four variants (F = 3).

			F = 3
(n, s)	IMOMA-QL	IMOMA-QL1	IMOMA-QL2	IMOMA-QL3	IMOMA-QL4
(50, 2)	1.24	1.14	1.02	1.03	1.13
(50, 4)	1.19	1.16	0.98	0.97	1.10
(50, 6)	1.13	1.07	0.91	0.94	1.02
(50, 8)	1.17	1.11	1.02	0.97	1.11
(50, 10)	1.13	1.11	0.96	0.94	1.05
(100, 2)	1.19	1.05	0.96	0.99	1.06
(100, 4)	1.16	1.10	0.99	0.98	1.08
(100, 6)	1.13	1.06	0.86	0.94	1.02
(100, 8)	1.18	1.14	0.96	0.97	1.12
(100, 10)	1.20	1.17	1.00	1.01	1.12
(150, 2)	1.29	1.18	1.04	1.04	1.11
(150, 4)	1.15	1.10	0.94	0.96	1.02
(150, 6)	1.25	1.16	0.99	1.00	1.08
(150, 8)	1.21	1.11	1.05	1.06	1.07
(150, 10)	1.23	1.18	1.03	1.03	1.11
(200, 2)	1.17	1.08	0.94	0.95	1.04
(200, 4)	1.17	1.13	1.00	0.97	1.07
(200, 6)	1.18	1.08	0.92	0.98	1.06
(200, 8)	1.15	1.08	0.95	0.95	1.07
(200, 10)	1.21	1.13	1.04	1.05	1.14

,

Table A3. HV of the proposed algorithm and its four variants (F = 4).

			F = 4
(n, s)	IMOMA-QL	IMOMA-QL1	IMOMA-QL2	IMOMA-QL3	IMOMA-QL4
(50, 2)	1.29	1.17	1.05	1.09	1.22
(50, 4)	1.20	1.13	1.03	1.09	1.18
(50, 6)	1.15	1.04	0.89	1.00	1.08
(50, 8)	1.18	1.16	0.97	1.06	1.15
(50, 10)	1.13	1.07	0.89	0.98	1.11
(100, 2)	1.19	1.07	0.92	1.01	1.10
(100, 4)	1.13	1.09	0.94	1.02	1.11
(100, 6)	1.11	1.00	0.88	0.97	1.06
(100, 8)	1.12	1.01	0.93	0.99	1.06
(100, 10)	1.17	1.13	0.95	1.08	1.13
(150, 2)	1.20	1.13	0.92	1.05	1.07
(150, 4)	1.17	1.07	0.91	1.06	1.10
(150, 6)	1.24	1.12	0.96	1.09	1.06
(150, 8)	1.13	1.03	0.90	0.94	1.04
(150, 10)	1.21	1.12	0.94	1.07	1.13
(200, 2)	1.22	1.12	0.96	1.09	1.15
(200, 4)	1.14	1.10	0.96	1.05	1.10
(200, 6)	1.13	1.05	0.93	0.96	1.09
(200, 8)	1.16	1.14	0.96	1.00	1.13
(200, 10)	1.21	1.12	1.02	1.09	1.15

,

Table A4. HV of the proposed algorithm and its four variants (F = 5).

			F = 5
(n, s)	IMOMA-QL	IMOMA-QL1	IMOMA-QL2	IMOMA-QL3	IMOMA-QL4
(50, 2)	1.33	1.25	1.01	1.12	1.20
(50, 4)	1.23	1.16	0.98	1.04	1.17
(50, 6)	1.25	1.19	0.96	1.10	1.14
(50, 8)	1.19	1.14	0.98	1.02	1.10
(50, 10)	1.25	1.17	1.03	1.04	1.17
(100, 2)	1.23	1.13	0.94	1.02	1.14
(100, 4)	1.27	1.18	1.00	1.11	1.14
(100, 6)	1.19	1.11	0.93	1.00	1.09
(100, 8)	1.25	1.15	0.98	1.04	1.15
(100, 10)	1.14	1.07	0.83	1.08	1.06
(150, 2)	1.23	1.07	0.93	0.97	1.07
(150, 4)	1.26	1.16	1.01	1.08	1.14
(150, 6)	1.24	1.14	0.97	1.03	1.08
(150, 8)	1.25	1.15	0.99	1.11	1.14
(150, 10)	1.16	1.06	0.89	1.02	1.03
(200, 2)	1.24	1.13	0.98	1.04	1.11
(200, 4)	1.26	1.20	0.97	1.12	1.21
(200, 6)	1.26	1.19	1.00	1.09	1.16
(200, 8)	1.24	1.16	1.00	1.10	1.18
(200, 10)	1.20	1.15	0.95	1.04	1.14

,

Table A5. HV of the proposed algorithm and its four variants (F = 6).

			F = 6
(n,s)	IMOMA-QL	IMOMA-QL1	IMOMA-QL2	IMOMA-QL3	IMOMA-QL4
(50, 2)	1.28	1.23	1.03	1.11	1.23
(50, 4)	1.25	1.15	1.00	1.13	1.18
(50, 6)	1.16	1.06	0.87	1.01	1.05
(50, 8)	1.21	1.19	0.94	1.08	1.15
(50, 10)	1.17	1.12	0.92	1.03	1.11
(100, 2)	1.32	1.18	1.02	1.09	1.19
(100, 4)	1.19	1.08	0.93	1.03	1.10
(100, 6)	1.14	1.05	0.85	0.96	1.03
(100, 8)	1.23	1.17	1.01	1.07	1.16
(100, 10)	1.12	1.07	0.87	1.00	1.07
(150, 2)	1.22	1.09	0.92	0.99	1.06
(150, 4)	1.30	1.20	1.05	1.12	1.16
(150, 6)	1.25	1.12	0.92	1.06	1.10
(150, 8)	1.18	1.10	0.90	1.00	1.07
(150, 10)	1.21	1.12	0.92	1.04	1.09
(200, 2)	1.20	1.08	0.93	1.01	1.10
(200, 4)	1.25	1.21	1.00	1.12	1.20
(200, 6)	1.16	1.09	0.90	0.96	1.07
(200, 8)	1.25	1.06	1.04	1.07	1.10
(200, 10)	1.24	1.16	1.00	1.01	1.06

,

Table A6. IGD of the proposed algorithm and its four variants (F = 2).

			F = 2
(n, s)	IMOMA-QL	IMOMA-QL1	IMOMA-QL2	IMOMA-QL3	IMOMA-QL4
(50, 2)	0.116	0.128	0.153	0.157	0.121
(50, 4)	0.110	0.133	0.160	0.157	0.127
(50, 6)	0.073	0.099	0.153	0.152	0.103
(50, 8)	0.060	0.099	0.127	0.093	0.065
(50, 10)	0.129	0.134	0.143	0.152	0.148
(100, 2)	0.138	0.152	0.177	0.182	0.165
(100, 4)	0.121	0.134	0.165	0.159	0.129
(100, 6)	0.069	0.081	0.117	0.106	0.072
(100, 8)	0.090	0.103	0.160	0.140	0.095
(100, 10)	0.100	0.124	0.182	0.143	0.137
(150, 2)	0.123	0.176	0.193	0.202	0.173
(150, 4)	0.132	0.148	0.176	0.182	0.159
(150, 6)	0.087	0.123	0.136	0.158	0.115
(150, 8)	0.093	0.135	0.134	0.129	0.127
(150, 10)	0.102	0.123	0.151	0.143	0.129
(200, 2)	0.132	0.137	0.161	0.167	0.148
(200, 4)	0.077	0.084	0.152	0.143	0.097
(200, 6)	0.105	0.122	0.163	0.159	0.123
(200, 8)	0.083	0.103	0.122	0.128	0.111
(200, 10)	0.117	0.132	0.167	0.152	0.142

,

Table A7. IGD of the proposed algorithm and its four variants (F = 3).

			F = 3
(n, s)	IMOMA-QL	IMOMA-QL1	IMOMA-QL2	IMOMA-QL3	IMOMA-QL4
(50, 2)	0.110	0.131	0.177	0.182	0.155
(50, 4)	0.108	0.144	0.169	0.162	0.132
(50, 6)	0.070	0.095	0.144	0.133	0.093
(50, 8)	0.094	0.140	0.161	0.156	0.121
(50, 10)	0.108	0.140	0.178	0.168	0.136
(100, 2)	0.082	0.107	0.157	0.165	0.130
(100, 4)	0.103	0.130	0.177	0.181	0.151
(100, 6)	0.072	0.098	0.161	0.149	0.098
(100, 8)	0.064	0.071	0.103	0.093	0.078
(100, 10)	0.071	0.078	0.138	0.141	0.075
(150, 2)	0.123	0.155	0.175	0.181	0.150
(150, 4)	0.083	0.119	0.154	0.157	0.110
(150, 6)	0.067	0.114	0.176	0.150	0.112
(150, 8)	0.050	0.087	0.149	0.123	0.107
(150, 10)	0.109	0.132	0.145	0.162	0.142
(200, 2)	0.117	0.136	0.156	0.165	0.168
(200, 4)	0.115	0.132	0.161	0.153	0.161
(200, 6)	0.076	0.102	0.187	0.151	0.113
(200, 8)	0.050	0.075	0.121	0.134	0.100
(200, 10)	0.062	0.090	0.165	0.166	0.083

,

Table A8. IGD of the proposed algorithm and its four variants (F = 4).

			F = 4
(n, s)	IMOMA-QL	IMOMA-QL1	IMOMA-QL2	IMOMA-QL3	IMOMA-QL4
(50, 2)	0.122	0.157	0.176	0.165	0.131
(50, 4)	0.115	0.153	0.168	0.170	0.120
(50, 6)	0.101	0.137	0.186	0.150	0.110
(50, 8)	0.107	0.149	0.163	0.139	0.130
(50, 10)	0.082	0.122	0.163	0.140	0.085
(100, 2)	0.097	0.130	0.176	0.154	0.131
(100, 4)	0.107	0.157	0.178	0.157	0.108
(100, 6)	0.093	0.138	0.181	0.127	0.112
(100, 8)	0.053	0.086	0.118	0.129	0.070
(100, 10)	0.069	0.091	0.106	0.139	0.080
(150, 2)	0.124	0.136	0.166	0.142	0.135
(150, 4)	0.105	0.127	0.143	0.140	0.131
(150, 6)	0.060	0.103	0.124	0.091	0.099
(150, 8)	0.097	0.145	0.137	0.139	0.125
(150, 10)	0.114	0.142	0.176	0.179	0.130
(200, 2)	0.107	0.133	0.169	0.155	0.121
(200, 4)	0.130	0.146	0.163	0.171	0.153
(200, 6)	0.101	0.125	0.167	0.152	0.132
(200, 8)	0.116	0.138	0.171	0.164	0.147
(200, 10)	0.105	0.122	0.150	0.142	0.108

,

Table A9. IGD of the proposed algorithm and its four variants (F = 5).

			F = 5
(n, s)	IMOMA-QL	IMOMA-QL1	IMOMA-QL2	IMOMA-QL3	IMOMA-QL4
(50, 2)	0.102	0.134	0.151	0.141	0.113
(50, 4)	0.105	0.126	0.165	0.138	0.127
(50, 6)	0.060	0.089	0.134	0.120	0.073
(50, 8)	0.123	0.159	0.180	0.169	0.154
(50, 10)	0.099	0.137	0.137	0.124	0.125
(100, 2)	0.082	0.114	0.106	0.113	0.105
(100, 4)	0.079	0.101	0.140	0.147	0.084
(100, 6)	0.088	0.113	0.156	0.146	0.090
(100, 8)	0.070	0.083	0.105	0.101	0.075
(100, 10)	0.061	0.098	0.124	0.128	0.093
(150, 2)	0.134	0.172	0.160	0.196	0.153
(150, 4)	0.099	0.114	0.162	0.157	0.118
(150, 6)	0.115	0.154	0.174	0.156	0.125
(150, 8)	0.111	0.144	0.150	0.133	0.118
(150, 10)	0.099	0.121	0.154	0.137	0.134
(200, 2)	0.140	0.141	0.178	0.178	0.152
(200, 4)	0.123	0.148	0.181	0.170	0.147
(200, 6)	0.073	0.089	0.121	0.098	0.089
(200, 8)	0.076	0.097	0.129	0.111	0.109
(200, 10)	0.127	0.149	0.168	0.158	0.152

and

Table A10. IGD of the proposed algorithm and its four variants (F = 6).

			F = 6
(n,s)	IMOMA-QL	IMOMA-QL1	IMOMA-QL2	IMOMA-QL3	IMOMA-QL4
(50, 2)	0.155	0.160	0.178	0.176	0.167
(50, 4)	0.119	0.135	0.172	0.163	0.139
(50, 6)	0.088	0.103	0.167	0.149	0.119
(50, 8)	0.096	0.119	0.118	0.153	0.125
(50, 10)	0.104	0.120	0.158	0.168	0.136
(100, 2)	0.096	0.107	0.151	0.141	0.114
(100, 4)	0.137	0.147	0.182	0.211	0.163
(100, 6)	0.075	0.082	0.148	0.109	0.077
(100, 8)	0.05	0.072	0.085	0.110	0.052
(100, 10)	0.061	0.063	0.130	0.105	0.066
(150, 2)	0.115	0.128	0.126	0.157	0.120
(150, 4)	0.118	0.122	0.185	0.188	0.132
(150, 6)	0.058	0.066	0.136	0.128	0.074
(150, 8)	0.100	0.121	0.164	0.148	0.011
(150, 10)	0.11	0.108	0.152	0.177	0.141
(200, 2)	0.130	0.135	0.172	0.183	0.159
(200, 4)	0.129	0.140	0.186	0.179	0.137
(200, 6)	0.100	0.118	0.161	0.135	0.126
(200, 8)	0.098	0.102	0.155	0.170	0.101
(200, 10)	0.110	0.125	0.182	0.182	0.125

. IMOMA-QL consistently achieved the best performance across all instances.

Figure 8 and Figure 9 display interval plots with 95% confidence intervals for HV and IGD metrics across all instances, comparing IMOMA-QL with its four variants. The results demonstrate that each component of IMOMA-QL contributes to performance improvements at varying degrees.

The Friedman test rankings with 95% confidence intervals are presented in

Table 4. Friedman test rankings across all algorithm variants.

Alogrithms	HV		IGD
	Rank	p-Value	Rank	p-Value
IMOMA-QL	1		1.000	0
IMOMA-QL1	2.32	3.56476 × 10−9	2.56	0
IMOMA-QL2	4.93	0	4.750	0
IMOMA-QL3	4.05	0	4.180	0
IMOMA-QL4	2.70	2.90878 × 10−14	2.48	5.68936 × 10−11

. IMOMA-QL achieved first-place rankings against all four variants with statistically significant improvements (all p-values < 0.05), demonstrating that each proposed enhancement contributes substantially to its superior performance. The results indicate that removing any of these components leads to a clear performance drop, demonstrating the necessity of each component.

4.5. Comparison of IMOMA and Other Algorithms

We select four comparative algorithms: IMPGA [34], MQSFLA [25], MOEA/D [35], and NSGA-II [36]. The first two are recent algorithms specifically designed for DHFSP, which is the same problem domain studied in this work. IMPGA minimizes both makespan and total tardiness—two of the three objectives optimized in our paper—and is composed of multiple populations that co-evolve in sub-regions, a greedy inter-factory job insertion neighborhood structure for local search, and a probability-sampling-based re-initialization procedure. MQSFLA also targets the minimization of makespan and total tardiness, sharing a highly similar objective set with our work. It incorporates a memeplex quality measurement mechanism, a search process guided by solution quality, and a novel memeplex shuffling that dynamically selects memeplexes based on evolution quality. The latter two, NSGA-II and MOEA/D, are classic multi-objective optimizers widely used in scheduling problems. NSGA-II employs non-dominated sorting and crowding distance to preserve solution diversity; in contrast, MOEA/D converts the multi-objective problem into a set of scalar subproblems. For IMPGA and MQSFLA, we directly adopt the parameter settings reported in their respective original papers. For NSGA-II and MOEA/D, we set the population size to 50, crossover probability pc to 0.7, and mutation probability pm to 0.2. Together, these algorithms provide diverse and strong benchmarks for evaluating our proposed method.

Each comparison algorithm was performed independently 10 times on each instance. The average HV and IGD values are presented in

Table 5. HV of the proposed algorithm and the comparison algorithms.

	Instances	IMOMA-QL	IMPGA	MQSLFA	MOEA/D	NSGA-II
F	2	1.25	0.95	0.94	0.64	0.75
	3	1.22	0.92	0.92	0.65	0.74
	4	1.21	0.91	0.87	0.61	0.73
	5	1.23	0.92	0.88	0.60	0.72
	6	1.20	0.88	0.90	0.64	0.75
n	50	1.22	0.93	0.90	0.69	0.73
	100	1.19	0.90	0.91	0.61	0.72
	150	1.24	0.90	0.91	0.63	0.74
s	2	1.19	0.93	0.97	0.69	0.78
	4	1.22	0.94	0.87	0.62	0.72
	6	1.25	0.93	0.91	0.66	0.75
	8	1.19	0.93	0.85	0.65	0.75
	10	1.21	0.86	0.92	0.62	0.69

 and 

Table 6. IGD of the proposed algorithm and comparison algorithms.

	Instances	IMOMA-QL	IMIPGA	MQSLFA	MOEA/D	NSGA-II
F	2	0.105	0.157	0.167	0.263	0.222
	3	0.086	0.144	0.145	0.247	0.201
	4	0.081	0.149	0.157	0.254	0.199
	5	0.105	0.161	0.177	0.268	0.224
	6	0.092	0.155	0.160	0.247	0.197
n	50	0.108	0.162	0.169	0.244	0.214
	100	0.083	0.144	0.149	0.259	0.200
	150	0.099	0.157	0.170	0.256	0.206
	200	0.085	0.150	0.157	0.266	0.212
s	2	0.085	0.147	0.156	0.251	0.199
	4	0.095	0.152	0.168	0.265	0.219
	6	0.099	0.157	0.159	0.259	0.212
	8	0.086	0.149	0.160	0.256	0.200
	10	0.103	0.161	0.162	0.249	0.214

. The complete experimental data for each instance are presented in Appendix A, Table A1, Table A2, Table A3, Table A4, Table A5, Table A6, Table A7, Table A8, Table A9 and Table A10. IMOMA-QL consistently achieved the best performance across all instances.

Figure 10 and Figure 11 display interval plots with 95% confidence intervals for HV and IGD metrics across all instances, comparing IMOMA-QL with its comparative algorithms. As evidenced by the figure, IMOMA-QL outperforms all other algorithms, yielding superior results in both HV (higher) and IGD (lower) metrics. To validate the superior performance of the proposed algorithm, the Friedman test rankings with 95% confidence intervals are presented in

Table 7. Friedman test rankings across IMOMA-QL and comparative algorithms.

	HV		IGD
	Rank	p-Value	Rank	p-Value
IMOMA-QL	1		1
IMPGA	2.420	1.60385 × 10−10	2.35	1.56633 × 10−9
MQSLFA	2.610	6.01519 × 10−13	2.8	8.88178 × 10−16
MOEAD	4.860	0	4.72	0
NSGA2	4.100	0	4.13	0

. It can be seen that IMOMA-QL outperforms the other comparison algorithms.

Figure 12 displays the 3D scatter plots representing the non-dominated set obtained from five distinct methods applied to one of the problems. We found that the non-dominated solution sets obtained by the respective algorithms form different layers. The solutions obtained by IMOMA are close to the point at which all target values are lowest. It was clearly found that the non-dominated set obtained by IMOMA-QL is better than the non-dominated set obtained by the other algorithms.

The superior performance of IMOMA-QL can be attributed to four key algorithmic innovations: Specifically, the hybrid initialization strategy improves the diversity and quality of the initial population, providing a better starting point for a search. The genetic operators enhance global exploration and recombination of high-quality solutions. The multi-neighborhood search mechanism diversifies local search patterns, improving the chance of escaping local optima. Finally, the Q-learning-guided variable neighborhood search adaptively selects promising neighborhoods based on search feedback, further enhancing search efficiency.

5. Conclusions

In the field of multi-objective optimization for tardiness-related scheduling problems, most existing studies focus on optimizing makespan along with only one tardiness-related objective. This study addresses DHFSP with SDST, optimizing three critical objectives: makespan, total tardiness, and the number of tardy jobs. This work emphasizes tardiness-related objectives, which are crucial in real-world manufacturing scenarios where meeting the due date is essential—such as just-in-time production, order-driven manufacturing, and supply chain scheduling with strict delivery commitments. To solve this problem, we propose a multi-objective memetic algorithm enhanced with Q-learning-guided variable neighborhood search (VNS). Extensive numerical experiments and comparisons with algorithms demonstrate that the proposed method significantly improves solution quality, convergence speed, and robustness in handling multi-objective DHFSP.

Despite the promising results, this study has certain limitations. The proposed model and algorithm operate under a set of standardized assumptions, such as deterministic processing times and static job availability. Consequently, they cannot be directly applied to real-world scheduling environments.

Future research can extend this work in several meaningful directions. A promising avenue is to investigate more comprehensive and environmentally conscious multi-objectives. For example, meaningful problem variations to explore could include minimizing makespan, total tardiness, and total energy consumption, or alternatively, minimizing makespan and maximizing tardiness and total energy consumption. Addressing such integrated problems would require developing new models that capture the energy dynamics of machines and designing efficient algorithms capable of balancing productivity.