LLM-Assisted Reinforcement Learning for U-Shaped and Circular Hybrid Disassembly Line Balancing in IoT-Enabled Smart Manufacturing

Abstract

With the sharp increase in the number of products and the development of the remanufacturing industry, disassembly lines have become the mainstream recycling method. In view of the insufficient research on the layout of multi-form disassembly lines and human factors, we previously proposed a linear-U-shaped hybrid layout considering the constraints of employee posture and a Duel-DQN algorithm assisted by Large Language Model (LLM). However, there is still room for improvement in the utilization efficiency of workstations. Based on this previous work, this study proposes an innovative layout of U-shaped and circular disassembly lines and retains the constraints of employee posture. The LLM is instruction-fine-tuned using the Quantized Low-Rank Adaptation (QLoRA) technique to improve the accuracy of disassembly sequence generation, and the Dueling Deep Q-Network(Duel-DQN) algorithm is reconstructed to maximize profits under posture constraints. Experiments show that in the more complex layout of U-shaped and circular disassembly lines, the iterative efficiency of this method can still be increased by about 26% compared with the traditional Duel-DQN, and the profit is close to the optimal solution of the traditional CPLEX solver, verifying the feasibility of this algorithm in complex scenarios. This study further optimizes the layout problem of multi-form disassembly lines and provides an innovative solution that takes into account both human factors and computational efficiency, which has important theoretical and practical significance.

1. Introduction

Due to the rapid advancement of technology, new products are continuously emerging on the market. At the same time, End-of-Life (EOL) products are steadily accumulating, and these products are increasingly recycled and reused. The Disassembly Line Balancing Problem (DLBP) has emerged in response to this context. Gungor and Gupta [1] conducted the first scientific research on the DLBP in 2004. Since then, many scholars have examined the DLBP for a variety of disassembly line layouts and proposed corresponding solution approaches. Notably, Altekin [2] proposed the linear disassembly line, while Agrawal [3] introduced the U-shaped disassembly line. The linear disassembly line is a cost-effective option and relatively easy to construct, whereas the U-shaped disassembly line utilizes available space more efficiently. Additionally, the parallel disassembly line improves the utilization of individual workstations, and the bilateral disassembly line serves as an extension of the linear layout. In practical applications, both the linear disassembly line [4] and the U-shaped disassembly line [3] are widely adopted. Meanwhile, scholars have also conducted research on related layouts with a profit-oriented approach [5,6].

Due to differences in product quality, service life, and other variable factors, the recycling of End-of-Life products (ELOPs) involves considerable uncertainty. Each EOL product may require specific treatment or additional care during the disassembly process. Consequently, human participation in disassembly activities is of vital importance. Tang, Zhou, and other researchers [7] pointed out that if human factors are not considered during disassembly system design, more complex and uncertain problems may arise. Kara et al. [8] suggested that incorporating ergonomic factors could better address the structural variability of EOL products and the uncertainties introduced by human intervention. One particularly important human factor is the posture of workers during disassembly tasks. Requiring workers to use a fixed posture throughout operations can significantly reduce overall disassembly efficiency [9]. Therefore, intelligent disassembly system design increasingly emphasizes human-centric modeling and optimization.

Meanwhile, the rapid integration of Internet of Things (IoT) technologies into manufacturing environments introduces additional complexities. IoT-enabled disassembly lines can leverage connected sensors and devices for real-time data acquisition, monitoring, and control, which can improve the efficiency and adaptability of the disassembly process [10]. However, this integration also introduces challenges related to cybersecurity and data privacy [11]. Thus, there is a growing need for solutions that simultaneously address operational optimization, human factors, and cybersecurity challenges.

With respect to solution approaches, many scholars have also attempted to use different approaches for solving problems in various environments [12,13,14]. In recent years, reinforcement learning algorithms have demonstrated strong potential in solving large-scale, complex optimization problems in disassembly contexts [15]. In particular, integrating reinforcement learning with emerging techniques such as Large Language Models (LLMs) offers new opportunities for improving solution quality and efficiency. Based on the above research context, our previous work proposed a hybrid disassembly line balancing method combining LLM-assisted disassembly sequence generation and a Duel-DQN algorithm to enhance efficiency [16]. However, there remains room for improvement in workstation utilization and adaptability to more complex layouts.

Therefore, this paper proposes a novel U-shaped and circular hybrid disassembly line layout (UC-HDLBP) to further optimize workstation utilization while considering worker posture constraints. A new solution framework is introduced, wherein an instruction-fine-tuned LLM using Quantized Low-Rank Adaptation (QLoRA) improves disassembly sequence generation, and a redesigned Duel-DQN optimizes task allocation to maximize overall profits. Furthermore, the proposed LLM-assisted framework is designed with extensibility in mind, offering a pathway to integrate cybersecurity and privacy-preserving mechanisms into IoT-enabled smart remanufacturing environments.

The contributions of this work are summarized as follows:

• We propose a novel hybrid disassembly system combining U-shaped and circular disassembly line layouts, considering worker posture constraints, and formulate a profit-maximization mathematical model.
• We design a QLoRA-based instruction-fine-tuning approach for LLMs and redesign the Duel-DQN algorithm to efficiently solve the complex UC-HDLBP.
• We demonstrate that the framework can be extended to address cybersecurity, privacy, and trust challenges in IoT-enabled disassembly environments, making it suitable for intelligent, secure manufacturing applications.

The remainder of this paper is organized as follows: Section 2 describes the problem formulation. Section 3 presents the methodology. Section 4 reports and analyzes experimental results. Section 5 discusses cybersecurity and privacy extensions. Finally, Section 6 concludes the paper and outlines future research directions.

2. Problem Description

2.1. U-Shaped and Circular Disassembly Line Balance Problem

The layouts of hybrid disassembly lines are diverse, and the choice of layout can significantly affect operational efficiency, resource utilization, and worker performance. To further improve workstation utilization beyond traditional designs, we propose the U-Shaped and Circular Hybrid Disassembly Line Balancing Problem (UC-HDLBP). As shown in Figure 1, the system consists of two disassembly lines: Line 1 adopts a U-shaped configuration, and Line 2 adopts a circular configuration. When a product enters the disassembly lines, workstations on the U-shaped disassembly line have two decision opportunities to perform disassembly tasks due to the ability to work on both sides, while the circular disassembly line can be regarded as a linear disassembly line with connected ends, enabling workstations within it to have multiple decision opportunities to perform disassembly tasks. The two disassembly lines can be activated separately or simultaneously. This layout allows for more flexible product allocation arrangements. The final optimization objective is to determine the optimal product allocation, disassembly sequence, and task allocation across these production lines to maximize total profit.

Given the increasing adoption of smart remanufacturing systems equipped with IoT sensors and real-time monitoring, modern disassembly lines must also address challenges related to cybersecurity, data privacy, and operational trust. In particular, sensitive operational data and worker-related information collected during disassembly must be safeguarded. Consequently, disassembly task planning must not only optimize operational objectives but also consider data sensitivity, privacy protection, and secure task execution—particularly in environments where IoT devices are embedded throughout the production process.

In this context, our UC-HDLBP formulation explicitly considers human factors by imposing posture constraints on workers. We classify workstations into two types: standing-type and sitting-type. Workers at each workstation must use the corresponding posture to perform tasks, reducing fatigue and improving disassembly efficiency. Each disassembly task for a product is associated with an optimal required posture, and task allocations must comply with these requirements. For example, if the optimal disassembly posture for a task is standing, the task can only be performed by workers at standing-type workstations.

2.2. Mathematical Model

The part compositions of each product may differ, and thus, when disassembling different products, the disassembly steps must conform to the structure of the product, meaning that the disassembly must satisfy the precedence and conflict relationships between disassembly tasks of the product. Such relationships can be represented by an AND/OR graph. Figure 2 shows an AND/OR graph of a washing machine, where <1> represents Component 1, which is composed of basic parts No. 1 to No. 6, and arcs 1–13 represent 13 disassembly tasks. Considering these relationships between disassembly tasks, we formulate a mixed-integer linear programming model based on the following assumptions:

• The relevant parameters of ELOPs are constant and known.
• ELOPs may undergo complete or partial disassembly.
• The disassembly line and workstation parameters are known and constant.
• The time required to perform each disassembly task is fixed.
• Task assignments must satisfy both precedence and conflict constraints.

2.2.1. Symbol Definition

The mathematical symbols involved in the UC-DLBP model are as follows:

$\mathbb{P}$	Set of all ELOP to be disassembled, $\mathbb{P}$ = {1, 2, …, P}.
${\mathbb{I}}_p$	Set of all components/parts in product p, ${\mathbb{I}}_p=\{1,2,\dots ,I_p\}$.
${\mathbb{J}}_p$	Set of all tasks in product p, ${\mathbb{J}}_p=\{1,2,\dots ,J_p\}$.
$\mathbb{L}$	Set of all disassembly lines, $\mathbb{L}=\{1,2\}$ $\mathbb{W}=\{1,2,\dots ,W\}$.
$\mathbb{S}$	U-shaped disassembly line edge selection set,  $\mathbb{S}=\{1,2\}$.
${\mathbb{W}}^1$	Set of all U-shaped workstations, $\mathbb{W}=\{1,2,\dots ,W^1\}$.
${\mathbb{W}}^2$	Set of all circular workstations, $\mathbb{W}=\{1,2,\dots ,W^2\}$.
${\mathbb{I}}_j^{p,D}$	Set of the relationship between components i and task j in product p.
${\mathbb{J}}_j^{p,R}$	Set of tasks that conflict with task j in product p.
${\mathbb{J}}_j^{p,S}$	Set of immediate tasks for task j in product p.
P	Number of products.
$I_p$	Number of components/parts in production p.
$J_p$	Number of tasks in production p.
$W^1$	Number of U-shaped workstations.
$W^2$	Number of circular workstations.
$v_{pi}$	The value of component i in product p.
$t_{pj}$	Time to execute the j-th task of the p-th product.
$c_{pj}$	Unit time cost of executing the j-th task of the p-th product.
$c_l$	Unit time cost of opening of the l-th disassembly line.
$c_{lw}$	Fixed cost of opening of the w-th workstation in l-th.
$a_{pj}$	Posture to execute the j-th task of the p-th product.
$b_{lw}$	Configuration of opening of the w-th workstation in the l-th disassembly line.

2.2.2. Relation Matrix

This section is mainly used to define the components of the product and the relationships between disassembly tasks. Through the definition of the relationship matrix, we can mathematically define the product information conveyed in the AND/OR graph. In the following formulas, p represents the number of products, i denotes the component number, and j and k represent the task numbers.

The disassembly matrix is denoted as $D=\left[d_{ij}^p\right]$, which is used to characterize the association between components and disassembly tasks.

$$d_{ij}^p=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}\mathrm{component}i\mathrm{is}\mathrm{obtained}\mathrm{by}\mathrm{task}j\mathrm{in}\mathrm{product}p;\hfill \\ -1,\hfill & \mathrm{if}\mathrm{component}i\mathrm{is}\mathrm{disassembled}\mathrm{by}\mathrm{task}j\mathrm{of}\mathrm{product}p;\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

The precedence matrix, denoted as $S=\left[s_{jk}^p\right]$, is used to depict the relationship between two disassembly tasks.

$$s_{jk}^p=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}\mathrm{task}j\mathrm{is}\mathrm{a}\mathrm{predecessor}\mathrm{of}\mathrm{task}k\mathrm{in}\mathrm{product}p;\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

The incidence matrix $R=\left[R_{jk}^p\right]$ is used to define the conflict relationships between disassembly tasks.

$$r_{jk}^p=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}\mathrm{there}\mathrm{is}\mathrm{a}\mathrm{conflict}\mathrm{between}\mathrm{task}j\mathrm{and}\mathrm{task}k\hfill \\ & \mathrm{conflict}\mathrm{with}\mathrm{each}\mathrm{other}\mathrm{in}\mathrm{product}p;\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

2.2.3. Decision Variables

$$z_{pl}=\left\{\begin{array}{cc}1,\hfill & \mathrm{product}p\mathrm{is}\mathrm{performed}\mathrm{at}\mathrm{the}\hfill \\ & \mathrm{disassembly}\mathrm{line}l.\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

$$x_{pjws}^1=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}\mathrm{disassembly}\mathrm{task}j\mathrm{in}\mathrm{product}p\mathrm{is}\mathrm{performed}\hfill \\ & \mathrm{at}\mathrm{the}s\text{-}\mathrm{th}\mathrm{position}\mathrm{in}\mathrm{workstation}w\mathrm{in}\hfill \\ & \mathrm{the}\mathrm{U}\text{-}\mathrm{shaped}\mathrm{disassembly}\mathrm{line}.\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

$$x_{pjw}^2=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}\mathrm{disassembly}\mathrm{task}j\mathrm{in}\mathrm{product}p\mathrm{is}\mathrm{performed}\hfill \\ & \mathrm{at}\mathrm{the}\mathrm{workstation}w\mathrm{in}\mathrm{the}\mathrm{circular}\mathrm{disassembly}\hfill \\ & \mathrm{line}.\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

$$y_l=\left\{\begin{array}{cc}1,\hfill & \mathrm{open}\mathrm{the}l\text{-}\mathrm{th}\mathrm{disassembly}\mathrm{line}.\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

$$u_{lw}=\left\{\begin{array}{cc}1,\hfill & \mathrm{open}\mathrm{the}\mathrm{workstation}w\mathrm{in}\mathrm{the}l\text{-}\mathrm{th}\mathrm{disassembly}\mathrm{line}.\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

$$T^l,\mathrm{the}\mathrm{cycle}\mathrm{time}\mathrm{of}\mathrm{the}l\text{-}\mathrm{th}\mathrm{disassembly}\mathrm{line}.$$

The above formulas define the decision variables of UR-DLBP, which represent the specific allocation of product disassembly tasks.

2.2.4. Maximizing Profit Model

$$\begin{array}{c}\hfill maxf=\sum _{p\in \mathbb{P}}\sum _{j\in {\mathbb{J}}_p}(\sum _{w\in {\mathbb{W}}^1}\sum _{s\in \mathbb{S}}\left(\sum _{i\in {\mathbb{I}}_p}{v}_{pi}{d}_{pij}-c_{pj}{t}_{pj}\right)\\ \hfill +\sum _{w\in {\mathbb{W}}^2}\left(\sum _{i\in {\mathbb{I}}_p}{v}_{pi}{d}_{pij}-c_{pj}{t}_{pj}\right)x_{pjw}^2)\\ \hfill -\sum _{l\in \mathbb{L}}\left(c_l{t}^l+\sum _{w\in {\mathbb{W}}^l}{c}_{lw}{u}_{lw}\right)\end{array}$$

(1)

The mathematical model of the UC-HDLBP in this paper aims to maximize profits. This profit includes the profits obtained from performing disassembly tasks within the U-shaped disassembly line and the circular disassembly line, and deducts the costs of performing disassembly tasks, as well as the costs for activating the workstations and the disassembly lines.

2.2.5. Constraints

To align with real-world scenarios, a series of constraint conditions must be defined to ensure that no illegal operations occur during disassembly, which would violate real-world logic—including the relationships between product components and tasks defined earlier, the relationships between disassembly postures, and so on.

(1)

U-shaped Disassembly Line Constraint:

$$\begin{array}{cc}\hfill & x_{pjws}^1\le u_{lw},\forall p\in \mathbb{P},\forall j\in {\mathbb{J}}_p,l=1,\hfill \\ \hfill & \forall w\in {\mathbb{W}}^1,\forall s\in \mathbb{S}and{a}_{pj}=b_{lw}\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill & x_{pjws}^1\le z_{pl},\forall p\in \mathbb{P},\forall j\in {\mathbb{J}}_p,l=1,\hfill \\ \hfill & \forall w\in {\mathbb{W}}^1,\forall s\in \mathbb{S}and{a}_{pj}=b_{lw}\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill & \sum _{j\in {\mathbb{J}}_p}\sum _{s\in \mathbb{S}}{t}_{pj}{x}_{pjws}^2\le T^2,\forall w\in {\mathbb{W}}^2\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill \sum _{w\in {\mathbb{W}}^2}& \left(w\left(x_{pjwl}^1-x_{pkwl}^1\right)+\left(2{\mathbb{W}}^1-w\right)\left(x_{pjw2}^1-x_{pkw2}^1\right)\right)\hfill \\ \hfill & +2{\mathbb{W}}^1\left(\sum _{w\in {\mathbb{W}}^1}\sum _{s\in \mathbb{S}}{x}_{pkws}^1-1\right)\le 0,\hfill \\ \hfill & \forall p\in \mathbb{P},\forall j,k\in {\mathbb{J}}_{p}and{s}_{pjk}=1\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill & s_{pjk}\left(\sum _{w\in {\mathbb{W}}^1}\sum _{s\in \mathbb{S}}{x}_{pkws}^1-\sum _{w\in {\mathbb{W}}^1}\sum _{s\in \mathbb{S}}{x}_{pjws}^1\right)\le 0,\hfill \\ \hfill & \forall p\in \mathbb{P},\forall j,k\in {\mathbb{J}}_p,\forall w\in {\mathbb{W}}^1\forall s\in \mathbb{S}\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill & \sum _{w\in {\mathbb{W}}^1}\sum _{s\in \mathbb{S}}\left(x_{pjws}^1+x_{pkws}^1\right)\le 1,\hfill \\ \hfill & \forall p\in \mathbb{P},\forall j,k\in {\mathbb{J}}_{p}and{r}_{pjk}=1\hfill \end{array}$$

(7)

The above constraints are the conditions that must be met when products are allocated to the U-shaped disassembly line. Formulas (2) and (3) involve the constraints of employees’ postures—that is, the disassembly posture required for the product must be consistent with the disassembly posture of the assigned workstation, and the disassembly line and the corresponding workstation must be in the open state. Formula (4) ensures that the work cycle on the disassembly line will not be longer than the operating cycle of the disassembly line. Formulas (6) and (7) are the relationships that the disassembly tasks allocated on the U-shaped disassembly line need to satisfy, including the precedence relationship and the conflict relationship.

(2)

Circular Disassembly Line Constraint:

$$\begin{array}{cc}\hfill & x_{pjw}^2\le u_{lw},\forall p\in \mathbb{P},\forall j\in J_p,l=2,\hfill \\ \hfill & \forall w\in {\mathbb{W}}^2,a_{pj}=b_{lw}\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill & x_{pjw}^2\le z_{pl},\forall p\in \mathbb{P},\forall j\in J_p,l=2,\hfill \\ \hfill & \forall w\in {\mathbb{W}}^2,a_{pj}=b_{lw}\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill & \sum _{j\in {\mathbb{J}}_p}{t}_{pj}{x}_{pjw}^2\le T^2,\forall w\in {\mathbb{W}}^2\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill & s_{pjk}\left(\sum _{w\in {\mathbb{W}}^2}{x}_{pkw}^2-\sum _{w\in {\mathbb{W}}^2}{x}_{pjw}^2\right)\le 0,\hfill \\ \hfill & \forall p\in \mathbb{P},\forall j,k\in {\mathbb{J}}_p,\forall w\in {\mathbb{W}}^2\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill & \sum _{w\in {\mathbb{W}}^2}\left(x_{pjw}^2+x_{pkw}^2\right)\le 1,\hfill \\ \hfill & \forall p\in \mathbb{P},\forall j,k\in {\mathbb{J}}_{p}and{r}_{pjk}=1\hfill \end{array}$$

(12)

$$\begin{array}{c}\hfill y_2=1\end{array}$$

(13)

$$\begin{array}{c}\hfill \sum _{w\in {\mathbb{W}}^2}{u}_{lw}={\mathbb{W}}^2,l=2,\forall w\in {\mathbb{W}}^2\end{array}$$

(14)

The above constraints are the conditions that must be satisfied when products are allocated to the circular disassembly line. Formulas (8) and (9) are used to restrict the posture issues of products and workstations, and at the same time ensure that products can only be allocated to the disassembly lines and workstations that are turned on. Formula (10) ensures that the cycle time of the circular disassembly line is longer than the cycle time of each workstation. Formulas (11) and (12) ensure that the disassembly tasks allocated to the circular disassembly line satisfy the precedence relationship and the conflict relationship. Formulas (13) and (14) are to maximize the utilization of the characteristics of the circular workstations, requiring the disassembly line and the internal workstations to remain in the open state.

(3)

Hybrid Disassembly Line Constraint:

$$\begin{array}{c}\hfill \sum _{l\in \mathbb{L}}{z}_{pl}=1,\forall p\in \mathbb{P}\end{array}$$

(15)

$$\begin{array}{c}\hfill \sum _{w\in {\mathbb{W}}^1}{x}_{pjw}^1+\sum _{w\in {\mathbb{W}}^2}{x}_{pjw}^2\le 1,\forall p\in \mathbb{P},\forall j\in {\mathbb{J}}_p\end{array}$$

(16)

$$\begin{array}{cc}\hfill & z_{pl}\le y_l,\forall p\in \mathbb{P},\forall l\in \mathbb{L}\hfill \end{array}$$

(17)

$$\begin{array}{cc}\hfill & u_{lw}\le y_l,\forall w\in {\mathbb{W}}^l,\forall l\in \mathbb{L}\hfill \end{array}$$

(18)

$$\begin{array}{cc}\hfill & z_{pl}\in \{0,1\},\forall p\in \mathbb{P},\forall l\in \mathbb{L}\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill & x_{pjws}^1\in \{0,1\},\forall p\in \mathbb{P},\forall j\in {\mathbb{J}}_p,\forall w\in {\mathbb{W}}^l\hfill \end{array}$$

(20)

$$\begin{array}{cc}\hfill & x_{pjw}^2\in \{0,1\},\forall p\in \mathbb{P},\forall j\in {\mathbb{J}}_p,\forall w\in {\mathbb{W}}^2\hfill \end{array}$$

(21)

$$\begin{array}{cc}\hfill & y_l\in \{0,1\},\forall l\in \mathbb{L}\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill & u_{lw}\in \{0,1\},\forall w\in {\mathbb{W}}^l,\forall l\in \mathbb{L}\hfill \end{array}$$

(23)

$$\begin{array}{cc}\hfill & T^l\in {\mathbb{R}}_{+},\forall l\in \mathbb{L}\hfill \end{array}$$

(24)

The above constraints are the constraints of the hybrid disassembly line. Formulas (15) and (16) ensure that each product can only be allocated to one disassembly line, and the disassembly tasks within the product are executed at most once. Formulas (17) and (18) ensure that only the disassembly lines and workstations in the open state can be used. Formulas (19)–(24) define the variable ranges of the model parameters.

3. Large Language Model Assisted Duel-DQN

The UC-HDLBP presents a hybrid disassembly line layout that combines U-shaped and circular configurations. Compared with traditional linear disassembly lines, the hybrid layout introduces greater complexity in task sequencing and resource allocation. To efficiently solve the UC-HDLBP while maintaining adaptability to smart IoT-enabled environments, we adopt a two-stage reinforcement learning approach that integrates a fine-tuned Large Language Model (LLM) with a redesigned Duel-DQN algorithm.

As illustrated in Figure 3, the proposed framework decomposes the problem into two stages: (1) the Disassembly Sequence Problem (DSP), where the LLM generates efficient disassembly sequences and allocates products to disassembly lines; and (2) the Disassembly Line Balancing Problem (DLBP), where the Duel-DQN assigns specific disassembly tasks to workstations under posture and precedence constraints. Information flows between the two stages iteratively to refine decision-making based on performance feedback.

Importantly, by instruction-fine-tuning the LLM, we not only enhance its domain adaptability for disassembly sequencing but also prepare the framework for future extensions involving cybersecurity and privacy policies in IoT-enabled manufacturing environments. This design consideration ensures that trust, privacy, and data sensitivity factors can be incorporated into task allocation strategies without extensive re-engineering.

3.1. Instruction-Fine-Tuning

Instruction-fine-tuning is a technique that adapts pre-trained LLMs to specific tasks by providing structured input–output pairs based on domain knowledge. By fine-tuning the LLM on a curated dataset of disassembly scenarios, we guide it to generate accurate disassembly sequences aligned with ergonomic and operational constraints.

To achieve efficient and scalable fine-tuning, we apply the Quantized Low-Rank Adaptation (QLoRA) method [17], which reduces the computational and memory requirements of LLM adaptation by quantizing model parameters to lower-precision formats. This approach is particularly suitable for edge or resource-constrained environments, further supporting potential IoT applications.

3.1.1. Instruction Set Construction

Each instruction instance consists of three elements:

• Instruction: The high-level prompt describing the disassembly task.
• Input: Specific product-related information provided to the LLM.
• Output: The expected disassembly sequence and line assignment.

The instruction set is systematically generated from product AND/OR graphs to cover all feasible disassembly sequences. Algorithm 1 outlines the automatic construction process to ensure dataset completeness.

An Example of Instruction:

instcution: The “personal computer” contains the following disassembly sequences: [1, 3, 7, 11], [1, 2, 5, 10], and [1, 3, 6, 9]. It also includes two disassembly lines: 0 and 1. Only output the selected disassembly lines and disassembly sequences.

input: Select a disassembly sequence for the personal computer and allocate the disassembly line.

output: 1,[1, 3, 7, 11]

Algorithm 1 Instruction set Generation Algorithm

Require: Name (the name of product p) and ${\mathbb{I}}_j^{p,D}$ (set of conflict matrices of products) Ensure: Instruction set $IS$ Initialization: Instruction unit $IS\leftarrow \varnothing$, Disassemble sequence set ${\mathbb{D}}_p\leftarrow \varnothing$, String $S\leftarrow \varnothing$ ${\mathbb{D}}_p\leftarrow$ Traverse ${\mathbb{I}}_i^{p,D}$ to obtain all disassembly sequences. $S\leftarrow$ “Name” contains the following disassembly sequences. for d in ${\mathbb{D}}_p$ do   $S\leftarrow S+d$ end for $IS\leftarrow <$$Instruction,S$ + Only output the selected disassembly line and disassembly sequence.>, <$input$, Select a disassembly sequence of “Name” and allocate the disassembly line.> for d in ${\mathbb{D}}_p$ do   $S\leftarrow d$.   for $temp$ in $[0,1]$ do    $I{S}^{\prime }\leftarrow IS$+$<output,temp+S>$.    $\mathbb{IS}\leftarrow \mathbb{IS}+I{S}^{\prime }$   end for end for return  $IS$

3.1.2. Instruction Set Generation

Algorithm 1 details the generation of the instruction dataset used to fine-tune the LLM, ensuring broad coverage of possible product configurations and sequencing options.

3.1.3. Expected Fine-Tuning Result

After fine-tuning, the LLM is expected to accurately map given product structures to efficient disassembly sequences and allocate products to appropriate disassembly lines. Furthermore, the fine-tuned LLM forms a foundation for future extensions involving privacy-preserving sequencing or trust-aware task allocation in IoT-integrated settings.

Q: Select a disassembly sequence for the personal computer and allocate the disassembly line.

A: 0, [1, 2, 5, 10]

If the LLM can output a dialogue like the above after fine-tuning, it indicates that the LLM has successfully completed the task and has the ability to solve the DSP.

3.2. Design of the Duel-DQN Algorithm

3.2.1. State Space

The state space encodes the current allocation of disassembly tasks across workstations, including information about worker posture constraints and task assignments. Formally, the state is defined as follows:

$$\begin{array}{cc}\hfill state=& \left\{x_{pjs}^1,x_{pj}^2,z_p,X(x_{pjs}^1,x_{pj}^2,z_p)\right\},p\in P,j\in {\mathbb{J}}_p,s\in \mathbb{S}\hfill \\ & (x_{pjs}^1,x_{pj}^2)\in [0,max({\mathbb{W}}^1,{\mathbb{W}}^2)],z_p\in \mathbb{L},X\in \{0,1\}\hfill \end{array}$$

(25)

Among them, $X(x_{pjs}^1,x_{pj}^2,z_p)$ represents the current state on the disassembly line. When X is 1, it means that a product is being disassembled on the disassembly line. The value of $x_{pjs}^1$ indicates that task j of product p is assigned to the s side of workstation $x^1$ of the U-shaped disassembly line, and the value of $x_{pj}^2$ indicates that task j of product p is assigned to workstation $x^2$ of the circular disassembly line. $z_p$ indicates that task p is assigned to disassembly line z.

3.2.2. Action Space

A U-shaped disassembly line can be simply understood as the combination of two straight disassembly lines. A circular disassembly line can be considered as two disassembly lines connected in a loop. Therefore, in the design of the action space, actions for both of these two types of disassembly lines can be designed as variants of the straight disassembly line. We assume that the U-shaped disassembly line and the circular disassembly line have the same number of workstations.

Table 1. Action selection code of UC-HDLBP.

	Disassembly Line Type
Action Value	U-Shape	Circular
$[0,1,\dots ,max(2{\mathbb{W}}^1,{\mathbb{W}}^2)/2]$	Select entrance side	Select workstation.
$[max(2{\mathbb{W}}^1,{\mathbb{W}}^2)/2,\dots ,max(2{\mathbb{W}}^1,{\mathbb{W}}^2)-1]$	Select exit side	Select workstation.
$max(2{\mathbb{W}}^1,{\mathbb{W}}^2)$	do not perform the disassembly task

shows the action values designed in this section. Note that the action values on the exit side of the U-shaped disassembly line are designed in reverse order in the table. Since the circular disassembly line has no space restrictions, it does not have illegal values like the straight disassembly line. When the number of workstations on both sides is different, there will inevitably be illegal values on one side. In this case, an additional judgment can be made during the training phase.

3.2.3. Reward Design

In the DSP stage, the reward for the agent is the profit difference between the current selection and the previous selection. The reward structure in the DLBP stage is more complex, aiming to achieve the complete disassembly of products while avoiding an excessive number of tasks assigned to a single workstation. Positive rewards will be given for behaviors that comply with the disassembly rules, while negative rewards will be imposed for behaviors that violate the rules or fail to execute tasks. When the disassembly of a product is completed, the agent will also receive additional rewards.

3.2.4. Train Algorithm

Based on the above settings of the Duel-DQN environment parameters, we will train the agent for disassembly task allocation. As shown in Algorithm 2, the process first involves generating instructions using the instruction set generation algorithm mentioned above to fine-tune the LLM. Subsequently, according to the LLM’s decisions, the agent continues the subsequent disassembly task allocation activities and updates the state space. Finally, the profit obtained in this round is returned to the LLM to reconsider the latest decision for a new round of attempts. It can be observed that the algorithm only receives positive rewards when the allocation of disassembly tasks conforms to the optimal disassembly postures.

Algorithm 2 Task Allocation Phase Algorithm

Require: LLM, IS, set of disassembly sequences $\mathbb{D}$, action A, selected disassembly line l, set of workstations ${\mathbb{W}}^l$, state $\mathbf{X}$, $Reward$ Ensure: $\mathbf{X}$, $Profit$ Initialization: $Profit\leftarrow 0$,$\mathbb{D}\leftarrow 0$,$l\leftarrow 0$ LLM←FineTuning(LLM,IS) while Not meeting the termination criteria do   $\mathbb{D},l\leftarrow$ AskLLM(Profit)   if l is a U-shaped disassembly line $and$ A not legitimate then     $Reward\leftarrow$ impose a severe penalty   else     continue   end if   for p in $\mathbb{D}$ do     for j in ${\mathbb{D}}_p$ do       if A == $len\left({\mathbb{W}}^l\right)$ then        ${\mathbf{X}}_{lpj}\leftarrow A$        $Reward\leftarrow$ impose a slight penalty         return $\mathbf{X}$, $Reward$, $Profit$       else if $a_{lA}==b_{lA}$ then        ${\mathbf{X}}_{plj}\leftarrow A$        $Profit\leftarrow$ calculate the profit obtained from the distribution        $Reward$ += $Profit$       else        $Reward\leftarrow$ impose an extreme penalty        return $\mathbf{X}$, $Reward$, $Profit$       end if     end for   end for   send $Profit$ to LLM end while return $\mathbf{X}$, $Profit$

4. Experiment and Results

4.1. Experimental Cases

The experimental cases of this paper are shown in

Table 2. Case Description.

Case ID	Product Quantity				Number of Disassemble Line Workstations
	PC	Washing Machine	Radio	Mechanical Module	U-Shaped	Circular
1	1	1	1	1	5	5
2	2	2	2	3	6	6
3	3	5	5	6	8	8
4	10	10	10	10	10	10

, where different cases correspond to disassembly environments of varying scales, including the number of products and workstations.

4.2. Comparison of Fine-Tuning Effects of Trained Products

The content of this section mainly focuses on observing the effect of fine-tuning. The LLM model used in this experiment is InternLM2-7B. Currently, we have two models: the original model and the fine-tuned model. The dataset we selected for fine-tuning includes the products mentioned in Table 2. The steps of this experiment are as follows:

• Load the fine-tuned model and the original model, respectively.
• Conduct 15,000 questions, with the scope limited to the disassembly sequences of the four products in Table 2 Use the AND/OR diagrams of the corresponding products as the result verification, and record the number of legal disassembly sequences recommended by the model.
• Before the experiment, enable the model to understand the products in the form of describing the AND/OR diagrams. When starting the test, ask questions to make the model provide the disassembly sequences.

Figure 4 shows the experimental results demonstrating the accuracy rates of the models in solving DSP under two different scenarios, respectively. On the left side, the figure presents the accuracy rates of the original model and the fine-tuned model when directly asked questions. It can be observed that when asking questions directly without providing relevant hints or training to the model, the model has no clue about the direction of the problem. Consequently, within 15,000 queries, the number of times the model can obtain the correct disassembly sequence and provide correct answers is very limited. The accuracy rate after fine-tuning is 100%. This is because all tested questions fall within the content scope of the dataset. As long as the model’s fine-tuning training is successful, the model will naturally memorize these answers. On the right side, the original model after dialogue training is displayed. Through dialogue training, the original model has understood the problem. Although it can provide answers, there is still an error rate. Since the fine-tuned model has already achieved a 100% accuracy rate in direct question-and-answer scenarios, there is no need to conduct dialogue training for it.

For the UC-HDLBP, using the advantages of the LLM to solve the product allocation and DSP of this issue can reduce the difficulty of solving it by the Duel-DQN. Therefore, for this problem, pre-fine-tuning an LLM is a relatively good solution.

4.3. Comparison of the Fine-Tuning Effects for Untrained Products

This section explores whether the model can perform as well as in the previous section when it encounters a brand-new product. In this section, a brand-new virtual product will be prepared, and the AND/OR diagram of this virtual product is shown in Figure 5.

In the experiment of this section, we will also employ the original model and the fine-tuned model for comparison. We will conduct 100 simulations for each model, respectively, and record the results.

As illustrated in Figure 6, although the fine-tuned LLM shows some capability to generate plausible disassembly sequences, its performance on entirely unseen products remains limited. This outcome reflects the fact that the fine-tuning process primarily memorized training instances rather than generalizing abstract AND/OR graph reasoning. Future work will address this limitation by incorporating structure-aware instruction-tuning or graph-based input representations.

Nevertheless, for smart manufacturing systems where product structures are pre-known and defined, the current fine-tuned LLM is sufficient and provides a practical solution foundation.

4.4. Comparison of Iteration Efficiency

We next compare the iterative learning efficiency between the Duel-DQN alone and the LLM-assisted Duel-DQN framework. Case 3 (medium-large scale) was selected as the benchmark.

In the experiment of this section, Case 3 is taken as an example to observe the training situation;for sitting and standing postures, the time cost parameters for performing disassembly are fixed at 4 and 10, respectively, and the costs for activating workstations are fixed at 100 and 150, respectively. The final data obtained are as follows.

Figure 7 shows the results of the iterations. This section has tracked 100,000 iterations. It can be seen that around the 70,000th step, the LLM-assisted Duel-DQN begins to stabilize, while the Duel-DQN remains in an exploration state. The Duel-DQN algorithm does not finally stabilize and catch up with the LLM-assisted DQN algorithm until around the 95,000th step.

The results clearly show that the LLM-assisted Duel-DQN converges approximately 26% faster than the traditional Duel-DQN: it stabilizes around 70,000 steps, compared to about 95,000 steps for the Duel-DQN without LLM assistance. This is because the pre-structuring of disassembly sequences by the LLM effectively reduces the search complexity of Duel-DQN, accelerating convergence and improving sample efficiency. Such efficiency gains are critical for deployment in IoT-enabled systems where computational resources and response times are constrained.

Actually, in subsequent iterations, the solution results of the LLM-assisted Duel-DQN algorithm continue to improve further, while the Duel-DQN remains completely stagnant at its peak value achieved at 95,000 steps. The primary factor contributing to this phenomenon is the performance limitation of the Duel-DQN algorithm when solving large-scale cases, which will be discussed as a limitation in the next section.

4.5. Comparison and Analysis of Results

Table 3. Profit comparison between Duel-DQN and Cplex.

Case ID	Profit
	CPLEX	LLM-Assisted Duel-DQN	DQN	Duel-DQN
1	1344	1299	1036	1296
2	2369	2266	1447	1452
3	5562	5458	3518	1685
4	12,624	12,246	141	139

shows the final profits obtained by different algorithms under different cases. Through the comparison of all the data, we can find that the profit obtained by the solution proposed in this paper is quite close to that obtained by CPLEX. Due to the relatively large scale of the problem in Case 4, there was a situation of insufficient memory when using DQN and Duel-DQN, resulting in not very good final training results. However, the Duel-DQN assisted by the LLM under the same conditions was able to obtain a solution, which indicates that the solution method mentioned in this section can still solve problems in large-scale cases when there is a lack of computing resources, reducing the economic consumption in terms of computing power.

The LLM-assisted Duel-DQN consistently achieves results close to the optimal CPLEX solution, outperforming both the DQN and Duel-DQN baselines. In large-scale cases (e.g., Case 4), the baseline methods (DQN and Duel-DQN) suffer from memory exhaustion or convergence failures, leading to varying degrees of result degradation. The LLM-assisted method, however, maintains stable performance.

This phenomenon primarily stems from the fact that Duel-DQN requires additional computational resources and storage space to maintain its dual-network structure compared to the ordinary DQN [18]. In small-to-medium-scale cases (e.g., Cases 1 and 2), Duel-DQN can normally obtain better solutions than the DQN algorithm. However, as the problem scale increases (e.g., Cases 3 and 4), the expansion of the problem’s state space leads to a rapid increase in computational resource requirements, and Duel-DQN becomes unable to solve problems effectively. As previously mentioned, the integration of LLM assistance can reduce the actual problem scale that DQN needs to solve, thereby ensuring iterative efficiency and results under limited computational resources.

This demonstrates that our framework is not only more efficient but also more scalable and robust—key properties for future deployment in distributed, IoT-enabled, or privacy-sensitive environments.

5. Cybersecurity and Privacy in IoT-Enabled Disassembly Systems

The LLM-assisted Duel-DQN algorithm demonstrates superior performance in optimizing the disassembly sequence (as shown in Table 3 and Figure 7), which is crucial in an IoT-enabled disassembly line where real-time data on product condition and task dependencies, gathered by sensors, needs to be processed rapidly and accurately. The algorithm’s ability to handle the complex precedence relationships derived from IoT sensor data allows for more efficient and adaptive disassembly. In IoT-enabled smart manufacturing, disassembly lines involve sensors, edge devices, and communication protocols that handle sensitive operational and worker-related data. While the primary focus of our work is on optimization, the LLM-assisted Duel-DQN framework is inherently extensible and can be adapted to enhance cybersecurity and privacy protections in such environments. To clarify our intended direction, this section outlines a solution-oriented vision that leverages components of our proposed system:

• Privacy-Aware Task Sequencing: The fine-tuned LLM can be enhanced to prioritize the early disassembly of components containing confidential or security-sensitive data (e.g., memory chips in smart devices). This enables safe disposal or isolation of sensitive components early in the workflow.
• Federated Learning for Worker Privacy: Our reinforcement learning agent can be integrated with federated learning frameworks [19] to enable decentralized training across IoT-enabled workstations, where posture, fatigue, or biometric data are processed locally. This prevents raw data transmission and improves compliance with privacy regulations.
• Secure Communication for Resource-Constrained Devices: Lightweight cryptographic protocols [20] and attribute-based encryption [21] can be implemented within the control layer to secure the transmission of disassembly instructions between edge devices and central coordinators. Our framework supports these protocols due to its modular decision-making architecture.
• Trust-Aware Reward Design: Privacy or safety constraints can be embedded into the reinforcement learning reward function—for example, penalizing task allocations that require a worker to reveal identifiable biometric data or exceed ergonomic risk thresholds. The LLM’s instruction-tuning mechanism enables these soft constraints to be flexibly encoded.

• Practical Integration: These features have not yet been implemented in the current experiment but are technically feasible given the modular structure of our architecture. For example, by combining federated optimization with edge inference on IoT devices, future extensions can support privacy-by-design learning loops. Our goal in this section is to articulate how LLM-based decision engines can act as gateways to integrate optimization, learning, and secure communication for intelligent disassembly environments.

6. Conclusions and Future Work

This paper addresses the challenges of disassembly line balancing in complex hybrid layouts by proposing the U-shaped and Circular Hybrid Disassembly Line Balancing Problem (UC-HDLBP). A corresponding mathematical model was developed, considering both workstation utilization and worker posture constraints. To efficiently solve this problem, a two-stage LLM-assisted Duel-DQN framework was introduced, leveraging instruction-fine-tuning of large language models via Quantized Low-Rank Adaptation (QLoRA) and a redesigned reinforcement learning structure.

Experimental results demonstrate that the proposed method significantly improves iteration efficiency compared with traditional reinforcement learning approaches, achieving near-optimal performance while reducing computational overhead. The fine-tuned LLM effectively guides disassembly sequencing under known product scenarios, while the Duel-DQN efficiently allocates disassembly tasks across hybrid lines. The solution method also proves robust when handling larger-scale problems, offering clear advantages in scalability and learning stability.

Beyond operational optimization, this work further establishes a foundation for addressing cybersecurity, privacy, and trust challenges in IoT-enabled smart remanufacturing environments. By instruction-tuning the LLM and designing flexible reward structures, the framework can be readily extended to prioritize privacy-preserving sequencing, federated learning, and lightweight secure communication in distributed, resource-constrained settings.

In future work, we plan to enhance the generalization capability of the LLM by incorporating graph-based disassembly representations and structure-aware instruction tuning, enabling adaptive sequencing for previously unseen products. Additionally, we aim to integrate explicit privacy and trust constraints into the optimization process, allowing dynamic task allocation under varying security requirements. Combining our current approach with developments in distributed learning, lightweight encryption, and blockchain-based auditing could further improve the resilience and transparency of smart disassembly systems. These directions align with broader goals of sustainable manufacturing, ethical AI deployment, and secure IoT integration.