Knowledge Graph-Embedded Time-Serial-Data-Driven Bottleneck Analysis of Textile and Apparel Production Processes

Abstract

There is a lack of high correlation and reuse potential among multiple manufacturing data for textiles and apparel. Moreover, the material flow traceability between production workstations is not clear, making it difficult to detect potential production bottlenecks. This paper proposes a knowledge graph embedded time serial data-driven bottleneck analysis of textile and apparel production processes. Firstly, a dynamic information association model is established to organize global manufacturing information, including the static data and time-series data features. Also, a textile-corpus-oriented knowledge extraction model is designed to construct a time-series knowledge graph for textile and apparel production (TKG4TA). Then, a temporal knowledge-driven production process bottleneck prediction model is presented based on manufacturing knowledge in the textile and apparel industry. Of these, textile knowledge is transformed into embeddings using a graph convolutional network (GCN). In turn, the context-associated information features are learned by the long short-term memory (LSTM) to predict the bottlenecks in the textile and apparel production process. Finally, a typical process flow in a shirt manufacturing workshop is used as a case study. It shows that the F1 value of the proposed method for named entity recognition and relationship extraction is up to 80.3%, and 50.6%, respectively. The performance of the proposed model for bottleneck prediction is improved by 8.2% and 14.92% compared to only the use of GCN or LSTM in the mean absolute error. This model may provide a solid foundation for the temporal knowledge-graph-driven bottleneck analysis of shirt manufacturing.

1. Introduction

Textile and apparel production is an important economic pillar in China [1,2,3]. Currently, the traditional apparel production mode is rapidly shifting to customized garment production, increasing the demand for the intelligent customization of orders [4]. However, the production process of textiles and apparel is complex. A large amount of multi-source manufacturing data (process data, time-series stream data) accumulated in each production workstation are independent of each other and have a severely low reuse rate. Also, the textile and apparel production workstations are highly volatile. Moreover, there are many types of equipment and the information exchange between equipment types is blocked. This leads to unclear traceability of material flow between manufacturing workstations, making process monitoring difficult and potential production bottlenecks hard to detect. This lack of correlation and reuse of global data, including both real-time manufacturing data and historical manufacturing data from the whole production workstations, poses a great challenge to the flexible production and fast bottleneck prediction in the textile and apparel industry.

Exploring how to correlate and utilize manufacturing data from a global perspective is a key to predicting bottleneck workstations and controlling quality for meeting the requirements of textile and apparel production.

Current, research into the information modeling of production resources is dominated by ontology and knowledge graphs. Leveraging ontology in the textile and apparel field can be useful in describing complex relationships between objects [5,6]. However, it lacks fine-grained descriptions in massive data. Further, knowledge graphs have attracted attention in the textile industry due to their ability to efficiently organize massive, multi-source, heterogeneous data with complex semantic relationships [7,8], but there are fewer explorations on the unified organization of dynamic data. In [9], the knowledge graph facet model provided an idea for establishing dynamic knowledge modeling.

On the other hand, a bottleneck means that imbalance and inefficiency in the line can cause the operation process to be hampered, resulting in a queue of semi-finished products [10]. In [11], the ranked position weight and region approach method are used to overcome bottlenecks in the garment industry. Some scholars are currently focusing on bottleneck analysis based on data science methods such as convolutional neural network (CNN) and long short-term memory (LSTM) in the textile industry [12,13,14]. The above studies reflect somewhat the intelligent decision analysis of the textile and apparel production process. Moreover, in combining knowledge and data science methods to drive textile and apparel manufacturing [15], the domain knowledge related to the text corpus is integrated into a bidirectional long short-term memory (BiLSTM)-based multitask-learning network to bridge the semantic gap between customer needs and design specifications. However, the implicit relationships contained in textile knowledge are not yet explored in depth enough. Fortunately, the fact that graph neural networks can deal with graph data, such as knowledge graphs, provides a feasible option to fully utilize dynamic data and knowledge for production process bottleneck analysis.

Summarizing the above, this paper proposes a knowledge-graph-embedded time-serial-data-driven bottleneck analysis of textile and apparel production processes. Firstly, a dynamic information association model integrating static data and time-series data is established. Then, a textile-corpus-oriented knowledge extraction method is designed to construct a time-series knowledge graph for textile and apparel production (TKG4TA). Further, a temporal knowledge-driven model for predicting bottleneck workstations is designed, based on manufacturing knowledge in the textile industry. Also, textile knowledge is transformed into embeddings using a graph neural network. In turn, the context-associated information features are learned by LSTM to predict the bottlenecks in the textile and apparel production process.

The main innovations of this paper are as follows: (1) We propose a dynamic information association model that integrates static and time-series data in textile and apparel production. (2) We design a textile-corpus-oriented knowledge extraction method and incorporate the information association model to construct a textile and apparel production time-series knowledge graph. (3) We present a temporal knowledge-driven bottleneck prediction model to predict bottleneck workstations in textile and apparel production lines.

The rest of this paper is organized as follows: Section 2 introduces the literature review. Section 3 details the knowledge modeling and graph generation in textile and apparel production processes. Section 4 proposes the temporal knowledge-graph-driven bottleneck analysis in detail. A case study is evaluated in Section 5. Finally, the conclusions and limitations of this study are summarized in Section 6.

2. Literature Review

2.1. Modeling Methods for Textile and Apparel Production Resources

Currently, modeling for production resources mainly focuses on ontologies and knowledge graph techniques. In [16], the authors focused on the domain of fashion and introduced Fashionpedia as a step toward mapping out the visual aspects of the fashion world to build upon the Fashionpedia ontology. In [5], the authors proposed an ontology-based knowledge base for user profiles and garment features in apparel recommender systems. In [17], the authors presented a new approach for constructing a garment pattern design knowledge base (GPDKB) using sensory analysis, ontology, and support vector regression (SVR) modeling, to formalize the complete knowledge of garment pattern design. In [6], the authors designed an advanced upcycling method in fashion practice by establishing material inventories and garment ontologies. However, there is a lack of description of fine-grained semantic relationships between massive amounts of textile and apparel data.

For the knowledge graph construction in the textile and apparel industry, the authors proposed a knowledge graph construction method for the textile field facing the industrial Internet, and the accuracy of knowledge is improved [7]. In [8], the authors presented a construction method of a knowledge graph for clothing recommendations based on expert knowledge. However, the above studies have given less consideration to the dynamic evolution of knowledge. In [9], the authors established a knowledge graph facet model for the custom apparel production process, which provided an idea for building dynamic knowledge modeling. In addition, one of the key techniques for knowledge graph construction is knowledge extraction [18,19,20,21]. In [22], the authors designed a Chinese-named entity recognition for clothing knowledge graph construction. Some scholars have introduced knowledge into the textile and clothing field, but the details of knowledge acquisition are less discussed [23,24,25].

2.2. Production Process Inspection and Analysis of Textile and Apparel Manufacturing Systems

Many researchers in the field of textile and apparel are currently employing data science methodologies for inspection and analysis. In reference [26], the authors utilized K-means clustering and support vector machines to investigate data mining-based solutions for apparel size assignment. Reference [27] introduced a probabilistic neural network model designed to predict and control garment fit levels. This model is based on inputs such as garment ease allowances, digital pressures, and fabric mechanical properties measured within a three-dimensional (3D) virtual environment. Reference [14] presented a prediction model for rotor yarn quality based on CNN and LSTM. This model accurately forecasts final yarn quality indicators given specific conditions. In reference [28], the authors have conceived and implemented an interactive system for fashion and garment design. This system is meticulously constructed through the systematic integration of various data-driven services, including garment design recommendations, 3D virtual garment fitting visualization, a repository of design knowledge, and the ability to adjust design parameters. In [29], the authors presented a new approach to garment fit assessment using a probabilistic neural network, aiming to promote the implementation of garment e-mass customization in the new era of Industry 4.0.

The above studies have made more use of data but less use of knowledge in the textile and apparel industry. Knowledge graphs, as a kind of graph data, can describe complex relationships in the physical world. In recent years, massive graph data can be processed using graph convolutional network (GCN) techniques [30]. This has been successfully applied to traffic prediction [31,32], medicine discovery [33,34], and industrial production [35,36]. Moreover, a novel semi-supervised graph convolutional network is presented to extract the textile image features through the image descriptors for the recognition of a few labeled textile defects [37]. This provides a viable idea for utilizing dynamic data and knowledge in textile and apparel for production process inspection and analysis.

3. Knowledge Modeling and Graph Generation in Textile and Apparel Production Processes

Based on the manufacturing data from the textile and apparel production process, a data-and-knowledge association model is constructed, as shown in Figure 1. It involves the workstation and task planning layer, the equipment and process layer, and the state data layer, respectively. Specifically, the information involved in the textile and apparel production process mainly derives from ERP, MES, and databases. (1) The workstation and task planning layer: The workstation is a more centralized and complete unit of data in the apparel manufacturing process. Whenever a new manufacturing order is placed, the material is assigned to a specialized production line. Then, according to the personnel and process configuration of the workstation, it passes through each process in turn. Finally, the material is stored and shipped. The task is to split the product processing content at each workstation according to the daily or weekly work plan issued by the shop. (2) The equipment and process layer: Equipment refers to small, medium, and large machining tools, which are configured to perform multiple or single machining tasks. Process is a series of machining operations accomplished at the same machining location or on the same machining equipment. This is developed by the process designer according to the product requirements. (3) The state data layer: It refers to the relevant data collected and stored in a time-series database at different collection intervals using devices such as sensors, monitoring equipment, and automatic collection instruments.

3.1. Data and Knowledge Modeling of Textile and Apparel Production Processes

For the static knowledge and dynamic time-series data contained in the textile and apparel production workshop, the definition of the time-series knowledge graph for textile and apparel production TKG4TA is given as follows.

TKG4TA = {E, R, S_K, D_K}

E = {e₁, e₂, …, e_n} denotes a set of knowledge entities, including concept entities, instance entities, and attribute value entities. The entity types are described in

Table 1. Types of entity for textile and apparel production.

Entity Type	Corresponding Entity Label
Brand	BR
Fashion_Style	FS
Suitable_Crowd	SC
Fabric	FA
Process	PR
Color	CO
Semi-finished_Products	SFP
Clothing_Length	CL
Sleeve_Pattern	SP
Device	DE
Tool	TO
Worker	WO
Material	MA

. On the one hand, specific entities are selected from the available data. On the other hand, the existing literature on knowledge extraction for textile and apparel production is also referenced [38]. R = {R_l, R_s, R_t} is a set of relationships, including sequential relationship (R_l), structural relationship (R_s), and time-series relationship (R_t), described in

Table 2. Relationship description of knowledge nodes.

Relationship Type	Relationship Name	Instruction
Sequential relationship (Rl)	FlowTo	Flow relationship from a workstation to the next workstation
	Parallel	Parallel relationship between processes
	next	Sequential relationship between processes
Structural relationship (Rs)	is	Manufacturing process information
	has	Basic affiliation
	has_Worker	Workers for workstations
	has_Device	Devices for workstations
	hasAttr	Static attributes
Time-series relationship (Rt)	HasAttr:T	The parameter value of the entity attribute involved at time T

. S_K = {e_h, r, e_t} is the set of static knowledge. D_K = {e_h, r, e_t, t} is the set of time-series knowledge. Of these, e_h and e_t denote the head entity and tail entity of the triples, respectively. r denotes the relationship between entities. t denotes the timestamp.

In the textile workshop, it is necessary to record the relevant resource knowledge, process knowledge, and processing status information involved in the manufacturing process. There is also a logical sequence and relationship flow among the knowledge types. In addition, basic information about semi-finished products, technical requirements, and quality control standards is also included. Thus, a data and knowledge ontology for textile and apparel production processes is designed, as shown in Figure 2.

The temporal relationship “HasAttr:T” defined in Table 2 is used to establish the relationship among personnel, equipment, process methods, and task descriptions involved in the machining process. In this way, each node related to the workstation is associated with entities, including human, machine, material, method, and environment in the production system. Also, there are some interactive relationships between the entities. Each entity has its unique attributes, including static and dynamic attributes.

3.2. Knowledge Graph Construction of Textile and Apparel Production Processes

Based on Section 3.1, to obtain the information from the textile and apparel production process to form structured knowledge, a BERT-BiLSTM-Attention-CRF joint-entity-relationship extraction model is designed, as shown in Figure 3. The design details of the model are given as follows. Also, the input with Chinese corpus corresponds to the English written in blue in Figure 3. It is worth noting that the data source is a Chinese document, but only the characters are different. The method proposed in this paper can be applied to other languages as well.

The possible overlapping entities in the corpus are taken into account in the BERT-BiLSTM-Attention-CRF. Specifically, the BIOES annotation model is used to annotate the predicted labels [39], as shown in Figure 3. Of these, entity and relationship types refer to Table 1 and Table 2. The label “1” indicates that the word belongs to the head entity of the triple. The label “2” indicates that the word belongs to the end entity of the triple. The label “OV” indicates an overlapping entity. An example of labeling of overlapping relationships related to textile and apparel production is shown in Figure 4. The meaning in Chinese corpus corresponding to the English is “The lining is cut into pieces on a cutting machine”. The types of labels for name entity recognition in textile and apparel production are summarized in

Table 3. Label types of name entity recognition in textile and apparel production.

Label Type	BIOES
Non-entity	O
General relationship entity	B-FS-1, B-SC-1, B-FA-1, …
Overlapping relationship entity	B-OV, I-OV, E-OV, S-OV, …

, whereas the abbreviations are referenced in Table 1.

Further, the labeled text data is fed into an entity-relationship extraction network to learn the sequence of labels from the corpus.

Specifically, the BERT encodes all sentences so that the vectors have contextual features. To efficiently capture the bidirectional semantic correlation of long sequences, the BiLSTM for contextual information extraction is given. The LSTM cell can be defined as follows.

$$f_t=\sigma \left(W_f\left[h_{t-1},x_t\right]+b_f\right)$$

(1)

$$i_t=\sigma \left(Wi\left[h_{t-1},x_t\right]+b_i\right)$$

(2)

$$o_t=\sigma \left(W_0\left[h_{t-1},x_t\right]+b_o\right)$$

(3)

$$C_t’=tanh\left(Wc\left[h_{i-1},x_t\right]+b_c\right)$$

(4)

$$C_t=f_t\odot C_{t-1}+i_t\odot C_t’$$

(5)

$$h_t=o_t\odot tanh\left(C_t\right)$$

(6)

where σ denotes the Sigmoid function. $b_f,b_i,b_o$, and $b_c$ denote time step bias, respectively. The output of $x_t$ at moment t is $h_t$. $o_t$ determines the direction and size of the output vector of the LSTM cell. $f_t$ denotes forgetful control. $i_t$ denotes input control. $C_t$ denotes the output state retained after passing the forgetful control. The control weight of the input oblivion is $W_i$. The control oblivion information weights are $W_f$. The weight of the control output is $W_0$. The weight of the control memory message is $W_c$.

The weight vector r_i given by the attention mechanism to the information in the sentence is shown below.

$$r_i={\sum }_{j=1}^N b_{\mathit{ij}}·y_i$$

(7)

where the sequence of sentence features extracted by BiLSTM is denoted as y_i. b_ij denotes the probability distribution of the word d_i as follows.

$$b_{ij}=\frac{\mathit{exp}(f\left(d_i,d_j\right))}{{\sum }_{k=1}^N\mathit{exp}(f\left(d_i,d_k\right))}$$

(8)

where d_i and d_j denote the i^th word and j^th word of a sentence. $f\left(d_i,d_k\right)$ denotes the similarity between the i^th word d_i and any word d_k in the sentence. The total number of words is N.

After that, the vectors are decoded and classified by the conditional random fields (CRFs). The CRF layer uses a state transfer function p(y|x) described as follows:

$$p(y|x)=\frac{\mathit{exp}(score\left(x,y\right))}{Z\left(x\right)}$$

(9)

$$Z\left(x\right)=\sum _{y`}\mathit{exp}(score\left(x,y`\right))$$

(10)

$$score\left(x,y\right)={\sum }^{ }Emit\left(x_i,y_i\right)+Trans\left(y_{i-1},y_i\right)$$

(11)

where Z(x) is a normalization factor, Trans denotes the transfer matrix of the CRF. Emit denotes the prediction probability. A higher score (x, y) means more accurate prediction. Thus, the following equation gives the computational description of the reliable output.

$$y^{*}=argmax(\mathit{log}(soft\mathit{max}(score\left(x,y\right))))$$

(12)

where y* denotes the predicted label of the output.

4. Bottleneck Prediction for Workstations in Textile and Apparel Production Based on the TKG-LSTM Model

Based on the knowledge modeling and graph construction of the massive data from the textile and apparel production process in Section 3, the static data and time-series data are linked by a knowledge graph named TKG4TA. This provides a knowledge base for detecting the bottleneck workstations. The reason is that the production cell needs to respond to a task passed from the upstream workstation and the task required by the downstream workstation. However, the location of production bottlenecks is usually non-static. Moreover, there are various manufacturing factors associated with bottleneck prediction. Hence, a real-time bottleneck prediction model for textile and apparel production workstations based on temporal knowledge graph–long short-term memory (TKG-LSTM) is designed, as shown in Figure 5. The model integrates the network properties of CompGCN and LSTM. CompGCN is a multi-relational graph convolutional network which can capture the dependencies of bottleneck-related static or time-serial manufacturing knowledge in TKG4TA. LSTM can capture the long-term dependencies in the production process, which leads to a better prediction.

Specifically, the inputs are subgraphs in TKG4TA, including static data and time-series data of the textile and apparel production process. This involves a close relationship with the knowledge nodes of “Human-Machine-Material-Method-Environment” in the production workshop system, including variables describing the state of semi-finished apparel products, variables describing the state of equipment, and variables describing the processing methods in the workshop.

Then, the subgraph in TKG4TA is encoded using CompGCN [40] to embed the representation of manufacturing data features, including the convolution and aggregation of subgraphs of TKG4TA. Further, after aggregation by the graph convolutional network, LSTM is used to learn the long-term dependence state features in the production process. Moreover, attention weights are added to make different state features play different roles in the bottleneck prediction. Finally, the bottleneck prediction for different workstations is executed by computing the updates.

To facilitate the analysis of the bottleneck workstation, a definition describing the basic elements of the workshop is given, as shown in

Table 4. Definition of basic elements in the textile and apparel production workshop.

Symbolic	Instruction
S = {S1, S2, …, Sk, …, Sn}	Collection of semi-finished products
D = {D1, D2, …, Dk, …, Dn}	Collection of devices
Bi	Buffer before Di
BTijk	The moment that semi-finished product i arrives at the buffer when it is to be machined in the jth process.

. Bottleneck workstations BN_k are those with the highest long-term utilization or those whose work-in-process inventory in the upstream buffer is the largest of all buffers in the system.

PWT denotes the sum of the waiting times of textile and apparel semi-finished products in the buffer area B_k of station D_k at time t. The larger the PWT, the weaker the ability to respond upstream. In addition, NWT denotes the sum of the waiting times for textile and apparel semi-finished products from station D_k′ in the buffer of station D_k′ at time t. The larger the NWT, the greater the ability to supply downstream. Thus, the workstation bottleneck BN_k at time t in the device D_k is described as follows.

$$PWT=\underset{i\in B_k}{{\sum }^{ }}\left(t-B{T}_{ijk}\right)$$

(13)

$$NWT=\underset{k^{\prime }\in D }{{\sum }^{ }}\underset{i\in B_k}{{\sum }^{ }}\left(t-B{T}_{i(j+1)k^{\prime }}\right)$$

(14)

The textile production unit needs to respond to both the tasks delivered by the upstream workstation and the tasks required by the downstream workstation. Therefore, the bottleneck value is defined as the negative value of the comprehensive ability of a production cell to satisfy upstream and downstream tasks. The cell with the largest bottleneck value is the bottleneck workstation. Bottleneck workstation BN_k is given in the following.

$$B{N}_k=-\left(-PWT+NWT\right)$$

(15)

Meanwhile, to further realize the design idea of the bottleneck prediction model of textile and garment production workstations based on TKG-LSTM, its key descriptions are given as follows.

(1)

GCN-based node embedding in the subgraph of TKG4TA

For predicting the bottleneck status of each workstation, it is first necessary to extract the bottleneck features of each workstation node, as shown in the bottom layer in Figure 5. The features related to the workstation bottleneck in TKG4TA are embedded. GCN is applied to deliver the feature embedding of each workstation node to the edge node to aggregate the feature vectors and compute the embedding features of the target workstation. The nodes and relationships are described in detail as follows.

$${\mathit{h}}_e^{(l+1)}=\sigma ({\displaystyle \sum _{i\in N_e}\frac{1}{{\lambda }_e}}{W}^{(l+1)}{\mathit{h}}_e^{(l)})$$

(16)

$${\mathit{h}}_r^{k+1}={\mathit{W}}_{\mathrm{rel}}^k{\mathit{h}}_r^k$$

(17)

$$\mathcal{G}\left(E_t\right)={\mathit{h}}_{e(t)}\oplus {\mathit{h}}_{r(t)}$$

(18)

${\mathit{h}}_e^{\left(l+1\right)}$ indicates the subgraph feature aggregation of the l + 1th layer with e_j as the key node in the TKG4TA. N_e is the first-order neighborhood of node e_j. ${\mathit{h}}_r^k$ indicates the relationship feature of the k layers. λ_e indicates a normalization constant. W_rel and ${\mathbf{W}}^{(l+1)}\in {ℝ}^{d(l+1)\times d(l)}$ denote a transformation matrix and a layer-specific trainable weight matrix, respectively. Then, the subgraph encoding $\mathcal{G}\left(E_t\right)$ in the TKG4TA can be represented by concatenating ${\mathit{h}}_{e\left(t\right)}$ and ${\mathit{h}}_{r\left(t\right)}$ at time t.

(2)

Long-term relationship feature representation in the workstation production process

Bottleneck workstations are also influenced by their historical station observations and future production planning. These factors exhibit time dependence and inter-correlation. LSTM is utilized to capture these properties after encoding by the GCN. A detailed description is given in the corresponding equations in Section 3. After encoding based on the LSTM, attention weights α are added to give different weights to different states to characterize the importance of each state, so that each state plays a different role in the prediction.

$$\alpha =softmax\left(h_{t-k}^l\ast h_t^l\right)$$

(19)

$$h_{t-k}^l=\alpha \ast h_{t-k}^l$$

(20)

$$y_{BN}=W\ast h_t^l+b$$

(21)

where l denotes the number of layers of the LSTM. W and b are the weights and biases of the fully connected layer. $h_t^l$ is the output of the last time step in the last layer of the TKG-LSTM. Finally, the bottleneck value y_BN of each production unit is output through the fully connected layer.

(3)

Training updates to the model

The goal is to minimize the error between the true bottleneck time and the predicted value time on the production line. $y_{BN}$ and ${\widehat{y}}_{BN}$ denote the true time and the predicted time, respectively. The loss function of the model is described as follows.

$$loss=\Vert y_{BN}-{\widehat{y}}_{BN}\Vert +\lambda L_{reg}$$

(22)

where $\lambda$ is the hyperparameter. L_reg is the L₂ regularization to avoid overfitting problems.

5. Case Study

This section demonstrates that the proposed models can utilize textile and apparel production data to construct a knowledge graph, and then predict the workstation bottlenecks in production. The partial workstations (WS) of a production line for shirts are selected as a case, as shown in Figure 6. The analytic details will be discussed in the follow-up.

5.1. Knowledge Extraction and Knowledge Generation for Textile and Apparel Production

The knowledge extraction for building the TKG4TA mainly comes from a large number of textile and apparel process documents and equipment operation and maintenance manuals, as well as the records from the information system log. Data cleaning is performed to remove the sentences without entity and relationship. To ensure data quality, several persons with empirical knowledge are invited to perform manual screening, labeling, and proofreading. Then, 1682 labeled sentences are obtained. On average, each sentence is about 63 words. Among them, the ratio of training set and validation set is 8:2, and the details of the dataset are shown in

Table 5. Statistics on data labeling in textile and apparel production.

Label Type	Training Set	Validation Set	Subtotal
label	28,569	7142	35,711
common label	11,152	2788	13,940
overlapping label	432	108	1586

.

To analyze the effectiveness of the various modules of the network for the proposed joint entity-relationship extraction model, BiLSTM-CRF [41], Word2Vec-BiLSTM-CRF [42], and BERT-BiLSTM-CRF [43] are employed as baseline. Key parameter settings for the models include a batch size of 64, a dropout of 0.3, a learning rate of 0.0015, and an epoch of 80. Adam is used as the optimizer. In addition, P (Precision), R (Recall), and F1 scores are used as the metrics for knowledge extraction. Finally, the performance comparison of the four models on the named entity recognition and relationship extraction tasks are summarized as shown in Figure 7 and Figure 8, respectively.

The comparison of the four models shows that the single named entity recognition and the BERT-BiLSTM-Attention-CRF have achieved better results and the difference is not significant. The F1 value of the proposed method for entity recognition is up to 80.3%, and the F1 value for relationship extraction is up to 50.6%. Compared with the BERT-BiLSTM-CRF model, the introduction of the attention mechanism increases the model by 7.2% in entity recognition and 5.4% in relationship extraction in terms of F1 value. Moreover, the BERT-BiLSTM-CRF achieved 3.6% more in F1 values than the Word2Vec-BiLSTM-CRF model on both tasks. This indicates that the embedding accuracy of BERT is greater than that of Word2Vec. Compared with the F1 value of the BiLSTM-CRF-based model, the performance of the model proposed in this paper in the relationship extraction task is improved by 11.2%. This indicates that the joint entity-relationship extraction method can effectively utilize the semantic relationship between the entity recognition and relationship extraction tasks to obtain overlapping relationships. Further, the generated triples are checked by textile professionals to safeguard the quality of the knowledge graph as much as possible.

After obtaining the triples by the joint entity-relationship extraction model, the various static and time-series data stored in the information system are mapped to the knowledge graph based on the ontology model. An example of the time-series knowledge graph for shirt production is constructed as shown in Figure 9. The relevant knowledge nodes and relationships in the shirt manufacturing process are displayed. Each entity includes static and dynamic attributes of the workstation in shirt manufacturing, which play a key role in keeping the production line running stably. The time-series knowledge graph of shirt production is stored in the neo4j graph database, which will provide the data basis for the bottleneck analysis model later.

5.2. Knowledge Graph-Based Bottleneck Identification in Textile and Apparel Production

Taking the partial flow of production lines for shirt manufacturing in Figure 6 as the subject of bottleneck analysis, its production line has 38 workstations with 101 workflows. We assigned 80% of the workstation data to the training set and 20% of the workstation data to the validation set for bottleneck prediction. The production line produces several major types of shirts each day, where production lines include both serial structures and parallel structures. Each workstation may contain at least one or several processes. The buffer capacity between workstations is recorded and used as input to associate the time-series knowledge graph. The statistics of the event log data from a shirt production line over a period of time are shown in

Table 6. Event log for a shirt production workstation.

Order ID	Process ID	Process Name	Start Time	End Time	Workstation ID	Device Name
1	1	Cut the left front panel	08/24:08:00	08/24:08:10	WS1	Fabric cutting machine
…	2	Cut the right front panel	08/24:08:13	08/24:08:21	WS 1	…
	3	Cut back piece	08/24:08:23	08/24:08:29	WS 2	…
	…	…	…	…	…	…
	46	Sew front chest pocket	08/24:11:05	08/24:11:23	WS 11	Flatbed sewing machine
	47	Sew front door plaque	08/24:11:26	08/24:11:32	WS 11	…
	49	Iron the hem	08/24:11:36	08/24:10:49	WS 12	Ironing table
	50	Cut sleeve lining	08/24:11:53	08/24:12:12	WS 13	Fabric cutting machine
	51	Lockstitch lapels	08/24:12:15	08/24:12:28	WS 14	Direct drive button sewing machine
	…	…	…	…	…	…
	79	Sew hanging	08/24:16:34	08/24:16:46	WS 22	Flatbed sewing machine
	80	Iron front pockets	08/24:16:51	08/24:17:08	WS 23	Ironing table
	81	Sew pocket lining	08/24:17:12	08/24:17:25	WS 24	Flatbed sewing machine
	82	Lockstitch cuffs	08/24:17:30	08/24:17:47	WS 25	Direct drive button sewing machine
	…	…	…	…	…	…
	90	Iron front door	08/25:09:41	08/25:09:52	WS 31	Ironing table
	91	Sew sleeve lining	08/25:09:58	08/25:10:14	WS 32	Flatbed sewing machine
	92	Iron collars	08/25:10:20	08/25:10:36	WS 33	Ironing table
	…	…	…	…	…	…
	99	Lockstitch inner pocket	08/25:13:32	08/25:13:44	WS 36	Direct drive button sewing machine
	100	Front body shaping	08/25:13:49	08/25:13:59	WS 37	Shirt shaping machine
3852	101	Back body shaping	08/25:14:06	08/25:14:21	WS 38	…

.

(1)

Metrics strategy

The root mean square error (RMSE) and the mean absolute error (MAE) are used as evaluation indicators for the bottleneck prediction model, which are described as follows.

$$RMSE=\sqrt{\frac{1}{N}\sum _{i=1}^N{\left({\widehat{y}}_{BN}-y_{BN}\right)}^2}$$

(23)

$$MAE=\frac{1}{N}\sum _{i=1}^N \mid \left({\widehat{y}}_{BN}-y_{BN}\right)\mid$$

(24)

(2)

Experimental parameter selection

The key experimental parameters for the bottleneck prediction model are shown in

Table 7. The key parameters for the bottleneck prediction model.

Parameter	Value	Instruction
gcn_dim	200	Number of hidden cells in GCN
gcn_layer	2	The number of layers in GCN
hidden_size	200	Hidden size in LSTM
λ	0.005	Regularized hyperparameters
batch_size	64	Number of batch size
learning_rate	0.0015	Model learning rate
epoch	1200	Number of epochs

. It is mainly divided into the parameter settings of GCN and LSTM. First, the dimension of the graph embedding is 200. The number of layers in GCN is 2. Then, the hidden size in LSTM is 200. The learning rate of the model is 0.0015 and the epoch is 1200. Figure 10 gives the curve of the loss function of the proposed bottleneck prediction based on TKG-LSTM. It can be seen that it converges around 600 epochs.

(3)

Experimental analysis

To measure the effectiveness of the TKG-LSTM in bottleneck prediction, LSTM and GCN are chosen as baseline, incorporating the bottleneck information including knowledge from TKG4TA and the event logs in Table 6. Then, the TKG-LSTM and the selected baseline in the bottleneck prediction task spanning is set to 30 min and 60 min. The comprehensive performance comparison is summarized in

Table 8. Performance comparison of the three bottleneck prediction models.

Algorithms\Metrics	30 min		60 min
	RMSE	MAE	RMSE	MAE
LSTM [44]	2.3352	1.8332	2.5581	2.0081
GCN [40]	2.5136	2.0486	2.7648	2.2533
TKG-LSTM (Ours)	2.1365	1.6942	2.2035	1.7474

.

As a result, it can be concluded that the TKG-LSTM model can better integrate historical information with multi-features compared to the LSTM and GCN, resulting in a better bottleneck workstation prediction. Among the proposed approaches, bottleneck prediction with time-series features has a better performance in modeling temporal features. In the bottleneck prediction tasks with both 30 min and 60 min spans, the performance of the TKG-LSTM is improved compared to the use of the GCN model or the LSTM model alone for bottleneck prediction. Also, the prediction accuracy is improved by 8.2% and 14.92% in the MAE metrics, respectively. This may be because those models only consider a single bottleneck state feature. In contrast, the proposed TKG-LSTM model can capture temporal knowledge features from the production workstation state data. TKG-LSTM is suitable for processing temporal knowledge graphs due to combining the respective advantages of GCN and LSTM, making the accuracy of the proposed model higher than that of LSTM or GCN on their own.

To further discuss the details of bottleneck workstation prediction, the results are given for several typical workstations based on Figure 6 for shirt manufacturing, as shown in Figure 11. Specifically, 600 time steps are selected to analyze the bottleneck prediction results. Figure 11a shows the variation of bottleneck values at WS 6. The blue and red curves indicate the true and prediction values, respectively. It can be seen that the curves of the predicted value and the true value of WS 6 are in good match. Moreover, the variations of bottleneck values are larger and more negative. According to Equation (15), it can be observed that this is because the shorter waiting time of the fabric to be processed in the cache area makes the waiting time of the fabric from WS 6 in the downstream sewing station longer. This also indicates that the fabric sewing capacity at WS 6 is relatively high. Therefore, it is quite unlikely that WS 6 will be a bottleneck workstation at the moment. Figure 11b shows the variation of bottleneck value at WS 15, which is also relatively consistent with the curve of the predicted and true values. Moreover, the bottleneck value fluctuates more frequently and remains high for a long time. This is because the waiting time for the sewing of fabrics in the buffer is long, which makes the waiting time for the fabrics from WS 15 in the downstream sewing station shorter. This also indicates that the fabric sewing capacity at WS 15 is weak. Thus, there is a higher probability that WS 15 will become a bottleneck workstation. It is necessary to pay attention to WS 15 in the shirt sewing process. These discussions also reflect the causes of potential bottlenecks in upstream and downstream production workstations for the textile industry. Based on the cause analysis of workstation bottlenecks, it is beneficial to enhance the efficiency in guiding and recommending subsequent textile production operations.

6. Conclusions and Future Work

In this paper, we propose a knowledge-graph-embedded time-serial-data-driven bottleneck analysis of textile and apparel production processes. Firstly, a dynamic information association model in textile and apparel production is established to organize the static data and time-series data from a global level. Then, a textile-corpus-oriented knowledge extraction model is designed to form the triples, taking a large number of documents related to textile process and equipment operation and maintenance manuals as data resources. The comparison of the four models shows that the F1 value of the proposed model for entity recognition is up to 80.3%, and for relationship extraction is up to 50.6%. Moreover, TKG4TA includes the static and dynamic attributes of the workstation in shirt manufacturing. In addition, a temporal knowledge-driven production process bottleneck prediction model, TKG-LSTM, is presented by incorporating GCN and LSTM. Finally, taking the partial flow of production lines for shirt manufacturing as the bottleneck analysis subject, the TKG-LSTM performance in the MAE metrics is improved by 8.2% and 14.92% compared to the use of GCN and LSTM only, respectively. Finally, we analyze several typical workstations for bottleneck prediction. The analysis indicates that the proposed method can predict the different workstations based on real-time data and knowledge from shirt manufacturing, which helps workers find potential bottleneck workstations in a workshop.

Although the proposed methods show superior results in generating TKG4TA and predicting bottleneck workstations in the textile industry, they still have some limitations. Firstly, the hyperparameters involved in the TKG-LSTM are selected for suitable bottleneck prediction in the textile industry. The selection of hyperparameters may need to be optimized in the future to work for different prediction applications. Secondly, there is insufficient consideration of interference factors for bottleneck finding in textile and apparel production. Therefore, it is desired to extend the textile manufacturing knowledge entity and relationship types to apply complex relationship extraction and knowledge graph generation tasks.