Automated Structuring and Analysis of Unstructured Equipment Maintenance Text Data in Manufacturing Using Generative AI Models: A Comparative Study of Pre-Trained Language Models

Abstract

Manufacturing companies face significant challenges in leveraging artificial intelligence for equipment management due to high infrastructure costs and limited availability of labeled data for failures. While most manufacturing AI applications focus on structured sensor data, vast amounts of unstructured textual information containing valuable maintenance knowledge remain underutilized. This study presents a practical generative AI-based framework for structured information extraction that automatically converts unstructured equipment maintenance texts into predefined semantic fields to support predictive maintenance in manufacturing environments. We adopted and evaluated three representative generative models—Bidirectional and Auto-Regressive Transformers (BART) with KoBART, Text-to-Text Transfer Transformer (T5) with pko-t5-base, and the large language model Qwen—to generate structured outputs by extracting three predefined fields: failed components, failure types, and corrective actions. The framework enables the structuring of equipment management text data from Manufacturing Execution Systems (MES) to build predictive maintenance support systems. We validated the approach using a large-scale MES dataset consisting of 29,736 equipment maintenance records from a major automotive parts manufacturer, from which curated subsets were used for model training and evaluation. Our methodology employs Generative Pre-trained Transformer 4 (GPT-4) for initial dataset construction, followed by domain expert validation to ensure data quality. The trained models achieved promising performance when evaluated using extraction-aligned metrics, including exact match (EM) and token-level precision, recall, and F1-score, which directly assess field-level extraction correctness. ROUGE scores are additionally reported as a supplementary indicator of lexical overlap. Among the evaluated models, Qwen consistently outperformed BART and T5 across all extracted fields. The structured outputs are further processed through domain-specific dictionaries and regular expressions to create a comprehensive analytical database supporting predictive maintenance strategies. We implemented a web-based analytics platform enabling time-series analysis, correlation analysis, frequency analysis, and anomaly detection for equipment maintenance optimization. The proposed system converts tacit knowledge embedded in maintenance texts into explicit, actionable insights without requiring additional sensor installations or infrastructure investments. This research contributes to the manufacturing AI field by demonstrating a comprehensive application of generative language models to equipment maintenance text analysis, providing a cost-effective approach for digital transformation in manufacturing environments. The framework’s scalability and cloud-based deployment model present significant opportunities for widespread adoption in the manufacturing sector, supporting the transition from reactive to predictive maintenance strategies.

1. Introduction

1.1. Research Background and Objectives

With the advent of Industry 4.0, the digital transformation of manufacturing industries has been rapidly advancing. Particularly, innovative approaches such as Digital Transformation (DX) and AI Transformation (AX) are being actively promoted, accelerating the adoption of artificial intelligence technologies in manufacturing. Currently, various manufacturing AI use cases are being developed and implemented in manufacturing processes, including Digital Twin, Predictive Maintenance, Quality Control, Supply Chain Management, Inventory Management, and Energy Management.

However, most manufacturing AI use cases currently being pursued are concentrated on the utilization of structured data, while unstructured data utilization is limited to processing image data in quality control applications. For manufacturing companies to effectively utilize collected data from Programmable Logic Controller (PLC) systems, environmental sensors, vibration and power data, and vision data in manufacturing AI, several critical technical and economic challenges exist.

First, there are cost burdens associated with infrastructure construction for data collection and issues with securing specialized personnel. The installation of IoT devices including sensors and network infrastructure requires substantial initial investment costs, and securing specialized personnel is essential for operating and maintaining these systems. Second, even after successfully collecting structured and vision data, there is a fundamental problem of extremely low ratios of labeled data for product defects and equipment failures required for AI model application. Particularly for small and medium-sized manufacturing enterprises (SMEs) in Korea, which already maintain high process management levels through traditional innovation methodologies such as Just-In-Time (JIT), Six Sigma, and Lean Manufacturing, there are structural difficulties in securing defect and failure data necessary for AI model training.

Due to the realistic constraints mentioned above, AI adoption in manufacturing remains in its early stages, and the development of more effective manufacturing data utilization strategies and technical solutions is urgently needed. In this context, attention should be paid to other forms of data assets possessed by manufacturing companies. While existing manufacturing AI research has mainly focused on sensor data and image data, manufacturing companies possess vast amounts of unstructured text data scattered throughout their operations, including quality reports, work orders, equipment inspection records, failure history reports, operator logs, customer complaint reports, and supplier communication records. This text data exists in various forms, including hardcopy paper documents, electronic documents such as PDF and Word files, and text information stored within Enterprise Resource Planning (ERP) and Manufacturing Execution System (MES) systems. Particularly for manufacturing companies with long operational histories, the accumulation scale of such data is more extensive and can be viewed as important information assets containing compressed manufacturing experience and know-how. This text data contains contextual information and expert knowledge about manufacturing processes that are difficult to capture with structured data alone, making it an underutilized dataset resource that can provide new breakthroughs for manufacturing AI.

Recent studies have highlighted the growing potential of large language models (LLMs) for supporting data integration and decision-making in manufacturing environments. However, existing research primarily focuses on conceptual frameworks or assistant-level applications, while empirical studies that operationalize LLMs to transform unstructured manufacturing text into analyzable structured knowledge remain limited. Moreover, the lack of practical and operational frameworks that bridge theoretical LLM capabilities with real manufacturing data has been identified as a key research gap in the current literature [1].

Recent work by Werheid et al. applied large language models to support requirement interpretation and engineering decision-making in the design phase of the product development value chain [2]. Their approach employs retrieval-augmented generation (RAG) to mitigate hallucination and provide fact-based recommendations for equipment selection tasks. In contrast, the present study addresses a different stage of the manufacturing value chain, namely the production and operation phase. By integrating generative models with Manufacturing Execution Systems (MES), this study focuses on transforming unstructured operational texts, such as equipment maintenance and failure records, into structured knowledge that can be systematically analyzed and utilized within manufacturing processes.

In this context, the rapid development of Natural Language Processing (NLP) technology, along with the emergence of Generative AI and Large Language Models (LLMs), has made systematic analysis of such unstructured text data realistically feasible. This technological advancement presents new opportunities to complement existing data shortage problems and expand the application scope of manufacturing AI. Most importantly, through NLP and generative AI technologies, it becomes possible to convert tacit knowledge accumulated by process managers and operators through long experience into explicit knowledge.

Therefore, the objective of this study is to develop a structured information extraction framework that automatically converts unstructured equipment maintenance texts into predefined semantic fields using generative NLP models, enabling systematic analysis and utilization of manufacturing text data. Rather than treating the problem as a pure summarization task, this study formulates maintenance text processing as a slot filling problem that extracts key operational information embedded in free-text records.

Instead of proposing new model architectures, this study focuses on developing a practical system that utilizes existing generative models to convert unstructured manufacturing text data into structured knowledge, enabling the discovery of implicit operational know-how and latent issues in real industrial environments.

1.2. Research Scope and Methodology

This study targets equipment management text data within the MES operated by Company A, which produces automotive parts. The equipment management data in the MES consists of eight fields: ‘Management Number’, ‘Equipment Number’, ‘Equipment Name’, ‘Repair Date’, ‘Failure Content’, ‘Repair Location’, ‘Repair Time’, and ‘Repair Cost’. Among these, the ‘Failure Content’ field contains mixed information about: (1) which component failed, (2) what the failure types were, and (3) how the failure problem was resolved.

This paper performs a structured information extraction process that converts the unstructured text data stored in the ‘Failure Content’ field into three predefined semantic fields: ‘Failed Components’, ‘Failure Types’, and ‘Corrective Actions’. The remaining eight fields, excluding the ‘Failure Content’ field, all consist of structured data, enabling the development of an analysis service platform capable of structured data analysis for a total of 11 fields after extraction and structuring process of ‘Failure Content’.

The overall structure of this study is as follows. Section 2 briefly examines the concept and application areas of NLP and reviews generative sequence-to-sequence models that provide the technical foundation for structured information extraction from unstructured text. It also analyzes the Transformer architecture underlying the BART, T5, and Qwen models adopted and evaluated in this study, along with the characteristics of each model, and reviews related research in the generative text processing and information extraction literature from the perspective of generative model architectures applicable to information extraction tasks. Section 3 analyzes the detailed content and characteristics of the unstructured text data targeted in this study and presents the entire pipeline from data collection through AI model construction to final web services. Section 4 proposes a method for constructing training datasets using the GPT-4 API based on collected unstructured text data. Using the constructed dataset, fine-tuning is performed on pre-trained models based on Transformer architecture (BART, T5) and a decoder-only Large Language Model (Qwen). Section 5 conducts performance evaluation using extraction-aligned metrics, including exact match and token-level precision, recall, and F1-score, to assess field-level extraction accuracy, while ROUGE scores are reported as supplementary indicators of lexical overlap. Section 6 proposes a method for constructing structured databases suitable for final analysis using industry-specific terminology dictionaries and regular expressions for data extracted and structured through generative models (BART). It also presents platform development plans capable of providing services such as time series analysis, correlation analysis, frequency analysis, and variance analysis using the converted structured data. Finally, Section 7 proposes conclusions and future research directions.

2. Literature Survey

2.1. Overview of Natural Language Processing

Natural Language Processing (NLP) is a core area of artificial intelligence (AI) that bridges the communication gap between humans and machines, enabling humans to interact with machines using natural language while allowing machines to effectively process and understand such linguistic inputs [3,4]. Modern voice interfaces such as Amazon Alexa, Google Assistant, and Apple Siri represent successful real-world implementations of breakthrough developments in the NLP field [3].

NLP is extensively applied to various language processing tasks including text summarization, semantic analysis, document classification, question-answering systems, and natural language inference [3]. These applications play important roles not only in academic research but also in practical industrial sectors. Recently, chatbot technologies for worker assistance and problem-solving have begun to be implemented in manufacturing, demonstrating the continuous expansion of NLP technology’s industrial application scope. Particularly in the text summarization field, the effective introduction of Transformer architecture and attention mechanisms has played a decisive role in dramatically improving technical performance [5], and these advances subsequently enabled generative models to be applied to broader tasks such as structured information extraction, significantly enhancing the ability to extract and condense core information from large-scale text data.

Due to these technological advances, recent NLP models are achieving remarkable accuracy, often demonstrating excellent performance without additional training or with minimal fine-tuning [3]. However, these developments present new challenges in selecting the most suitable model for specific NLP tasks, making appropriate model selection based on the characteristics and requirements of the intended NLP task essential for optimal results [3]. Particularly for structured information extraction from manufacturing text data, which is the target of this study, more careful model selection is required considering manufacturing industry specificities such as personnel and cost constraints, and computing cost burdens from GPU usage during training and service operations. Therefore, an optimized approach considering the balance between performance and practicality is more important in manufacturing environments.

2.2. Text Summarization

Although the primary task addressed in this study is structured information extraction rather than abstractive summarization, text summarization research is reviewed because modern generative sequence-to-sequence models originally developed for summarization provide the technical foundation for generative extraction and slot filling tasks. In particular, these models demonstrate strong capabilities in semantic understanding and controlled text generation, which are directly applicable to extracting predefined fields from unstructured maintenance records.

Text summarization is the process of generating concise, accurate, and fluent abstracts from extensive documents, representing a technology that automatically extracts and compresses core content from extensive text materials without human intervention [6]. The core objective of this technology is to generate summaries that are easy to understand and natural while preserving the important meanings and contexts contained in the original text [6]. Modern society is experiencing exponentially increasing data generation speeds alongside accelerated digital transformation, with various forms of text data such as news articles, product reviews, and literary works available in online environments accumulating geometrically [4]. Particularly, massive datasets like increasing online reviews are causing difficulties for users to select meaningful information, and information overload phenomena are intensifying [7]. In manufacturing sites, vast amounts of unstructured text data including equipment inspection records and quality reports are continuously generated, but many managers are unaware of their existence and value. This results in important operational knowledge remaining unutilized, necessitating technical alternatives that can structure and analyze such data. In this environment, technology capable of automatically providing accurate and timely summary information is gaining attention as a key solution for enhancing efficient knowledge consumption and information accessibility.

While humans naturally tend to summarize key points when encountering long texts [8], there are limitations in manually processing vast amounts of information. Consequently, Automatic Text Summarization (ATS) technology has been developed [8], with rapid advances in Machine Learning, Artificial Intelligence (AI), and Natural Language Processing (NLP) technologies contributing significantly to improving summarization technology performance [9]. Automated summarization systems enhance spatial and temporal efficiency compared to traditional manual summarization [4,9], reduce data storage burdens, and enable rapid access to core information and derivation of important insights. Particularly in applications such as review summarization, extracting and providing only core information to users enhances information search efficiency and can even support consumer decision-making processes [7].

Text summarization is not simply reducing the number of sentences, but rather a high-dimensional language processing technology that identifies and preserves complex semantic relationships and core information embedded in original texts. This remains a continuous challenge in Natural Language Understanding and Information Retrieval fields [10], and has been successfully applied to various domains including news article summarization, review summarization, and novel summarization [11,12]. This broad applicability demonstrates that text summarization technology provides substantial value in text mining and data analysis, going beyond being merely a sentence summarization tool to become an important language technology.

In the natural language processing field, text summarization is broadly classified into two main approaches: extractive summarization and abstractive summarization [13,14]. Text summarization technology has undergone continuous evolution since early research in 1958, with statistics-based extractive summarization using TF-IDF [15] or Bayesian models [16] being mainstream initially.

Extractive summarization generates summaries by scoring and selecting important sentences or phrases from original texts and combining them [14,17,18]. While this approach is easy to implement and has high accuracy by using original text expressions directly, it has limitations in that natural connections between contexts may be lacking, potentially degrading summary consistency.

Subsequently, with the introduction of machine learning algorithms [19] and Bayesian learning models capable of analyzing correlations between words [16], understanding of semantic structures within texts improved, increasing interest in abstractive summarization approaches. Abstractive summarization is an approach that understands original texts and generates new sentences and vocabulary based on this understanding to construct summaries [20], offering the advantage of providing cohesive summaries that encompass the meaning of entire documents, similar to human summarization methods.

The development of abstractive summarization technology was accelerated by the development of LSTM networks for sequential data processing [21] and the emergence of encoder–decoder structures utilizing them [21,22]. With the introduction of attention mechanisms [23,24], the ability to simultaneously consider all words within sentences became possible, dramatically improving contextual understanding capabilities.

These structural improvements ultimately led to the Transformer architecture [24], and based on this, powerful pre-trained language models such as BERT (for representation learning) [25], PEGASUS [26], UNiLM [27], and GPT (for generative summarization) [28] emerged, enabling text summarization technology to achieve significant breakthroughs.

While extractive summarization continues to maintain advantages of computational efficiency and implementation ease, abstractive summarization is widely applied in fields requiring advanced language processing capabilities due to its ability to provide richer and more natural expressions. Such abstractive summarization demonstrates strengths in effectively identifying and structuring complex semantic relationships embedded in unstructured data, particularly in fields dealing with professional and domain-specific text data such as manufacturing. Accordingly, although the primary task addressed in this study is structured information extraction (slot filling) rather than classic abstractive summarization, we leverage Transformer-based generative models originally developed for abstractive summarization as the core architecture for generating structured outputs. Specifically, this study applies BART, T5, and Qwen models to equipment failure-related maintenance texts in manufacturing to extract three predefined fields: failed components, failure types, and corrective actions.

Table 1. Recent Studies on Text Summarization.

Papers	Dataset	Applied Models	High-Performance Model
Automated Text Summarization of News Articles [9]	Internet news articles	BART, T5	BART
Abstractive Text Summarization Using BART [6]	Internet news articles	BERT, Roberta, T5, BART	BART
An Analysis of Abstractive Text Summarization Using Pre-trained Models [8]	Internet news articles	PEGASU, T5, BART	BART
Automated News Summarization Using Transformers [4]	Internet news articles	BART, T5, PEGASUS	T5
Comparative Analysis of Summarization Methods for Skin Care Product Reviews: A Study on BERT, BART, and T5 Models [7]	Product review data	BERT, BART, T5	BART
Automatic text summarization using transformer-based language models [29]	Internet news articles	BART, T5	BART
Exploring Abstractive Text Summarization for Podcasts: A Comparative Study of BART and T5 Models [5]	Podcasts	BART, T5	BART
Enhancing Product Design through AI-Driven Sentiment Analysis of Amazon Reviews Using BERT [30]	Product review data	BERT, T5	BERT
Exploratory analysis on the natural language processing models for task specific purposes [3]	Internet news articles	BERT, BART, T5	BART

shows major research trends in the recent text summarization field. Analysis results confirm that BART and T5 models are being utilized as core techniques for text summarization tasks across various domains. First, examining domain-specific application status, internet news articles appeared as the most common research subject, which is attributed to news texts having relatively structured formats suitable for verifying summarization model performance. Additionally, studies utilizing product review data are increasing, reflecting the growing importance of consumer opinion analysis alongside the expansion of e-commerce platforms. Second, regarding AI model utilization patterns, the BART model showed the highest utilization rate, being adopted in 7 out of 9 studies, while the T5 model was utilized in 6 studies. Particularly, multiple studies conducting comparative analyses of both models are confirmed, which can be interpreted as academic attempts to verify performance differences between different architectures (encoder–decoder structure of BART vs. text-to-text transformation structure of T5). Recently, limited attempts are being made to utilize generative language models with diverse domain adaptability through large-scale pre-training, such as LLM family Qwen models, for summarization and structuring tasks, but academic research is rarely attempted. Third, from a research methodology perspective, most studies adopt approaches of fine-tuning pre-trained models to specific domains. This is analyzed as leveraging the advantages of transfer learning that can achieve effective performance even with limited domain-specific datasets.

However, existing studies are mainly concentrated on general text summarization, with relatively insufficient research simultaneously performing structured information extraction and slot filling. Particularly, research aimed at generating structured data such as failed components, failure types, and corrective actions from unstructured maintenance texts using not only BART and T5 but also LLMs like Qwen in specialized domains such as equipment management in manufacturing sites is rarely found. More importantly, most prior studies treat summarization as an end goal, whereas limited attention has been paid to transforming unstructured text into structured fields that can be directly utilized for quantitative analysis and downstream manufacturing analytics. This research gap highlights the distinctiveness of this study, which has clear differentiation from existing research in that it performs structured information extraction and data transformation enabling quantitative analysis beyond simple summarization.

2.3. Transformer Models

The Transformer network adopts an attention-based architecture without relying on recurrent structures, enabling effective modeling of global dependencies and parallel computation while preserving word order through positional encoding [4,24]. It consists of encoder–decoder layers connected by multi-head attention and feed-forward networks, allowing efficient contextual representation learning [24].

Building on this architecture, Transformer-based pre-trained language models such as BART, T5, and GPT-3 have demonstrated strong performance in abstractive text summarization [5] and have increasingly been adopted as general-purpose generative backbones for structured information extraction and slot filling tasks. Through large-scale pre-training and task-specific fine-tuning, these models provide transferable semantic representations that are well suited for downstream extraction tasks [31,32,33].

2.4. BART Model

BART (Bidirectional and Auto-Regressive Transformers) is a Transformer-based sequence-to-sequence model proposed by Lewis et al., which combines BERT’s bidirectional encoder with GPT’s auto-regressive decoder [31]. The model is pre-trained using a denoising autoencoder approach that restores corrupted inputs such as sentence permutation or span masking [6,29]. The encoder compresses input semantics using multi-head attention, while the decoder generates sentences auto-regressively based on the encoded representations. After pre-training, BART is fine-tuned for downstream tasks including summarization, translation, and document generation, demonstrating strong performance in text summarization.

BART is trained by optimizing reconstruction loss and can capture original content from large-scale unlabeled data to generate coherent summaries, making it widely used as a representative pre-trained model in the abstractive text summarization field, and increasingly adopted for generative information extraction tasks in domain-specific applications. The BART-base model contains approximately 140 million parameters.

2.5. T5 Model

T5 (Text-to-Text Transfer Transformer) is a Transformer-based model proposed by Raffel et al. [34] that formulates all natural language processing tasks within a unified text-to-text paradigm, where both inputs and outputs are represented as text strings. Unlike BERT-family models that output class labels or text spans, T5 converts diverse tasks into a common generation framework [9]. The model is based on transfer learning, being pre-trained on large-scale text data and then fine-tuned for various downstream tasks such as summarization [4,35]. Pre-training is conducted using unsupervised objectives such as span corruption, and the encoder–decoder architecture supports multi-task learning.

The encoder processes inputs through self-attention and feed-forward layers, while the decoder generates outputs auto-regressively using cross-attention mechanisms [34]. The T5-base model further enables flexible task adaptation through task-specific prefixes and relative scalar embeddings [8]. Owing to this unified design, T5 has been widely applied to abstractive summarization tasks, demonstrating effective semantic understanding and coherent text generation [29].

2.6. Qwen Model

Qwen (Qianwen) is a family of Transformer-based Large Language Models (LLMs) proposed by Alibaba Cloud, designed as pre-trained generative models for a wide range of natural language processing tasks [36]. Qwen adopts a decoder-only Transformer architecture and is pre-trained on trillion-token scale datasets including web text, domain-specific documents, and code, enabling strong adaptability to general text generation tasks such as question answering, dialogue, translation, reasoning, and summarization [37]. Although Qwen is not specifically optimized for summarization like BART or T5, its large-scale pre-training on diverse domains allows rapid domain adaptation and high performance even with limited fine-tuning data. These characteristics indicate that Qwen can be effectively applied not only to general text generation but also to summarization and structured information extraction tasks in specialized domains such as manufacturing equipment maintenance.

3. System Framework

3.1. Characteristics and Analysis of Target Dataset

This study analyzed equipment management text data stored in the Manufacturing Execution System (MES) operated by Company A, which produces core automotive components. The total collected equipment management data comprises 29,736 records, of which 3801 records were selected for model training and testing based on data quality and completeness considerations. The data consists of eight fields including “Management Number, Equipment Number, Equipment Name, Repair Date, Failure Content, Repair Location, Repair Time, and Repair Cost,” where the “Failure Content” field contains unstructured text data written by equipment managers with mixed information about failed components, failure types, and corrective actions.

Analysis of the “Failure Content” field reveals that the text exhibits freely written sentence structures without specific formats or rules, characterized by multiple meanings coexisting within single sentences. For example, as shown in the example presented in Figure 1, the sentence “Vertical motion motor brake noise occurrence lining price adjustment AS replacement scheduled” contains failed components (“motor brake”), failure types (“noise occurrence”), and corrective actions (“lining price adjustment,” “AS replacement scheduled”) described together, making clear information separation difficult. Such formats present limitations for direct application of conventional structured data analysis techniques.

However, this unstructured text data possesses an inherent structure that can be separated and structured into three main components: failed components, failure types, and corrective actions. Therefore, through structuring these data, various analyses become possible. For instance, failure frequency analysis by failed components, representative failure type pattern analysis, and time series analysis of failure trends for specific components can be performed, along with correlation analysis between equipment and failure types or between equipment and failed components. Furthermore, by linking with existing structured management items such as “Management Number,” “Equipment Number,” “Repair Date,” and “Repair Time,” multi-dimensional statistical analysis and integrated insights for process improvement are expected to be achievable.

Ultimately, when unstructured equipment management text data is systematically structured and continuously accumulated, it is anticipated that it can be utilized for advanced predictive analyses such as equipment failure prediction, predictive maintenance, and maintenance resource optimization in the future.

3.2. Pipeline Configuration

This study proposes an integrated system for automatically extracting and structuring key information from unstructured equipment management text data generated in manufacturing sites using generative models. To achieve this, a six-stage research pipeline was designed encompassing the entire workflow from data collection through generative artificial intelligence (AI) model construction, structured database development, to final web service implementation. The overall system structure proposed in this study is shown in Figure 2.

In manufacturing environments, maintenance records are written using non-standardized, domain-specific terminology that varies across sites and operators, making direct rule-based extraction difficult. To address this challenge, the proposed framework separates the extraction and standardization stages. Specifically, generative models are first used to extract failure-related information while preserving the original linguistic expressions, and domain-specific dictionaries are then applied in a subsequent step to normalize the extracted terms into structured representations. Traditional sequence labeling approaches, such as BIO tagging, require token-level annotations, which are costly and difficult to obtain at scale in manufacturing environments, while direct prompting of large language models in zero- or few-shot settings, although flexible, often suffers from response variability and deployment constraints in on-premise industrial systems.

Among various LLM options, Qwen was selected in this study because it enables reproducible fine-tuning and controlled deployment in on-premise industrial environments. In manufacturing contexts, practical requirements such as data confidentiality, organizational security policies, and cost considerations often limit reliance on proprietary or API-only LLMs. In contrast, Qwen provides an open-access, instruction-tuned LLM configuration that can be locally fine-tuned and evaluated within a consistent experimental setup, making it a suitable choice for manufacturing text extraction and structuring tasks. In addition, Qwen is suitable for processing Korean manufacturing maintenance texts, supporting its applicability to Korean-language industrial data.

3.2.1. Stage 1: Training Dataset Construction and Expert Validation

In the first stage, raw equipment management text data was refined using GPT-4-based large language models, and the quality and reliability of the training dataset were ensured through domain expert validation of the extracted structured data. This process functions as a crucial preprocessing stage that directly impacts model training performance.

3.2.2. Stage 2: Generative AI Model Development

In the second stage, representative generative models BART, T5, and Qwen were applied to develop generative information extraction models for failure content text, and extraction performance was compared among the three models. Through this process, model structures suitable for the target data were identified and practical applicability was evaluated.

3.2.3. Stage 3: Model Performance Evaluation

In the third stage, the extraction performance of each generative model was quantitatively evaluated using ROUGE scores. In addition, extraction-aligned evaluation metrics such as exact match (EM) and token-level precision, recall, and F1-score were employed to assess field-level slot correctness. Based on this evaluation, the best-performing model in terms of extraction accuracy was identified for experimental comparison. However, for subsequent database construction and structuring tasks, BART, which showed relatively lower performance but is a lightweight model, was utilized. This choice was made considering the optimal environment configuration for future utilization of this service by small and medium-sized enterprises.

3.2.4. Stage 4: Primary Structured Database Construction

In Stage 4, based on the output results of the selected generative model (BART), a primary structured database (DB) was constructed to store extracted and structured failure information in a structured format. This DB contains core information such as failed components, failure types, and corrective actions, and can be utilized for various statistical analyses and visualizations in the future.

3.2.5. Stage 5: Secondary Structured DB Construction and Standardization

In Stage 5, a secondary structured DB was constructed by applying standard word databases, domain terminology dictionaries, and regular expressions to more precisely classify and standardize the primarily structured data. This stage is a core process for improving data quality and ensuring analytical consistency.

3.2.6. Stage 6: Web-Based Service Implementation

In the final stage, a web-based visualization and analysis service was implemented to enable real-time querying, analysis, and utilization of the constructed structured data. This serves as a practical deliverable for verifying applicability in actual work environments, allowing users to intuitively utilize analysis results through various functions including equipment-specific failure pattern analysis, major component statistics, and predictive maintenance.

This step-by-step approach can contribute to supporting data-driven decision-making in manufacturing sites and establishing the foundation for AI-based predictive maintenance systems by structuring equipment management unstructured text and converting it into analyzable formats.

4. Dataset Construction and Model Training

4.1. Training Dataset Construction

This study proposes a training dataset construction methodology utilizing GPT-4 API to structure unstructured text data in the equipment management field. Input text is converted into JSON format as shown in Figure 3, and structured into three main elements: failed components, failure types, and corrective actions. For example, for the input sentence “Vertical motion motor brake noise occurrence, lining price adjustment AS replacement scheduled”, the GPT-4 API extracts the failed component as “vertical motion motor brake”, failure type as “noise occurrence”, and corrective action as “lining price adjustment AS replacement scheduled”, outputting them in structured JSON format. This approach aims to improve the efficiency of subsequent analysis and model training by systematically structuring unstructured data.

However, due to the characteristics of Large Language Models (LLMs), there exist limitations where information not present in the input text may be included in the output, or original content may be arbitrarily reconstructed. GPT-4 exhibits characteristics of reconstructing meanings during natural language generation processes beyond simple text segmentation functions, thus inherently carrying risks of unintended information addition or content modification. These issues can undermine the reliability and consistency of training data, requiring careful attention. To address these limitations, a validation procedure by domain experts was introduced. GPT-4 was employed solely to generate initial draft annotations for failed components, failure types, and corrective actions from the raw equipment maintenance texts. These initial annotations were subsequently reviewed by two domain experts: one is a manufacturing AI specialist with 30 years of experience at a global consulting firm, and the other is an equipment maintenance expert with 15 years of experience at a machinery industry association. The experts independently examined whether each GPT-4-generated annotation accurately reflected the original text based on the following criteria: (1) semantic accuracy-whether the annotation correctly captured the meaning of the original text, (2) completeness-whether critical information was missing, and (3) faithfulness-whether the annotation included content not explicitly present in the input text. Most GPT-4-generated annotations were consistent with expert judgments; however, corrections were applied in cases involving ambiguous expressions, domain-specific terminology, or complex multi-component failures. In cases where the two experts disagreed, consensus was reached through discussion, and the agreed-upon annotation was adopted as the final ground truth. This validation strategy improves the reliability of the training and evaluation datasets by incorporating expert review of LLM-generated annotations. The expert-validated annotations were then structured in dictionary format and parsed based on each key to construct datasets for model training, providing a reliable foundation for subsequent analysis and automatic information extraction model development.

This initial training data construction step was applied consistently across all three models (BART, T5, and Qwen) as a shared preprocessing procedure.

The exact prompt template used for GPT-4–based training data preparation is provided in Appendix B to ensure transparency and reproducibility.

4.2. Model Development

This study utilized BART, T5, and Qwen-based generative language models to automatically extract and convert unstructured text data into structured representations in the equipment management field.

Table 2. Overview of Applied Pre-trained Language Models and Their Specifications.

Model	Applied Models	Model Size
BART	KoBART (SK Telecom)	110 million
T5	pko-t5-base (PAUST)	250 million
Qwen2.5	Qwen2.5-0.5B-Instruct (Alibaba Cloud)	500 million

compares and summarizes the main characteristics of the three models used in this study.

The BART model is based on the KoBART model released by SK Telecom, containing approximately 110 million parameters. The T5 model utilizes the pko-t5-base model provided by the PAUST team, representing a large-scale pre-trained model with approximately 250 million parameters. Both models are specialized for Korean natural language processing and designed to be suitable for text generation tasks. For the Qwen model, Qwen2.5-0.5B-Instruct was utilized, which is a decoder-only generative Large Language Model (LLM) with approximately 500 million parameters. The exact prompt template used for Qwen-based model training and inference is provided in Appendix B to ensure consistent task formulation and reproducibility.

Generally, T5 and Qwen models, which have more parameters than KoBART, possess greater potential for learning complex language patterns. However, they have limitations of high computational resource consumption and increased GPU memory requirements during training and inference processes, which can result in cost burdens. In contrast, KoBART demonstrates stable performance in generation tasks such as structured information extraction even in resource-constrained environments, based on its relatively lightweight structure and Korean-specialized training.

Since the text data addressed in this study requires complex combinations of field-level information extraction and structuring, the three models were trained and evaluated to compare extraction performance and assess practical suitability. Particularly, this study aims to secure maximum performance with minimal model size from a practical perspective and consider applicability in actual service environments.

The dataset configuration for experiments was as follows. Among the total 29,736 original data records, 3001 records were used as training and validation datasets, with the ratio of training to validation data set at 8:2. Additionally, a separately constructed dataset of 800 new records was utilized as a test dataset to evaluate the final performance of the models. This data splitting strategy was designed to objectively verify the generalization performance of models and prevent overfitting.

For transparency and reproducibility, the exact input/output schematics, decoding settings, and language-specific output representations for the BART and T5 models are provided in Appendix A.

4.2.1. BART

The KoBART utilized in this study is a BART-based pre-trained model optimized for Korean natural language processing, which is effective for Korean text processing as it is pre-trained to reflect the linguistic characteristics of Korean. Based on an encoder–decoder structure, it is applicable to various natural language processing tasks including text summarization, generation, and structured information extraction. Due to these characteristics, it was selected as a suitable model for the equipment management information extraction and structuring tasks in this study.

Figure 4 illustrates the overall training and evaluation pipeline. The process includes data preprocessing and format conversion for information extraction tasks, initialization of a pre-trained KoBART model and tokenizer, configuration of key hyperparameters and optimization strategies, iterative training and validation with checkpoint selection based on validation performance, and preparation of the trained model for inference. Model performance was finally evaluated using ROUGE metrics and extraction-aligned measures, including exact match and token-level precision, recall, and F1-score. This pipeline serves as a reference implementation, and the same overall procedure was consistently applied to the T5 and Qwen models with architecture-specific adjustments.

Additionally, major hyperparameters were configured as follows, considering performance and training efficiency during the model training process. The batch size was set to 256, considering GPU memory constraints and training stability. The maximum number of epochs was determined to be 40 by analyzing the convergence pattern of validation loss, which is a value that allows sufficient learning while preventing overfitting. These hyperparameter settings contributed to optimizing the generalization performance of the model. The detailed hyperparameter configuration used for fine-tuning the KoBART model is summarized in

Table 3. Hyperparameter Settings for Fine-tuning the KoBART Model.

Parameters	Setting Values
batch_size	256
max_len	32
num_workers	4
lr	3 × 10−5
max_epochs	40
warmup_ratio	0.1

.

4.2.2. T5

This study utilized the pko-t5-base model, which is the Korean version of T5 (Text-to-Text Transfer Transformer), one of the generative AI models. The T5 model is based on a structure that processes both inputs and outputs in text format (text-to-text paradigm), enabling various natural language processing tasks to be performed within a single unified framework. This characteristic aligns with the purpose of this study to structure unstructured text data and provides a model architecture suitable for generative information extraction and field-level structuring tasks.

The T5 model possesses a considerable number of parameters compared to BART-family models, requiring more computational resources during training. Accordingly, the batch size was configured considering GPU memory constraints and model complexity, which is a measure to simultaneously ensure training stability and efficiency.

The optimal number of epochs was set to 30. This was the result of comprehensively considering validation loss convergence pattern analysis and overfitting prevention, representing a configuration tailored to the learning characteristics of the T5 model. These hyperparameter settings aimed to achieve a balance between model performance optimization and computational resource efficiency. The detailed hyperparameter configuration used for fine-tuning the T5 model is summarized in

Table 4. Hyperparameter Settings for Fine-tuning the T5 Model.

Parameters	Setting Values
batch_size	64
max_len	32
num_workers	4
lr	3 × 10−5
max_epochs	30
warmup_ratio	0.1

.

4.2.3. Qwen

This study utilized Qwen2.5-0.5B-Instruct, one of the generative AI models, for training. The Qwen model is a Large Language Model (LLM) that adopts a Transformer-based decoder-only architecture, capable of performing various natural language processing (NLP) tasks through auto-regressive text generation. The universal language understanding and generation capabilities acquired through large-scale pre-training align with the purpose of this study to structure unstructured text, particularly providing strengths in generative information extraction and field-level structuring tasks.

Qwen2.5-0.5B-Instruct possesses approximately 500 million parameters, providing the advantage of maintaining long-context processing and high language generation quality while managing computational resource consumption at a certain level.

Table 5. Hyperparameter Settings for Fine-tuning the Qwen Model.

Parameters	Setting Values
num_train_epochs	3
per_device_train_batch_size	2
gradient_accumulation_steps	2
learning_rate	1 × 10−4
warmup_ratio	0.03
optimizer	adamw_torch_fused

shows the main Supervised Fine-Tuning settings used for Qwen model training in this study.

5. Model Performance Evaluation and Results Analysis

This study utilized ROUGE (Recall-Oriented Understudy for Gisting Evaluation) metrics proposed by Lin [38] to evaluate the lexical overlap between model-generated outputs and reference labels in the context of generative information extraction. ROUGE is widely used in NLP research for measuring lexical overlap between generated and reference texts [6,9]. Although originally designed for summarization tasks, ROUGE is employed in this study as a supplementary indicator to measure surface-level lexical similarity between extracted fields and ground truth labels. ROUGE provides various variants based on the n-gram concept [4], and this study utilized ROUGE-1, ROUGE-2, and ROUGE-L scores. ROUGE-1 measures word-level similarity based on unigrams (single words), while ROUGE-2 evaluates contextual continuity based on bigrams (two consecutive words) [6]. ROUGE-L measures similarity considering word order based on the Longest Common Subsequence [9]. Each ROUGE metric is evaluated in three aspects: precision, recall, and F1-score. Recall measures the degree of necessary information inclusion as the proportion of words in the reference extraction output that overlap with the generated extraction output, while precision evaluates the degree of unnecessary information exclusion as the proportion of words in the generated extraction output that overlap with the reference extraction output. The F1-score is the harmonic mean of precision and recall, comprehensively indicating how accurately and completely the extraction information contains relevant content.

However, since the core task addressed in this study is structured information extraction (slot filling) rather than classic abstractive summarization, ROUGE alone is insufficient to fully capture field-level semantic correctness. Therefore, this study additionally adopts extraction-aligned evaluation metrics, including Exact Match (EM) and token-level Precision, Recall, and F1-score, to directly assess slot-level correctness and partial semantic overlap. These metrics are more suitable for short structured outputs and robust to synonymous but semantically correct expressions.

5.1. Evaluation Process

Model performance evaluation was conducted on a total of 800 test data records, and the ground truth labels for each test data were constructed through domain expert validation based on GPT-4 extraction results. Inference was performed using the three generative models trained in this study: BART (KoBART), T5 (pko-t5-base), and Qwen—as illustrated in Figure 5.

Each model was trained to generate structured extraction outputs (slot filling) for three items: failed components, failure types, and corrective actions based on input text, and the generated results were evaluated by comparison with ground truth labels. Model performance evaluation was conducted using both ROUGE-based metrics (ROUGE-1, ROUGE-2, ROUGE-L) and extraction-aligned metrics including Exact Match (EM) and token-level Precision, Recall, and F1-score. ROUGE metrics were used as supplementary indicators of lexical overlap, while EM and token-level metrics served as primary measures for structured extraction accuracy. ROUGE scores can quantitatively measure precision, recall, and F1-score at the lexical overlap level, making them suitable as complementary indicators for comparing relative generation tendencies among generative models.

5.2. Evaluation Results Analysis

The objective of this study is to evaluate and compare representative generative models for extracting and structuring failure-related information from unstructured equipment management text. Comparison of the three models’ test set performance revealed that the LLM model Qwen2.5 achieved the highest overall performance across both ROUGE and extraction-aligned metrics in all three comparison categories, demonstrating superior performance compared to T5 and BART models, as shown in

Table 6. Slot-level Extraction Performance (Exact Match and Token-level Precision, Recall, and F1-score).

Items	Models	Exact Match (EM)	Precision	Recall	F1-Score
Failed components	BART	0.473	0.794	0.802	0.798
	T5	0.564	0.832	0.838	0.835
	Qwen	0.635	0.860	0.867	0.863
Failure types	BART	0.395	0.521	0.519	0.520
	T5	0.468	0.670	0.545	0.601
	Qwen	0.486	0.629	0.644	0.636
Corrective actions	BART	0.387	0.729	0.768	0.748
	T5	0.415	0.826	0.644	0.724
	Qwen	0.513	0.805	0.793	0.799

and

Table 7. ROUGE Score Comparison as Supplementary Lexical Similarity Metrics.

Items	Models	Rouge Score
		ROUGE-1	ROUGE-2	ROUGE-L
failed components	BART	0.790	0.711	0.790
	T5	0.833	0.761	0.833
	Qwen	0.855	0.790	0.854
failure types	BART	0.571	0.260	0.571
	T5	0.641	0.285	0.641
	Qwen	0.670	0.327	0.670
corrective actions	BART	0.773	0.607	0.766
	T5	0.756	0.599	0.751
	Qwen	0.817	0.666	0.809

. This is attributed to the Qwen2.5 model having a larger number of parameters compared to BART and T5 models and the application of instruction tuning, which is characteristic of LLM models. Additionally, the T5 model’s superior performance over the BART model can be partially interpreted as resulting from its larger model size and text-to-text structural characteristics, which may provide advantages in generating concise and accurate slot outputs for relatively short extraction targets such as failed components and failure types.

At the same time, detailed analysis of BART and T5 models’ performance by category revealed distinct performance differences according to text length. For failed components and failure types, the T5 model achieved higher token-level Precision than BART, indicating its effectiveness in generating core terms with reduced noise for short text items, as illustrated in Figure 6 and Figure 7. In contrast, for corrective actions, which typically involve longer and more descriptive procedural expressions, the Qwen model consistently achieved higher Recall and F1-score, followed by BART and then T5, as shown in Figure 8. These results suggest that while Qwen demonstrates the most robust and balanced extraction performance overall, different models exhibit distinct strengths depending on the extraction category and evaluation metric.

5.3. Performance Improvement Directions

The following improvement directions are proposed to enhance the overall performance of the models. First, re-examination and refinement of the training dataset are necessary. Cases where model outputs were classified differently from the correct answers were identified in some categories, which may be related to the quality and labeling consistency of training data. Second, utilization of larger-scale pre-trained models (e.g., pko-t5-large, Qwen2.5-7B-Instruct, Qwen3) can be considered. Models with more parameters can process complex contextual information more effectively, thus performance improvements can be expected particularly for multi-sentence based input data. Third, scaling up the training data is also an important improvement strategy. By exposing models to more diverse expressions and contexts, generalization performance can be improved and robustness to new data can be strengthened.

These performance improvement directions are expected to contribute to enhancing the structured information extraction accuracy and slot-level extraction performance of models in future research.

5.4. Qualitative Error Analysis and Discussion

To complement the quantitative metrics, we conducted a brief qualitative error analysis to characterize typical cases in which structuring and classification become challenging. Importantly, the observed failures are primarily attributable to the inherent characteristics of industrial maintenance logs rather than model deficiencies. We summarize these challenges into three representative types. Type 1 (multi-event compression) occurs when multiple symptoms and multiple actions are condensed into a single entry, making one-to-one mapping between failure and action ambiguous (e.g., “A wiring disconnection was identified, followed by connector repair, valve replacement, and PLC output relay replacement.”). Type 2 (ambiguous or implicit corrective actions) arises when inspection, decision, and execution states are mixed or only partially described, obscuring whether an action was completed or merely planned (e.g., “The heating function failed; inspection was performed and repair was deemed infeasible.” or “Replacement was scheduled.”). Type 3 (non-standardized terminology and informal expressions) reflects frequent use of informal abbreviations, phonetic spellings, and non-standard terms that vary across operators and shifts, which can degrade semantic alignment and hinder robust normalization. These representative cases indicate that deterministic structuring is inherently difficult in real shop-floor logs, and they motivate future improvements such as state-aware action modeling and more robust normalization/mapping strategies.

6. Data Structuring and Analysis Service Implementation

6.1. Structured Database Construction

When input sentences are processed using BART, T5, and Qwen models, a primary structured database (DB) classified into three categories—failed components, failure types, and corrective actions—can be constructed. While this primary structuring result provides an important foundation for extracting and organizing unstructured text, information classified in text format still has limitations for quantitative analysis.

The ultimate goal of this study goes beyond simple text information extraction to convert each extracted information unit into quantified structured data that enables various statistical analyses such as frequency analysis, correlation analysis, and time series analysis. Specifically, by restructuring information into forms that can analyze the occurrence frequency of failed components and failure types, associations between corrective actions, and pattern changes over time, the objective is to establish an analytical framework capable of supporting data-driven decision-making. For this purpose, this study additionally performed a secondary structuring process, and the overall procedure of this process is shown in Figure 9.

Secondary structuring proceeds by applying differentiated structuring techniques according to the text characteristics of each category.

• For failed components and failure types, which show relatively structured terminology patterns, a regular expression-based rule application approach was utilized. Through this method, terms with identical concepts described in various expression formats were identified and unified into standardized forms, ensuring analytical consistency.
• In contrast, corrective actions are described in various ways using natural language and have high expression variability, making structuring through regular expressions alone limited. Accordingly, the application of topic-level grouping techniques is proposed to group sentences with similar meanings. This approach performs automatic grouping based on semantic similarity between texts and can be utilized as an effective methodology for supporting the structuring of corrective actions, rather than strict algorithmic clustering. The specific application and validation of topic-level grouping techniques were set as future research tasks.

The secondary structuring system consists of the following core components. First, the domain-specific terminology dictionary is used to map expressions for each category to standardized domain terms, enhancing data consistency by unifying terms with identical meanings but different expressions. The structuring module receives primary extraction results as input and applies algorithms optimized for each category to convert them into analyzable forms. In the currently implemented system, failed components and failure types are standardized through regular expression-based pattern matching, while clustering-based automatic grouping of corrective actions is planned for future implementation.

Through this secondary structuring process, data that began as unstructured text is ultimately converted into structured datasets that enable various statistical analyses such as frequency analysis, correlation analysis, and time series analysis. While this process is fully implemented for failed components and failure types, the structuring of corrective actions currently remains at an initial stage, with clustering-based semantic grouping positioned as future work. This represents one of the key contributions of this study in that it establishes an analytical framework that supports field-based decision-making and predictive analysis beyond simple text extraction.

6.2. AI Model Service Implementation

Among the three developed models, the BART model was selected for deployment considering the constraints of small and medium-sized manufacturing enterprises. Its practicality was confirmed through the implementation of a web service-based information extraction and analysis platform designed to enhance actual applicability. Although the Qwen model achieved the highest extraction performance in the experimental evaluation, the final deployed model was selected by considering practical deployment constraints in real manufacturing environments. In particular, the proposed system is designed to be integrated with existing MES and ERP systems and operated continuously in on-premise manufacturing settings, where GPU-based inference is often impractical due to cost, maintenance complexity, and organizational IT or security policies. From this perspective, the BART model was selected as a deployment-oriented solution, as it offers a favorable balance between extraction performance and computational efficiency. Unlike large-scale language models, BART can be reliably operated in CPU-only environments while maintaining stable inference behavior, making it more suitable for real-world manufacturing applications. The implemented system was confirmed to operate smoothly under CPU-based deployment conditions, supporting its feasibility for practical industrial use. This system was built based on the FastAPI framework and designed with a structure that receives text input in REST API format, performs inference through AI models, and returns results in real-time.

Despite its strong performance in specific categories, the T5 model was not selected for deployment due to considerations related to output consistency and operational stability in a continuously running on-premise service. When users input unstructured equipment management text through the web interface or API, the system utilizes the selected generative model (BART) to extract and identify the text into three categories: failed components, failure types, and corrective actions. Subsequently, the results undergo processing through a domain-specific terminology dictionary and rule-based structuring engine to be ultimately converted into analyzable structured data.

In this study, the primary research focus lies in the first-stage structuring of unstructured text using generative AI models, while the secondary structuring process is designed as a practical refinement step to transform the extracted results into forms suitable for analysis and utilization. The current secondary structuring process relies on rule-based mapping using domain-specific dictionaries and regular expressions, which was intentionally chosen to ensure transparency, interpretability, and stable operation in on-premise manufacturing environments. In this stage, the effectiveness of rule-based mapping depends more on the quality and coverage of the domain-specific terminology dictionary than on the underlying model itself, highlighting the importance of systematically capturing and incorporating shop-floor knowledge. However, this approach is inherently sensitive to spelling variations, informal expressions, and unseen lexical patterns frequently observed in manufacturing logs. To improve robustness against such variations, more flexible approaches such as fuzzy string matching or embedding-based semantic similarity could be integrated into the secondary structuring pipeline, which is left as future work.

To utilize structured data for decision-making in actual industrial sites, this study developed the following four analysis modules:

• Time Series Analysis Module supports analysis of changes in failure occurrence pattern trends over time.
• Correlation Analysis Module enables identification of causal relationships by analyzing statistical associations between failed components and failure types.
• Frequency Analysis Module enables identification of priorities based on the occurrence frequency of specific failure types or corrective actions.
• Anomaly Detection Module is utilized to identify abnormal signs or exceptional patterns that deviate from normal ranges.

Each analysis module was designed considering applicability in actual work environments, with emphasis placed on visualization and user interface configuration to enable users to intuitively interpret results and rapidly utilize them for decision-making. Together, these components establish an automated information processing system that encompasses the entire pipeline from unstructured text input to quantitative analysis.

Through this approach, an integrated analysis platform was established that can transform information originating from unstructured text data into quantitative analysis-based insights (Figure 10).

7. Conclusions and Future Research

7.1. Research Results

This study proposed an integrated framework that structures unstructured text data underutilized in manufacturing industries through generative AI models and converts it into analyzable formats. The major research achievements are as follows.

First, we developed a system that automatically extracts and structures manufacturing equipment management text into three structured categories—failed components, failure types, and corrective actions—by applying generative language models BART, T5, and Qwen to manufacturing equipment management text analysis to the best of our knowledge. Particularly, the hybrid approach combining training dataset construction using GPT-4 with domain expert validation presented a new methodology for securing high-quality training data aligned with extraction task.

Second, we validated the effectiveness of the proposed methodology through large-scale experiments conducted on curated training and test subsets derived from a full MES dataset of 29,736 records from an automotive parts manufacturing company. Based on extraction-aligned evaluation results including exact match and token-level precision, recall, and F1-score, with ROUGE used as a supplementary indicator, the Qwen model demonstrated the best overall performance across all extraction categories. Performance comparison results between T5 and BART models indicated that the T5 model achieved relatively stronger performance for short extraction targets such as failed components and failure types, while BART showed comparatively competitive performance, but did not outperform Qwen, for corrective action extraction, highlighting the importance of task- and metric-aware model selection rather than reliance on a single architecture.

Third, beyond simple text extraction, we constructed a database enabling quantitative analysis through a secondary structuring process utilizing domain-specific dictionaries and regular expressions. This quantitative analysis capability is fully realized for failed components and failure types, while corrective actions currently provide a foundational level of structuring to support further refinement in future research. Based on this, we developed a web-based analysis platform including functions such as time series analysis, correlation analysis, and frequency analysis, enhancing practical applicability.

Fourth, we presented an alternative solution that can support digital transformation of manufacturing companies with lower initial investment costs compared to existing IoT sensor-based data collection methods. Particularly, we demonstrated that AI-based equipment management system construction is possible without large-scale infrastructure investments that burden small and medium-sized manufacturing enterprises.

7.2. Limitations and Improvement Directions

First, the dataset used in this study is limited in size, as it is derived from real-world equipment management logs collected from an actual manufacturing environment. This limitation reflects practical constraints commonly encountered in industrial settings, where maintenance records are rarely publicly available and are often subject to confidentiality and access restrictions. Therefore, the primary objective of this study was not to achieve broad statistical generalization, but to examine the feasibility of generative AI-based structuring approaches under realistic conditions in which industrial maintenance texts are scarce, noisy, and organization-specific.

Second, since this study was conducted limited to equipment management data from automotive parts manufacturers, validation of generalization performance for other manufacturing fields or various text types is necessary.

Third, although ROUGE was used as a supplementary indicator, it exhibited limitations in some categories. Accordingly, the potential for performance enhancement through the utilization of larger-scale pre-trained models should be explored in future work.

Fourth, while secondary structuring for failed components and failure types was implemented based on regular expressions, automatic clustering-based grouping for corrective actions was not fully implemented. Therefore, the current analysis of corrective actions should be interpreted as preliminary, primarily due to the diversity of natural language expressions.

Finally, this study evaluates generalization only within a single company and a specific industrial domain; therefore, external validity is inherently limited. The findings should be interpreted within the context of the evaluated maintenance logging practices, vocabulary, and operational environment, and we intentionally refrain from making claims beyond this scope. Future work will test cross-company and cross-domain transfer by incorporating more diverse datasets and evaluating robustness under heterogeneous terminology and logging styles. Such multi-site validation is necessary to quantify generalization performance and to establish broader applicability of the proposed structuring pipeline.

7.3. Future Research Directions

To address the limitations of this study and build more advanced systems, we plan to conduct follow-up research in the following directions. First, we plan to develop methodologies that can achieve high performance even with small-scale datasets by utilizing recently developed large language models including Qwen and applying few-shot and zero-shot learning techniques. Additionally, by combining structured text data with time series analysis to build AI-based predictive maintenance systems that can predict equipment failures in advance, we will support the paradigm shift from reactive maintenance to preventive maintenance and realize improvements in equipment operation rates and reductions in maintenance costs. Furthermore, we plan to implement an integrated manufacturing intelligence platform by expanding the current equipment management analysis scope to quality inspection reports, customer claims, and supplier communications. Finally, we aim to provide customized AI solutions that reflect the characteristics and expertise of each domain by developing specialized models for major manufacturing fields.