Large Language Model-Based Autonomous Agent for Prognostics and Health Management

Abstract

Prognostics and Health Management (PHM), including fault diagnosis and Remaining Useful Life (RUL) prediction, is critical for ensuring the reliability and efficiency of industrial equipment. However, traditional AI-based methods require extensive expert intervention in data preprocessing, model selection, and hyperparameter tuning, making them less scalable and accessible in real-world applications. To address these limitations, this study proposes an autonomous agent powered by Large Language Models (LLMs) to automate predictive modeling for fault diagnosis and RUL prediction. The proposed agent processes natural language queries, extracts key parameters, and autonomously configures AI models while integrating an iterative optimization mechanism for dynamic hyperparameter tuning. Under identical settings, we compared GPT-3.5 Turbo, GPT-4, GPT-4o, GPT-4o-mini, Gemini-2.0-Flash, and LLaMA-3.2 on accuracy, latency, and cost, using GPT-4 as the baseline. The most accurate model is GPT-4o with an accuracy of 0.96, a gain of six percentage points over GPT-4. It also reduces end-to-end time to 1.900 s and cost to $0.00455 per 1 k tokens, which correspond to reductions of 32% and 59%. For speed and cost efficiency, Gemini-2.0-Flash reaches 0.964 s and $0.00021 per 1 k tokens with accuracy 0.94, an improvement of four percentage points over GPT-4. The agent operates through interconnected modules, seamlessly transitioning from query analysis to AI model deployment while optimizing model selection and performance. Experimental results confirmed that the developed agent achieved stable performance under ideal configurations, attaining accuracy 0.97 on FordA for binary fault classification, accuracy 0.95 on CWRU for multi-fault classification, and an asymmetric score of 380.74 on C-MAPSS FD001 for RUL prediction, while significantly reducing manual intervention. By bridging the gap between domain expertise and AI-driven predictive maintenance, this study advances industrial automation, improving efficiency, scalability, and accessibility. The proposed approach paves the way for the broader adoption of autonomous AI systems in industrial maintenance.

1. Introduction

1.1. Research Background

Prognostics and Health Management (PHM) is essential for ensuring the safety, reliability, and operational efficiency of industrial equipment [1]. These technologies minimize unexpected failures, optimize maintenance schedules, and reduce overall operational costs, making them indispensable in industrial operation and maintenance [2,3]. Among various PHM tasks, fault diagnosis and Remaining Useful Life (RUL) prediction are essential for predictive maintenance, as they enable early fault detection and provide accurate estimates of component degradation trends.

In practice, however, developing effective PHM systems is a highly challenging task for maintenance engineers and domain practitioners, especially those without advanced AI expertise. Current workflows demand extensive data preprocessing, careful model design, performance evaluation, and iterative refinement, all of which require deep knowledge in machine learning and data analysis [4,5]. This makes the barrier to entry prohibitively high for non-experts. As a result, despite the promise of AI-driven PHM, its deployment is often restricted to organizations with sufficient resources and dedicated AI specialists. This reliance on expert-driven workflows presents several challenges. Firstly, the steep learning curve and technical expertise required create barriers for non-specialists, limiting accessibility and adoption [6,7]. Secondly, manual processes demand considerable time and resources, making it difficult to scale these technologies across various industrial applications [8]. This inefficiency hinders the rapid and cost-effective deployment of PHM systems. Lastly, traditional approaches often struggle to adapt to new industrial environments and datasets, reducing their effectiveness in dynamic operational settings [9].

Therefore, there is a strong motivation to develop a new type of system that allows non-experts to interactively build PHM models through natural language dialog. Such a system should be able to capture the user’s objectives and required level of performance, translate these into concrete modeling tasks, and autonomously generate a fault diagnosis or RUL prediction system that aligns with those expectations. By transforming the process from manual, expert-intensive workflows into intuitive conversations, PHM can become more democratized, scalable, and adaptive to real-world industrial needs.

Thus, we develop an autonomous large language model-based autonomous agent that bridges the expertise gap in maintenance workflows by eliciting user objectives and performance targets, translating them into concrete steps for data preparation, modeling, and validation, and automatically producing fault diagnosis and RUL predictors. In doing so, the system broadens access to advanced PHM tools, reduces dependence on specialists, and supports efficient, scalable, and adaptive deployment in industrial settings.

1.2. Advances in LLMs

Recent advancements in LLMs have demonstrated their versatility across various tasks, including natural language processing, mathematical reasoning, and information retrieval [10,11]. These models leverage vast datasets and sophisticated architecture, enabling users with minimal knowledge to perform complex tasks [12]. Their potential has driven growing interest in applying LLMs to domain-specific applications. For instance, autonomous agents integrating LLMs with domain-specific APIs and algorithms have been proposed to solve targeted problems in certain industrial settings [13,14]. These agents leverage the LLM’s capabilities to perform domain-specific tasks while utilizing additional tools or algorithms as required. Furthermore, recent studies have extended this approach by developing systematic systems involving multiple agents [15,16,17,18].

From the perspective of PHM, these advances in LLMs present a unique opportunity. Unlike conventional AutoML methods, LLM-based agents are capable of interpreting user intent expressed in natural language, handling incomplete or ambiguous queries, and dynamically structuring workflows for data preprocessing, model training, and evaluation. This makes them particularly well suited to addressing the long-standing challenge of enabling AI non-specialists to design and deploy effective PHM systems.

In addition, PHM applications often involve multivariate time-series datasets derived from sensors such as vibration, temperature, and pressure. The preprocessing required for such data, which includes noise filtering, normalization, imputation, and feature extraction, has often been overlooked, even though it is critical for model performance [19]. When combined with external preprocessing modules, LLMs can automate and guide these steps through natural language instructions [20].

For example, a user query such as “Prepare the bearing dataset for fault diagnosis” can be translated into a structured sequence of preprocessing operations involving data cleaning, partitioning, and feature engineering. By bridging user intent with PHM-specific data preparation, LLMs improve both the usability and scalability of complex preprocessing pipelines, thereby reinforcing their potential for real-world PHM tasks.

1.3. Research Gap and Objective

Despite recent advancements demonstrating the versatility and potential of LLMs, several types of gaps remain in their application to PHM domains, where high accuracy, domain-specific knowledge, and reliability are essential. First, current LLMs are not explicitly trained on PHM data, which limits their ability to incorporate domain-specific knowledge required for accurate fault diagnosis and RUL prediction. Second, while LLMs excel in processing natural language, they have limitations in handling heterogeneous PHM datasets such as multivariate time-series sensor data, which require specialized preprocessing, feature extraction, and temporal modeling. Third, the reliability and consistency of LLM outputs are still a concern in accuracy- and reliability-critical applications, since incorrect parameter extraction or task misinterpretation can compromise the dependability of PHM workflows. Finally, existing approaches to applying LLMs in PHM face scalability challenges across diverse industrial contexts, as they often rely on expert-driven customization. This limitation highlights the need for autonomous agents that can flexibly adapt to different equipment, environments, and operating conditions, which is the focus of our proposed framework. To date, no comprehensive framework has been developed to leverage LLMs for automating the full lifecycle of model development while addressing these gaps. Key stages, including data preprocessing, model design, optimization, and performance evaluation, remain heavily reliant on expert intervention, limiting scalability and accessibility.

This study introduces a novel approach by integrating an LLM-based autonomous agent to address these issues and outlines its key advantages as follows:

• Simplified user interaction and robust query handling: The agent lowers the barrier to entry by allowing non-specialists to interact with the system using simple natural language queries. When queries are ambiguous or incomplete, the agent provides interactive guidance, prompting for missing inputs or suggesting valid alternatives. This functionality prevents workflow interruptions and enhances overall usability, making the system more accessible in practical industrial contexts.
• Automated model generation and execution: The proposed agent eliminates manual intervention by structuring and executing the complete workflow for fault diagnosis and RUL prediction, thereby reducing reliance on domain expertise and accelerating the development of predictive models.
• Comprehensive evaluation of multiple LLM engines: This study systematically compares LLMs, including GPT-3.5, GPT-4 mini, GPT-4o, LLaMA, and Gemini, assessing their effectiveness in parameter extraction, computational cost, and query processing. The comparative analysis provides insights into the optimal LLM choice for AI-based an autonomous agent.
• Domain-adapted automation with PHM-specific tools: The autonomous agent integrates PHM specific tools to ensure that model generation, evaluation, and performance assessment are aligned with industrial requirements. As illustrated in Figure 1, the agent dynamically activates specialized modules for data handling, model training, and evaluation, thereby tailoring the process to PHM tasks.

The rest of the paper is organized as follows. Section 2 reviews existing research on fault diagnosis and RUL prediction using machine learning and deep learning, as well as studies on LLMs and their applications in domain-specific tasks. Section 3 outlines the methodology, functionalities, and implementation of the proposed autonomous agent for fault diagnosis and RUL prediction, including the overall agent design and its components. Section 4 presents the evaluation results of the autonomous agent applied to various use cases, along with performance comparisons. In Section 5, we discuss the performance results, limitations, and potential strategies to address these limitations. Finally, Section 6 concludes the paper with conclusions.

2. Related Work

2.1. Fault Diagnosis and Remaining Useful Life Prediction

Machine learning (ML) and deep learning (DL) have great promise in fault diagnosis and RUL prediction, enabling meaningful insights from large datasets. These methods have advanced predictive systems through architectures like Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTM) networks, Attention models, and Hybrid models.

CNNs are widely used for analyzing sensor data, excelling in spatial feature extraction for applications like fault detection and RUL estimation [21]. Their effectiveness in processing high-dimensional data has made them crucial for pattern recognition in industrial settings. LSTM networks improve temporal modeling by capturing long-term dependencies in sequential data, making them well-suited for prognostic tasks in fault diagnosis and RUL prediction [22]. In commercial aviation, for instance, LSTM-based models trained on multivariate sensor trajectories like NASA’s C-MAPSS learn degradation trends to predict turbofan RUL. These models enhance prediction accuracy and prevent overfitting, enabling reliable maintenance scheduling and increased operational safety. Feeding historical sensor data into such deep learning architectures allows learning of degradation trends that are critical for precise RUL prediction [23,24]. Attention-based models, such as time-enhanced multi-head self-attention, enhance these capabilities by selectively focusing on relevant features, particularly for aero-engine RUL prediction [25]. Hybrid models further integrate these strengths, combining CNNs for spatial analysis and LSTMs for temporal dependency modeling. For example, CNN-LSTM architectures and multi-scale CNNs with bidirectional Gated Recurrent Units (GRU) have demonstrated improved diagnostic accuracy in applications like bearing fault detection and rotating machinery analysis [26,27,28]. In fault diagnosis, a representative bearing example uses raw accelerometer signals that are windowed and normalized, then passed through a one-dimensional CNN to distill local spectral temporal motifs and through an LSTM to capture sequence dynamics, after which a classifier outputs health-state labels such as normal, inner-race defect, outer-race defect, and ball defect [29,30].

Comprehensive reviews of PHM highlight the transformative impact of DL models in enabling predictive maintenance and scalable health monitoring, addressing critical efficiency and accuracy challenges in industrial applications [31,32,33]. Despite their success, prior research has also identified critical limitations in the adoption and scalability of these methods in real-world industrial settings. Developing and deploying ML and DL models often require extensive manual intervention by data scientists and AI specialists [34]. Data preprocessing, feature engineering, model selection, and hyperparameter tuning remain labor-intensive and time-consuming, making these methods inaccessible to non-specialists [35].

Although automated machine learning (AutoML) systems have been introduced to streamline these processes, they still fall short in fully addressing domain-specific complexities, leaving significant room for improvement [36,37]. Additionally, industrial maintenance professionals often lack the advanced technical skills required to interpret ML and DL model outputs and integrate them into decision-making workflows [38]. This expertise barrier restricts the practical adoption of these technologies, particularly in industries where non-specialists play a central role in operations [39].

To address these challenges, several studies have proposed solutions aimed at reducing reliance on expert knowledge and improving accessibility. Integrating ML and DL models with big data analytics frameworks has been explored to create more robust and user-friendly systems [40]. Furthermore, strategic management perspectives have emphasized the importance of designing AI systems that are both adaptable and scalable across diverse applications [41,42]. However, these methods still require significant computational resources and domain-specific adjustments. As such, the development of more automated and user-friendly frameworks remains a critical focus in advancing AI for fault diagnosis and RUL prediction.

2.2. Large Language Models

LLMs have significantly enhanced language understanding and generation capabilities in the field of Natural Language Processing (NLP) [43]. These capabilities, known as Language Modeling (LM), represent a fundamental approach to advancing machine language intelligence. LM predicts appropriate words based on the context of preceding and succeeding words in a sequence. Early research applied information theory to human language, demonstrating the potential of n-gram models to predict or compress natural language text [44]. This led to the development of Statistical Language Models (SLMs), which estimate the probability of word sequences. SLMs rely on the Markov assumption and n-gram models to predict the next word, making them simple and efficient tools for various NLP tasks [45]. However, SLMs have notable limitations, including data sparsity and an inability to understand long-range contexts. Techniques such as smoothing and back-off methods have been employed to mitigate these issues [46,47], but they remain inadequate for addressing the complexities of high-dimensional data [48,49].

To overcome these challenges, Neural Language Models (NLMs) were introduced, marking a significant advancement in language modeling. NLMs use neural networks to learn word sequences and calculate text probabilities [50]. They transform words into embedding vectors, capturing contextual meaning more effectively [51]. Architectures like Recurrent Neural Networks (RNNs), LSTM, and Transformer efficiently model relationships between words, enabling superior performance in tasks such as web search, machine translation, text generation, and sentiment analysis [10,11,52]. However, NLMs require large-scale data and high computational costs, and their lack of interpretability presents challenges in applications where trust and transparency are critical [53]. More recently, Pretrained Language Models (PLMs) and LLMs have revolutionized the NLP paradigm. PLMs are pretrained on massive amounts of unlabeled text data and fine-tuned for specific downstream tasks, achieving a balance between generality and task-specific performance [54]. LLMs, built on Transformer architectures with billions of parameters, further enhance language understanding and generation capabilities [39,40].

They demonstrate emergent abilities, allowing them to learn new tasks and solve complex problems. Examples such as OpenAI’s GPT-3.5 turbo, GPT-4, GPT-4o, GPT-4o-mini Meta’s LLaMA-3.2, and Google’s Gemini-2.0-flash showcase reasonable accuracy and efficiency in not only everyday tasks but also in understanding domain-specific terminology and linguistic nuances. These models have substantially advanced text analysis and query processing in specialized fields.

2.3. LLMs for Domain-Specific Applications

Advancements in LLMs have demonstrated their potential to address domain-specific challenges across various industries. For instance, in materials science, LLMs have been utilized to predict and generate metal–organic frameworks (MOFs), highlighting the potential to transform material design processes [55]. In structured data analysis, multi-agent LLM systems have been employed to handle tabular data queries, enabling more efficient and accurate data-driven insights [56]. In chemistry and life sciences, LLMs have shown promise in tackling molecular design problems, suggesting potential applications in drug discovery and chemical analysis [17]. In engineering and automation, LLMs have been integrated into workflows for electronic design automation (EDA), enhancing circuit design efficiency [57]. In cloud computing, LLM-based autonomous agents have been developed to analyze and resolve root causes of system issues, demonstrating utility in select cases [16].

Evaluations of LLM-based agents in real-world scenarios have further highlighted their adaptability and reliability [58]. In the energy sector, LLMs have been applied to detect gas leaks in natural gas valve chambers offering the potential to enhance safety and efficiency [59]. Autonomous systems and manufacturing have also benefited from LLM integration. Multi-agent frameworks leveraging LLMs have been developed to improve decision-making and control in autonomous driving [60]. In the manufacturing industry, LLM-powered decision-making systems have been utilized for managing carbon emissions, presenting opportunities to promote sustainable practices [61].

Similarly, LLMs have been applied in decentralized collaboration systems using smart contracts, enabling more efficient decision-making in distributed systems [62]. LLMs have also been explored for visual information seeking tasks, improving autonomous data collection and exploration [63]. Flexible modular production systems incorporating LLM-powered agents have shown potential in enhancing operational flexibility and scalability in manufacturing [64]. In scientific research, LLMs have been applied to predict organic synthesis pathways, simplifying chemical processes [65]. Autonomous chemical research has also employed LLMs to support experimental design and data analysis, demonstrating potential efficiency improvements [18].

Furthermore, the integration of LLMs with external tools has enhanced their problem-solving capabilities across diverse applications [15]. In project management, LLM-based agents have improved efficiency and adaptability in agile workflows [66]. In particle physics, LLM-inspired computational approaches have been used to analyze high-energy physics data, contributing to the exploration of phenomena beyond the Standard Model [67]. While LLMs have shown promise in a variety of domains, their application in PHM, which are key tasks in industrial maintenance, remains largely unexplored. This study aims to address this gap by developing a LLM-powered autonomous agent tailored for these critical tasks, paving the way for more efficient and automated solutions in the maintenance industry.

3. Method

3.1. Overall Strategy for Autonomous Agent

The autonomous agent is designed to process user queries in natural language and generate models for fault diagnosis or RUL prediction in an efficient and structured manner. Unlike traditional PHM workflows that require manual dataset preprocessing, algorithm selection, and parameter tuning, the proposed agent allows users to initiate the entire process through simple natural language instructions. This design significantly lowers the barrier to entry, enabling non-specialists to perform complex tasks such as fault diagnosis or RUL prediction without programming expertise or in-depth AI knowledge. For example, when a user inputs a query like “Conduct fault diagnosis on my bearing dataset”, the system automatically interprets the request, prepares the data, selects and trains appropriate models, and evaluates their performance.

Beyond simplifying access for non-specialists, the agent is also designed to remain robust when user inputs are ambiguous or incomplete. If a required parameter such as the dataset path is missing, the Parameter evaluator interactively prompts the user to provide the missing information or suggests a default option. Likewise, if unsupported models or evaluation metrics are requested, the system suggests valid alternatives from its supported list. By guiding users in this way, the agent ensures that tasks can proceed smoothly even with imperfect queries, thereby preventing workflow interruptions and further enhancing overall usability.

As illustrated in Figure 2, the agent consists of four interconnected modules: Parameter extractor, Parameter evaluator, Task executor and Answer generator. These modules work together in a coordinated manner, ensuring a smooth transition from query processing to model generation and result delivery.

The Parameter extractor analyzes the user query to identify necessary parameters (e.g., target category, dataset path) and optional parameters (e.g., model name, evaluation flag). To ensure reliable extraction, prompt engineering techniques are applied, allowing the module to accurately extract parameters. Based on this structured approach, the LLM engine processes the query and extracts key parameters.

To account for diverse query structures and potential ambiguities, we define eight use case categories, each representing a different query characteristic. These predefined cases help ensure that the extraction process remains consistent and adaptable across various input scenarios. Detailed description and analysis of these use cases can be found in Section 4.1.

The extracted parameters are then passed to the Parameter evaluator, which verifies their validity and completeness. If any necessary parameters are missing, the evaluator prompts the user to input and updates the parameters accordingly. This process is repeated until all required information is provided. If optional parameters are missing, the agent suggests a list of available parameters and use the parameters selected by the user. If not, apply the default settings for smooth execution.

Once validation is complete, the structured parameters are passed to the Task executor, which configures and initiates the model generation process. The Task executor is composed of three main components: the Manager, Tool, and Task evaluator, each playing a crucial role in executing and optimizing the workflow. The Manager structures the job flow based on the extracted parameters and activates the appropriate submodules within Tool. The Tool includes six key submodules: Model optimizer, Model tuner, Model trainer & evaluator, Data loader, Data preprocessor, and Model generator, responsible for data preparation, model selection, hyperparameter tuning, training, and evaluation.

After a model is generated, the Task evaluator within Task executor assesses its performance based on predefined metrics. If the model does not meet the required criteria, the system triggers an optimization cycle. This process continues until an optimal model is obtained or the maximum optimization limit is reached. After optimization, the Task executor selects the top 5 models based on evaluation flag and saves their details into a Comma Separated Values (CSV) file. Answer generator then extracts information about the best-performing model and formats it into a structured response. The following sections provide a detailed explanation of each module in the autonomous agent.

3.2. Query Analysis of Autonomous Agent

3.2.1. Parameter Extractor

The Parameter extractor of the autonomous agent is based on an LLM engine and is responsible for analyzing user queries and extracting parameters. As previously described, these parameters consist of necessary parameters and optional parameters.

The necessary parameters are essential for the autonomous agent to execute tasks based on the user query. In this study, we define the target category and dataset path as the necessary parameters required for executing model generation tasks. The target category specifies the type of model to be generated, such as fault diagnosis (binary fault classification or multi-fault classification) or RUL prediction, while the dataset path provides the location of the data required for model training.

Additionally, we designate model name, evaluation flag, and evaluation value as optional parameters to accommodate diverse user requirements and allow greater flexibility in model configuration. The model name allows users to specify architectures such as CNNs, LSTM networks, attention model, and hybrid models, while the evaluation flag enables user to select performance criteria such as Validation Loss, Root Mean Squared Error (RMSE), and F1 Score.

Since users may express the same intent using different wording and structures, the extracted parameters can vary significantly, potentially reducing the accuracy of the Parameter Extractor. This variation propagates through subsequent stages of the autonomous agent, directly impacting the effectiveness of the Task Executor. Therefore, selecting an appropriate LLM engine is crucial to ensuring robust parameter extraction and maintaining overall system reliability.

We evaluated the performance of parameter extraction with six widely used LLM engines [68,69]. To determine the most suitable LLM engine for this study, we conducted experiments on 100 use cases covering various scenarios across eight predefined use-case types. To ensure a fair performance comparison, each use case was repeated 10 times per engine. The engines were assessed based on accuracy (exact match at the use-case level), end-to-end response time (seconds), and API cost in USD per 1 k tokens (input plus output where applicable).

Table 1. Average performance comparison results of commonly used LLM engines.

Engine	Time(s)	Cost ($)	Accuracy
GPT-3.5 turbo	1.206	0.00053	0.68
GPT-4	2.809	0.01114	0.90
GPT-4o	1.900	0.00455	0.96
GPT-4o-mini	1.489	0.00225	0.91
Gemini-2.0-flash	0.964	0.00021	0.94
LLaMA-3.2	1.952	0	0.53

provides a comparative analysis of each LLM engine’s performance, summarizing their effectiveness in parameter extraction across these metrics. Additionally, overall results of parameter extraction, response time, and cost, which compare the performance of each engine across the eight use case categories comprising 100 queries, can be found in Tables S1–S3 of Supplementary Materials.

The experimental results showed that GPT-4o achieved the highest accuracy 0.96 while maintaining lower response time and cost compared to GPT-4. Although GPT-4 had a slightly lower accuracy 0.90, it exhibited higher latency of 2.809 s and higher cost $0.01114. Additionally, Gemini-2.0-Flash and GPT-4o-mini achieved competitive accuracy of 0.94 and 0.91, with faster response times 0.964 s and 1.489 s and lower costs, suggesting their suitability for scenarios that prioritize efficiency over absolute accuracy. These results emphasize the need to select an LLM engine that aligns with the specific requirements of each task. While GPT-4o is well-suited for applications demanding high accuracy, Gemini-2.0-Flash and GPT-4o-mini offer cost-effective options when processing speed is a priority.

Since precise operation and outcome generation are crucial in the autonomous agent, GPT-4o was chosen as the LLM engine for this study. Furthermore, the agent is designed to accommodate multiple LLM engines, providing flexibility in model selection based on constraints such as computational cost, response time, and accuracy. This adaptability ensures that the framework can be tailored to various operational scenarios, optimizing performance according to application-specific requirements.

Additionally, as illustrated in Figure 3, the prompt engineering techniques of the Parameter extractor define the extraction task the LLM must perform, while the user specifies the desired extraction conditions. These conditions are predefined based on parameter categories, including expected formats and constraints. The extraction task request is formulated through prompt engineering and sent to the LLM engine, which processes the request and returns a response containing a parameter dictionary extracted from the user query. This parameter dictionary includes essential values such as target category, dataset path, evaluation flag, model list, and target evaluation value, which are necessary for subsequent steps in the autonomous agent workflow. Algorithm 1 presents a simplified pseudocode outlining the query processing and parameter extraction procedure. More detailed version, including can be found in Table S4 of Supplementary Materials.

Algorithm 1 Parameter extractor

$\mathrm{Procedure} \mathrm{extract}\_\mathrm{parameter} (Q$$, T$$, \mathit{O}$) Inputs: $Q$: A user query $T$: Query type $\mathit{O}$: Extraction options Output: A dictionary of extracted parameters $1: \mathrm{Define} \mathrm{system}\_\mathrm{role} \mathrm{and} \mathrm{message} \mathrm{based} \mathrm{on} T$ $2: \mathrm{If} T$ == 0: 3:    system_role ← “Extract all parameters” 4:    message ← {“validations”: “Ensure missing values are assigned None”} $5: \mathrm{Else} \mathrm{if} T$ == 1: 6:    system_role ← “Extract necessary parameters” 7:    message ← {“validations”: “Validate missing local paths”} $8: \mathrm{Else} \mathrm{if} T$ == 2: $9:\hspace{1em} \mathrm{system}\_\mathrm{role} \leftarrow “\mathrm{Extract}” + (“\mathrm{models}” \mathrm{if} \mathit{O}$$\left[1\right] == 1 \mathrm{else} “\mathrm{evaluation} \mathrm{method}” \mathrm{if} \mathit{O}$[1] == 2 else “evaluation method and models” $10:\mathrm{message} \leftarrow \{“\mathrm{validations}”: [\mathit{O}$$\left[2\right]\left[0\right], \mathit{O}$[2,1]]} $11: \mathrm{Else} \mathrm{if} T$ == 3: 12:    system_role ← “Extract dataset path” 13:  Return LLM_engine (system_role, message)

3.2.2. Parameter Evaluator

The Parameter evaluator verifies the parameters extracted by the Parameter extractor to ensure that both necessary and optional parameters required by the Task executor are correctly identified. First, it checks for the presence of essential parameters, such as the target category and dataset path. If any of these parameters are missing due to LLM inference errors or user input omissions, the Parameter evaluator prompts the user for the missing information. Upon receiving a response, the Parameter extractor reprocesses the query, and the Parameter evaluator revalidates it. This iterative process continues until all necessary parameters are accurately extracted.

Then, the Parameter evaluator conducts a two-step validation of the dataset path. It first checks for the existence of the dataset folder; if not, the system requests a new path from the user. When the user provides a revised path, the Parameter extractor re-extracts and the Parameter evaluator revalidates it. This process repeats until a valid dataset path is obtained. Subsequently, the system checks whether the required data files are present within the folder. If any files are missing, the system prompts the user for additional input and repeats the verification cycle until all required files are correctly specified.

After validating the necessary parameters, the Parameter evaluator proceeds to assess the optional parameters. Although these are not required for execution, they still undergo validation to ensure correctness. If any optional parameters are missing, the Parameter evaluator provides suggested values or allows users to modify them. Additionally, if an extracted optional parameter is unsupported by the Task executor, the system suggests an alternative valid parameter.

Throughout this process, the Parameter evaluator dynamically interacts with the user using an LLM-based messaging system, applying structured prompt engineering techniques to ensure clarity. The overall workflow is illustrated in Figure 4, with key steps summarized in Algorithm 2. A detailed implementation is provided in Table S5 of the Supplementary Materials.

Algorithm 2 Parameter evaluator

$\mathrm{Procedure} \mathrm{evaluate}\_\mathrm{parameter} (Q$) Method necessary_parameter_checker (query) Inputs: $Q$: query (Dictionary of extracted parameters) Output: request_query (LLM-generated message) or False 1:   If missing parameters exist: $2:\hspace{1em} \mathrm{Return} \mathrm{LLM}\_\mathrm{engine}\_\mathrm{request} (Q$) 3:   Return False --- Method dataset_checker (path) Inputs: $P$: path (Dataset directory path) Output: (Boolean, request_query) 1:  If path exists and contains files: Return (True, False) $2: \mathrm{Else} \mathrm{Return} (\mathrm{False}, \mathrm{LLM}\_\mathrm{engine}\_\mathrm{request} (P$)) --- $\mathrm{Method} \mathrm{unsupport}\_\mathrm{parameter}\_\mathrm{checker} (Q$) Inputs: $Q$: query (Dictionary of extracted parameters) Output: suggestion_message, p_type, p $1: {\mathit{U}}_{\mathit{m}}$: models not in supported list $2: {\mathit{U}}_{\mathit{e}}$: evaluation flag not in supported list $3: {\mathit{S}}_{\mathit{m}}$: supported models list $4: {\mathit{S}}_{\mathit{e}}$: supported evaluation flags list $5: \mathrm{If} {\mathit{U}}_{\mathit{m}}$$\mathrm{or} {\mathit{U}}_{\mathit{e}}$: $6:\hspace{1em} \mathrm{p}\_\mathrm{type}, \mathrm{p} \leftarrow (1, {\mathit{S}}_{\mathit{m}}$$) \mathrm{if} {\mathit{U}}_{\mathit{m}}$ Else $7:\hspace{1em} \mathrm{p}\_\mathrm{type}, \mathrm{p} \leftarrow (2, {\mathit{S}}_{\mathit{e}}$$) \mathrm{if} {\mathit{U}}_{\mathit{m}}$ Else $8:\hspace{1em} \mathrm{p}\_\mathrm{type}, \mathrm{p} \leftarrow (3, [{\mathit{S}}_{\mathit{m}}$$, {\mathit{S}}_{\mathit{e}}$]) if U_m Else $9:\hspace{1em} \mathrm{Return} \mathrm{LLM}\_\mathrm{engine}\_\mathrm{request} (Q$), p_type, p 10: Return False

3.3. Task Executor

The Task executor is activated once the Parameter Evaluator validates the extracted keywords. As mentioned in Section 3.1, the Task executor comprises three core components: Manager, Task Evaluator, and Tool. The Manager structures the job flow by interpreting keywords and dynamically configuring the optimization process. The Task Evaluator monitors model performance and determines whether to terminate the optimization cycle or continue refinement. The Tool module consists of various functional submodules, including the Model Optimizer, Model Tuner, Model Trainer & Evaluator, Data Loader, Data Preprocessor, and Model Generator, which collectively handle model training and tuning tasks. A detailed explanation of each module is provided in the following sections.

To simulate real-world scenarios where users input arbitrary datasets, we designed the autonomous agent using widely adopted benchmark datasets in the PHM domain, with primary focus on fault diagnosis and RUL prediction. Specifically, the FordA datasets [70] ere used for binary fault classification, the Case Western Reserve University (CWRU) [71] Bearing Dataset for multi-fault classification, and the Commercial Modular Aero-Propulsion System Simulation (C-MAPSS) [72] dataset for RUL prediction.

By utilizing these datasets, we simulate scenarios where users input arbitrary datasets, ensuring that the agent can generalize across various real-world applications. The Task executor follows an iterative optimization process, progressively refining models through multiple cycles. Initially, the system starts with a narrow search space for hyperparameter tuning to ensure efficient exploration while minimizing computational overhead. As the optimization cycles progress, the range of tunable hyperparameters is gradually expanded, allowing the model to explore a broader search space and achieve incremental performance improvements. This stepwise approach balances computational efficiency with model accuracy, ensuring that resources are effectively utilized while optimizing performance. At the end of the process, the executor selects the top 5 models based on their evaluation flag and saves the results as a CSV file. The workflow is presented in Algorithm 3 as pseudocode.

Algorithm 3 Task executor

$\mathrm{Procedure} \mathrm{Task} \mathrm{executor} (\mathit{K}$) Inputs: $\mathit{K}$: a dictionary to construct job flow Output: A CSV file containing information on the top 5 models $1: \mathrm{Load} \mathrm{configuration} \mathrm{file} \mathrm{based} \mathrm{on} \mathit{K}$ $2: \mathrm{Manager} \mathrm{update} \mathit{C}$ (config) with keywords and struct a job-flow activating Tool 3:  num_cycle ← 1 4:  all_results ← ∅ 5:  Perform Optimization: 6:   While num_cycle ≤ max_cycle do 7:   Adjust hyperparameter range based on num_cycle 8:   Perform model tuning: 9:    Initialize the model components: $10:\hspace{1em} \hspace{1em}\mathrm{model}\_\mathrm{list} \leftarrow \mathrm{Model} \mathrm{generator} (\mathit{C}$) $11:\hspace{1em} \mathrm{data} \leftarrow \mathrm{Data} \mathrm{loader} (\mathit{C}$) 12:   Train and Evaluate models: 13:   result ← Model trainer & evaluator (model_list, data) 14:   Store results in all_results ← all_results ⋃ results 15:   Task evaluator analysis results and determine whether to increase num_cycle or not 16: Select top 5 models from all_results 17: Save a CSV file with the top 5 models 18: Return a CSV file

3.3.1. Manager

The Manager is responsible for coordinating the overall optimization process, serving as the central controller of the Task executor. The Manager activates specific submodules of the Tool that are suitable with extracted parameters, structuring a job flow that enables the system to execute the optimization process. As illustrated in Figure 2, the Manager collaborates closely with the Task evaluator, ensuring that model performance is continuously monitored and the optimization process is dynamically adjusted based on evaluation results.

When the Task evaluator returns feedback of re-optimization cycle, the Manager initiates an additional optimization cycle, modifying specific hyperparameters including batch size, learning rate and the number of epochs. This optimization continues until an optimal model is obtained, or the maximum optimization limit is reached. In this study, the maximum number of optimization cycles is set to five, considering both training time and computational efficiency. However, this limit can be adjusted based on resource availability and specific optimization requirements.

Through this series of processes, the Manager ultimately generates and stores a single CSV file, which is later provided to the Answer generator. This file contains information on the top 5 models based on performance metrics, including details such as the model name, corresponding performance score, and the storage path of the models.

3.3.2. Tool

The Tool comprises six main components: Model optimizer, Model tuner, Model trainer & evaluator, Data loader, Data preprocessor, and Model generator. Each module, activated by the job flow structured by Manager, performs its specific role in completing the assigned task. Figure 5 illustrates the overview of the Tool’s submodules, detailing the specific functions of each component in the optimization and model generation process.

The Data loader constructs data pipelines based on a predefined batch size, ensuring efficient data handling during training and evaluation. To enable unbiased model selection and prevent temporal and label leakage, it establishes train/validation/test partitions using a file structure aware strategy. This design is adopted for general applicability. In our study, the official Train/Test splits of FordA for binary fault classification and CMAPSS for RUL prediction were preserved, and a 20% validation subset was carved from training portion. CWRU for multi-fault classification, provided as class-wise folders without a predefined split, was partitioned using a stratified 60/20/20 rule. The Data preprocessor applies normalization and missing value imputation for fault diagnosis, while windowing, clipping, and conditionally Principal Component Analysis (PCA) are used for RUL prediction. Especially, conditional PCA is adopted for improving numerical conditioning and generalization. For prediction in PHM domain, strongly correlated sensor channels often induce multicollinearity, which can make the regression mapping ill-conditioned and hinder stable optimization [73,74]. To address this, the agent computes variance inflation factors (VIFs) on the training features and, when the maximum VIF exceeds 100, fits a PCA on the training split to decorrelate and compress the feature space. In this study, PCA dimensionality was tuned over {5, 7, 10}. The Model trainer & evaluator executes training and assesses performance using predefined evaluation flag applying early stopping to prevent unnecessary computation. The Model tuner dynamically adjusts model configurations based on evaluation feedback, refining the initial search range for parameters such as batch size and learning rate.

Additionally, as mentioned in Section 3.3.1, the Model optimizer interacts with the Task evaluator to adjust the tuning process dynamically. It enables the Model tuner to progressively explore a broader set of hyperparameters based on evaluation feedback. The Model generator module leverages deep learning architectures to ensure robustness and reliability in PHM tasks. By utilizing widely recognized models, it effectively captures temporal dependencies, extracts meaningful features, and models complex relationships in sequential sensor data.

For fault diagnosis, the module supports architectures such as LSTM, CNN-LSTM, MLP, TCN, and Transformer, which effectively capture spatial and temporal dependencies in sensor data. LSTM is widely applied in fault classification due to its ability to model dynamic system behaviors and identify fault patterns by capturing long-term dependencies in sensor signals [75]. For clarity, the LSTM updates are written separately as:

$${\mathit{f}}_t= \sigma ({\mathit{W}}_{\mathit{f}}{\mathit{x}}_{\mathit{t}}+ {\mathit{U}}_{\mathit{f}}{\mathit{h}}_{t-1}+ {\mathit{b}}_f)$$

(1)

$${\mathit{i}}_t= \sigma ({\mathit{W}}_i{\mathit{x}}_t+{\mathit{U}}_i{\mathit{h}}_{t-1}+{\mathit{b}}_i)$$

(2)

$${\tilde{\mathit{c}}}_t= tanh ({\mathit{W}}_c{\mathit{x}}_t+{\mathit{U}}_c{\mathit{h}}_{t-1}+{\mathit{b}}_c)$$

(3)

$${\mathit{c}}_t= {\mathit{f}}_t⨀ {\mathit{c}}_{t-1}+{\mathit{i}}_t⨀ {\tilde{\mathit{c}}}_t$$

(4)

$${\mathit{o}}_t=\sigma ({\mathit{W}}_o{\mathit{x}}_t+{\mathit{U}}_o{\mathit{h}}_{t-1}+{\mathit{b}}_o)$$

(5)

$${\mathit{h}}_t={\mathit{o}}_t⨀ \mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}\left({\mathit{c}}_t\right)$$

(6)

where $x_t$ is the input at time $t$, $h_t$ and $c_t$ are the hidden and cell states, $W$, $U$, $b$ are learnable parameters, $\sigma (·)$ and $tanh(·)$ are sigmoid and hyperbolic tangent activations, and $⨀$ denotes element-wise multiplication, For binary and multi-fault classification rules are given by:

$$\widehat{p}= \sigma ({\mathit{w}}^{⊺}{\mathit{h}}_T+\mathit{b})$$

(7)

$$\widehat{y}=1\left[\widehat{p}\ge \tau \right] (default \tau =0.5, tuned under class imbalance)$$

(8)

$$\widehat{\mathit{y}}=softmax \left(\mathit{W}{\mathit{h}}_T+\mathit{b}\right), \hspace{1em}\widehat{c}=argmax {\widehat{\mathit{y}}}_k$$

(9)

so that (7) applies a logistic link to the sequence summary $h_T$ to obtain the posterior for the positive class, and (8) implements the decision threshold $\tau$. (9) yields K-way posteriors that sum to one and select the most probable class.

CNN-LSTM enhances classification accuracy by combining spatial feature extraction with temporal modeling; a representative 1D convolution block is:

$$y_{c,t}= {\sum }_m{\sum }_{k=0}^{K-1}{\mathit{w}}_{c,m,k}{\mathit{x}}_{m,t+s·k}+ {\mathit{b}}_c$$

(10)

$$z_{c,t}=ReLU\left(y_{c,t}\right), p_{c,t}=MaxPool \left(z_{c,·}\right)$$

(11)

whose stacked outputs form a temporal embedding subsequently consumed by an LSTM head. MLP provides a simple yet effective feed-forward baseline and has been successfully used in gear-fault tasks [76]. TCN utilizes dilated convolutions to efficiently capture long-range dependencies in time-series fault data, achieving competitive classification performance compared to recurrent networks [77]. Transformer models leverage self-attention mechanisms to effectively model complex fault patterns and have been shown to outperform recurrent architectures in failure mode classification [78].

For RUL prediction, the module supports architectures such as LSTM, Bi-LSTM, CNN-LSTM, and Transformer, which are well-suited for capturing temporal dependencies in sequential sensor data. LSTM is widely used for RUL estimation due to its ability to model long-term dependencies in degradation patterns. LSTM effectively predicts the RUL of industrial components by capturing nonlinear degradation trends [75]. Bi-LSTM captures both forward and backward dependencies, providing a more comprehensive representation of degradation trends. This structure reduces information loss and enhances predictive accuracy, outperforming unidirectional LSTM models in RUL estimation [22]. CNN-LSTM enhances RUL prediction by integrating convolutional layers for spatial feature extraction with LSTM for temporal modeling. This hybrid model improves predictive accuracy in complex degradation scenarios by learning both local and sequential dependencies [28]. Transformer models leverage self-attention mechanisms to efficiently capture long-range dependencies in sequential data. They outperform recurrent models in RUL prediction by effectively modeling complex temporal relationships [25]. For the Transformer backbone used for RUL regression, the input token matrix $\mathit{X}\in {\mathbb{R}}^{T\times d}$ is combined with positional encodings:

$${\mathit{Z}}^{\left(0\right)}=\mathit{X}+\mathit{P}$$

(12)

$$\mathit{Q}={\mathit{Z}}^{\left(l\right)}{\mathit{W}}_Q, \mathit{K}={\mathit{Z}}^{\left(l\right)}{\mathit{W}}_K, \mathit{V}={\mathit{Z}}^{\left(l\right)}{\mathit{W}}_V, Attn \left(\mathit{Q},\mathit{K},\mathit{V}\right)=softmax \left({\displaystyle \frac{\mathit{Q}{\mathit{K}}^{⊺}}{\sqrt{d_k}}}\right)\mathit{V}$$

(13)

$$MHA \left({\mathit{Z}}^{\left(l\right)}\right)=Concat ({Attn}_1, \dots , {Attn}_H){\mathit{W}}_o$$

(14)

$${\tilde{\mathit{Z}}}^{\left(l\right)}=LN \left({\mathit{Z}}^{\left(l\right)}+MHA\left({\mathit{Z}}^{\left(l\right)}\right)\right), {\mathit{Z}}^{(l+1)}=LN \left({\mathit{Z}}^{\left(l\right)}+FFN\left({\tilde{\mathit{Z}}}^{\left(l\right)}\right)\right)$$

(15)

$$FFN \left(\mathit{Z}\right)=\varphi \left(\mathit{Z}{\mathit{W}}_1+{\mathit{b}}_1\right){\mathit{W}}_2+{\mathit{b}}_2$$

(16)

which feeds a backbone-agnostic linear readout to produce scalar RUL:

$$\widehat{r}= {\mathit{w}}^{⊺}{\mathit{h}}_T+\mathit{b}$$

(17)

In our implementation, ${\mathit{h}}_T$ is instantiated as the last hidden state for LSTM (concatenated forward/backward states for Bi-LSTM), the final hidden state of the LSTM head for CNN–LSTM, and the global average over final token embeddings for Transformer.

Hereafter, given the user-specific model list, the agent instantiates each backbone and trains in on the prepared train, validation, and test dataset. Optimization uses stochastic gradient descent or Adam [79] as configured. The binary fault classification and multi-fault classification optimize cross entropy with sigmoid or softmax heads, while RUL prediction optimizes mean squared error with a linear head. We monitor accuracy, precision, and recall for classification, and root mean squared error together with asymmetric score for RUL prediction. Early stopping on validation loss, learning rate reduction on plateaus, and checkpointing of the best validation model are enabled. After training, the model is evaluated on the held-out test set and the agent exports the metrics, predictions, final weights, and the configuration. Each submodule’s pseudocode is provided in Tables S6–S10 of the Supplementary Materials.

3.3.3. Task Evaluator

As illustrated in Figure 2, the Task evaluator determines whether the task’s objectives have been achieved based on results obtained at the end of an optimization cycle. To make this decision, the Task evaluator also utilizes parameters extracted from the query, and its role varies depending on the presence and values of two parameters: the evaluation flag and the target evaluation value. In cases where multiple models are evaluated simultaneously, these parameters help prioritize the selection of the most optimal model.

Concretely, selection and stopping are driven by metrics computed on the validation split. The evaluator applies the user-specified evaluation flag as the governing criterion: loss-type measures such as validation loss, root mean square error, and the asymmetric score are minimized, whereas accuracy-type measures such as accuracy, precision, recall, and F1 score are maximized. When a target evaluation value is supplied, success is declared once the governing metric satisfies the target, that is, less than or equal to the target for loss-type measures and greater than or equal to the target for accuracy-type measures, and the search terminates. When no evaluation flag is provided, validation loss serves as the default criterion. The test split remains untouched during selection and is evaluated only once after the search. After the final cycle, all trials are ranked by the governing criterion, and the top five configurations are returned together with the checkpoint path of the best model for downstream use. Further details on the implementation and decision-making process of the Task Evaluator are provided in Algorithm 4 as pseudocode.

Algorithm 4 Task evaluator

$\mathrm{Procedure} \mathrm{task}\_\mathrm{evaluator} (\mathrm{sorted}\_\mathrm{result}, K_v$$, C_E$, ascending) Inputs: sorted_result: a list containing sorted evaluation results $K_v$: threshlod value for evaluation $C_E$: metric for evaluation performance ascending: boolean flag indicating sorting order Output: satisfied: boolean indicating if optimization criteria is met best_result: best evaluation result 1:   best_result ← sorted_result [0] $2:\hspace{1em} \mathrm{best}\_\mathrm{score} \leftarrow \mathrm{best}\_\mathrm{result} [C_E$] $3:\hspace{1em} \mathrm{If} K_v$ is None do 4:   return True, best_result $5:\hspace{1em} \mathrm{Else} \mathrm{If} (\mathrm{ascending} \mathrm{and} \mathrm{best}\_\mathrm{result} \le K_v$$) \mathrm{or} (\mathrm{not} \mathrm{ascending} \mathrm{and} \mathrm{best}\_\mathrm{result} \ge K_v$) do 6:   return True, best_result 7:   Else do 8:   return False, best_result

3.3.4. Answer Generator

The Answer generator is implemented using the GPT-4o-based LLM engine to deliver task execution results to the user. Like the Parameter extractor and Parameter evaluator, it is designed to accommodate multiple LLMs depending on the task requirements. It plays a key role in generating messages based on the CSV file. First, the Answer generator reads the CSV file produced by the Task executor, which contains the top-performing models ranked by evaluation flag. It then converts this data into a structured dataframe, extracting key details such as model names, evaluation scores, and storage paths. Using this information, the system identifies the best-performing model and generates a well-structured response for the user.

Since a simple text listing approach may reduce readability compared to directly reading a CSV file, the system must provide a more intuitive message. To address this, the Answer generator employs prompt engineering techniques to ensure that the response is presented in an intuitive and interactive format. Instead of simply listing CSV contents, it reformats the information into natural language, making it easier for users to interpret the results. The simplified workflow of the Answer Generator is presented in Algorithm 5, while a more detailed version with implementation specifics is available in Table S11 of the Supplementary Materials.

Algorithm 5 Answer generator

Procedure generate_answer () Inputs: None Output: M: response_message 1:  result_files ← list_files (get_latest_directory (“./result”)) 2:  best_result ← first_row(read_csv (result_files [0])) 3:  column_names ← [“model”, col (3), “path”] 4:  result_values ← [best_result [1], round (best_result [2], 4), best_result [3]] 5: result_summary ← create_dict (column_names, result_values) 6:  request_query ← “Summarize:“ + str (result_summary) + “in a clear format.” 7:  Return LLM_generate (“Generate summary”, request_query)

4. Results

4.1. Evaluation Setup

As mentioned in Section 3.1, to account for diverse query structures and potential ambiguities, we define eight use cases. These use cases enable a systematic assessment of the autonomous agent’s ability to process various query types and ensure the robustness of the Parameter extractor and Parameter evaluator in handling diverse input scenarios. The predefined use case types are follows:

• Case 1. Vague query: Lacks optional details but can run with default values or require additional user input.
• Case 2. Well-defined query: Contains both necessary and optional parameters, requiring no further input.
• Case 3. Query with missing necessary parameters: Lacks essential parameters, requiring user input for validation.
• Case 4. Query with unsupported parameters: Contains undefined parameters, making execution impossible.
• Case 5. Lexical drift query: Includes errors (e.g., typos, grammatical mistakes) that hinder Parameter extractor.
• Case 6. Query optimizable for result: Can meet evaluation criteria by increasing optimization cycles.
• Case 7. Query unoptimizable for result: Cannot meet evaluation criteria even with more optimization cycles.
• Case 8. Query casing dataset path issue: Specifies an incorrect dataset path, requiring user correction.

4.2. Performance Evaluation for Agent

Further illustrating these use cases, Table S13 of Supplementary Materials presents a comprehensive list of 100 queries categorized by use case type, providing an overview of the various query formulations utilized in this study. The experimental results obtained from processing these 100 queries are detailed in Section 4.2, where we evaluate the autonomous agent’s performance across different query types. The agent’s ability to accurately extract and validate parameters, efficiently execute model generation tasks, and process queries across different LLM engines is assessed.

In addition, to validate the effectiveness of the proposed agent, we also conducted a baseline comparison between the agent-generated models and expert-designed models reported in prior studies using the same datasets. As summarized in

Table 2. Comparison of agent generated model performance with literature-reported.

Task	Dataset	Metric	Methods	Value
Binary fault classification	FordA	Accuracy	Ours	0.97
			Wichard [80]	0.95
			Adeodato et al. [81]	0.95
			Schlegel et al. [82]	0.89
Mult-fault classification	CWRU	Accuracy	Ours	0.95
			Aziz et al. [83]	0.97
			Raj et al. [84]	0.99
RUL prediction	C-MAPSS (FD001)	Asymmetric Score	Ours	380.74
			Muneer et al. [85]	223.00
			Ensarioğlu et al. [86]	437.20

, the models produced by the agent achieved comparable predictive performance to literature-reported baselines, with only marginal differences in accuracy and asymmetric score. These results confirm that the agent can autonomously generate models that are not only valid and reliable but also competitive with manually engineered approaches, thereby demonstrating its practical utility.

Figure 6 provides a detailed visualization of the agent’s internal workflow, specifically for use case 3, illustrating the interactions between its modules, including the flow of inputs and outputs at each stage. Concretely, consider a user query that omits the dataset path: “Create predictive systems with CNNLSTM and BILSTM for useful life estimation; RMSE must not exceed 30.” The parameter extractor parses the request into structured fields (target category, model list, evaluation flag, target value), and the evaluator detects the missing path and prompts the user. After the path is supplied, the system re-parses and validates completeness; the Manager then orchestrates execution, and the Tool instantiates the CNN-LSTM and Bi-LSTM backbones, trains them with early stopping, learning-rate scheduling, and checkpointing, and logs validation metrics.

The Task evaluator applies the governing metric RMSE, compares it with the user-specified target, selects the best configuration, and reports the chosen model, test metrics, and artifact locations. Figure 6 is annotated to reflect these stages from user query through parameter extraction and validation to orchestration, training, and model selection so the flow of inputs and outputs is explicit.

Table 3. Definition of Use Case for Evaluation.

Use Case Number	Use Case Name	Explanation
1	Vague query	Lacks optional details but can run with default values or require additional user input.
2	Well-defined query	Contains both necessary and optional parameters, requiring no further input.
3	Query with missing necessary parameters	Lacks essential parameters, requiring user input for validation.
4	Query with unsupported parameters	Contains undefined parameters, making execution impossible.
5	Lexical drift query	Includes errors (e.g., typos, grammatical mistakes) that hinder Parameter extractor.
6	Query optimizable for result	Can meet evaluation criteria by increasing optimization cycles.
7	Query unoptimizable for result	Cannot meet evaluation criteria even with more optimization cycles.
8	Query causing dataset path issue	Specifies an incorrect dataset path, requiring user correction.

defines the eight use cases for the experiment.

Table 4. Summary of autonomous agent processing across different use case types.

Use Case	Process	Value
1	User query input	“Develop models to predict remaining operational life; data is available at ‘./dataset’.”
	[Trial 1] Parameter extractor	{‘target_category’: ‘prediction’, ‘dataset_base_dir’: ‘./dataset’, ‘evaluation_flag’: None, ‘model_list’: None, ‘target_evaluation_value’: None}
	[Trial 1] Parameter evaluator	False
	User’s additional query input	“I will choose cnn combined lstm.”
	[Trial 2] Parameter extractor	{‘target_category’: ‘prediction’, ‘dataset_base_dir’: ‘./dataset’, ‘evaluation_flag’: ‘VALIDATION_LOSS’, ‘model_list’: [‘CNNLSTM’], ‘target_evaluation_value’: None}
	[Trial 2] Parameter evaluator	True
	Task evaluator	True—Cycle 1
	Answer generator	Model Training Result Summary: - Model Used: CNNLSTM_1_1 - Validation Loss: 400.0396 - Model Saved To: ./result Summary: The experiment utilized the CNNLSTM_1_1 model, achieving a validation loss of 400.0396. The trained model has been successfully saved at the specified location for future use and reference.

illustrates Use case 1 as a representative example, and results for the remaining user queries are provided in Table S14 of the Supplementary Materials. Complete results are provided in Tables S14–S21 of the Supplementary Materials.

For deployment cost considerations, the annual operating cost can be approximated as per-query API cost × agent runs per year × LLM calls per run, if > 1. Using the per-query costs measured in this study, see Supplementary Table S2, 100 runs per day, about 36,500 per year, with one LLM call per run correspond to approximately $165.71 for GPT-4o, $7.74 for Gemini-2.0-Flash, and $406.50 for GPT-4, while self-hosted models such as Llama-3.2 incur no API fee but may require on-premises infrastructure costs. Actual deployments may vary with prompt length and the number of calls per run.

5. Discussion

The proposed autonomous agent enables non-specialists to effectively automate PHM model generation for fault diagnosis and RUL prediction through natural language interaction, while also providing options and guidance to handle ambiguous or incomplete queries, thereby integrating LLM-based query processing, parameter validation, and task execution into a seamless workflow. Through systematic evaluation across 100 use cases, the system demonstrated its ability to extract relevant parameters, validate user inputs, and execute model development tasks with minimal manual intervention. One of the demonstrated results was the comparative analysis of multiple LLM engines, including GPT-3.5-turbo, GPT-4, GPT-4o, GPT-4o-mini, Gemini, and LLaMA, revealed their respective trade-offs in accuracy, computational cost, and response time. GPT-4o, with the highest accuracy 0.96 and a balanced computational cost $0.00455, was selected as the optimal model for this study.

Additionally, the Parameter extractor processes natural language queries, extracting necessary and optional parameters, while the Parameter evaluator verifies completeness and correctness. The Task executor structures the job flow and activates necessary tools for data loading, preprocessing, model selection, training, evaluation, and optimization, ensuring an efficient execution process. The system also employs an iterative optimization mechanism, where models are retrained and hyperparameters are adjusted until the best-performing configuration is achieved. Finally, the Answer generator compiles results into a structured output, enhancing user interaction through LLM-based natural language responses. The effectiveness of this automation was confirmed through experimental evaluations, demonstrating significant reductions in manual intervention, processing time, and complexity in AI model configuration.

Despite these advantages, several challenges remain in LLM-based parameter extraction and dataset adaptability. The reliance on LLMs for query processing presents trade-offs between accuracy, cost, and response time, with incorrect extractions potentially requiring multiple validation cycles, increasing computational overhead. Selecting the most cost-efficient and task-optimized LLM engine for different query processing stages remains a key factor in improving system performance. Another challenge concerns dataset adaptability within the Task executor. This study utilized predefined open datasets for each task. While these datasets were effective for evaluating the agent’s performance, real-world applications require greater flexibility to accommodate datasets with varying formats, feature distributions, and preprocessing requirements. It is difficult to standardize data ingestion and transformation across different user-provided datasets.

To address these limitations, future research should focus on adaptive LLM selection strategies and enhanced dataset handling mechanisms. Instead of relying on a single LLM engine for all processes, a hybrid LLM approach can be employed, where lightweight models (e.g., GPT-4o-mini, Gemini-2.0-flash) handle early-stage parameter extraction, while higher-accuracy models (e.g., GPT-4o) are reserved for final validation and answer generation. This dynamic model allocation would optimize both processing time and cost, ensuring an efficient and scalable system. Furthermore, a modular and adaptive data ingestion pipeline should be developed to enhance dataset handling. This includes automated dataset structure detection, which allows the system to identify key features, missing values, and formatting inconsistencies. Additionally, a dynamic preprocessing framework should be implemented to automatically adjust feature engineering steps based on dataset characteristics, minimizing manual intervention. By integrating these mechanisms, the system can handle a broader range of datasets more effectively, improving usability, adaptability, and automation across diverse industrial applications.

Overall, this study demonstrates that LLM-powered automation can significantly improve AI-driven predictive modeling, reducing dependency on manual query structuring and parameter selection, the system bridges the gap between domain expertise and AI-driven predictive modeling, enhancing accessibility for non-experts. Addressing these remaining challenges through hybrid LLM strategies and adaptive dataset processing will further enhance the autonomous agent’s flexibility and real-world applicability, making it a more robust and efficient tool for fault diagnosis, RUL prediction, and other predictive maintenance tasks.

6. Conclusions

In this study, we proposed an autonomous agent that leverages LLMs to automate the process of fault diagnosis and RUL prediction in industrial PHM applications. By integrating natural language processing capabilities with domain-specific tools, the agent efficiently extracts key parameters from user queries, validates inputs, and executes the model development pipeline. The experimental results demonstrated the agent’s ability to minimize manual intervention while improving accessibility for non-experts, thereby enhancing scalability and usability in industrial maintenance operations.

From a PHM perspective, the LLM functions as an orchestration layer that translates natural language maintenance intents into an end-to-end workflow covering data preparation, model configuration, training, evaluation, and model selection while upholding standard validation discipline. Instead of requiring expert scripting, the agent automatically chooses task-appropriate modeling patterns for fault diagnosis and RUL prediction, applies consistent objective and metric regimes, and returns artifacts that PHM stakeholders can act on. This abstraction reduces the expertise burden, improves reproducibility through logged configurations and checkpoints, and tightens the link between analytics outputs and maintenance planning.

Through systematic evaluations, we compared the performance of various LLM engines, including GPT-4o, GPT-4, Gemini, and LLaMA, identifying GPT-4o as the most suitable model due to its high accuracy and balanced computational efficiency. Furthermore, the autonomous agent successfully processed diverse query types, ensuring robustness in parameter extraction, validation, and task execution. The iterative optimization mechanism implemented in the Task executor enabled efficient hyperparameter tuning, ensuring optimal model performance for both fault diagnosis and RUL prediction.

Despite its promising performance, certain challenges remain, particularly in dataset adaptability and computational efficiency. The reliance on LLMs for query processing introduces trade-offs between accuracy, processing time, and cost, while the handling of diverse datasets requires further improvements in adaptive preprocessing mechanisms. Future research should explore hybrid LLM strategies that dynamically allocate different models for various processing stages to enhance cost-effectiveness and accuracy. Additionally, the development of a more flexible data ingestion and preprocessing framework will be crucial for extending the applicability of the autonomous agent across diverse industrial datasets.

Overall, this study highlights the potential of LLM-based autonomous agents for PHM. By bridging the gap between domain expertise and predictive modeling, our approach enables more efficient, scalable, and user-friendly solutions for fault diagnosis and RUL prediction. Further advancements in adaptive LLM selection and data handling strategies will further enhance the robustness and practicality of the proposed system, paving the way for broader adoption in industrial PHM applications.