Data-Driven Digital Twin Framework for Predictive Maintenance of Smart Manufacturing Systems

Abstract

A Digital twin (DT) enables the acquisition and subsequent analysis of large amounts of process data. Various machine learning (ML) algorithms exist for analysis and prediction that can be used in this scenario. However, there is very little understanding of the relative merit of these methods. This paper proposes a DT framework in the context of predictive maintenance in smart manufacturing to compare the prediction efficacy of prevalent ML models. Data-driven models were developed using machine learning algorithms to predict surface roughness and power consumption during a CNC turning operation. Three process parameters, namely cutting velocity, feed rate, and depth of cut, and two dependent parameters, surface roughness and power consumption, were selected for model development. Seven ML algorithms were tested for each response parameter: Linear Regression, XGB Regressor, Random Forest Regressor, Average Ensemble, AdaBoost Regressor, SVR, and MLP. The results of the comparative analysis of the ML algorithms showed that the Random Forest Regressor is the best prediction model for surface roughness, with the highest R² (94.2% ± 2.4%), lowest MAE (0.011 ± 0.002), lowest MAPE (15.6% ± 4.0%), and lowest RMSE (0.017 ± 0.003), while the XGB Regressor demonstrated the best performance for power consumption prediction, with the highest R² (98.9% ± 0.5%), lowest MAE (22.513 ± 4.424), lowest MAPE (3.0% ± 0.5%), and lowest RMSE (42.650 ± 8.933). The best-performing machine learning algorithm was subsequently utilized in the data-driven models, helping to achieve an improved surface finish. This enables predictive maintenance, reducing energy usage for more sustainable production.

1. Introduction

Technological advancements have significantly transformed manufacturing systems under the emerging idea of Industry 4.0 technologies, namely the Internet of Things (IoT), machine learning, and big data analytics [1,2,3]. Another new evolution in the fourth Industrial Revolution, namely cyber-physical production systems (CPPS), provides critical inputs for semi-autonomous and autonomous decision-making for the future of manufacturing systems [4,5]. Cyber-physical production systems combine physical manufacturing systems with advanced networking technologies, enabling efficient monitoring and handling of physical processes [6]. CPPS enables real-time, enhanced performance in manufacturing systems that leads to customized product development and resource-efficient production, increasing reliability and resilience in production processes [7].

Integrating these technologies into the existing manufacturing systems enables better decision-making by enhancing overall quality, productivity, and sustainability [8,9]. These technologies enable the manufacturing systems to become ‘smart’. Smart systems are knowledge-driven, well-connected, and dynamically adaptive to changes [10]. As a result, manufacturers can achieve higher quality standards and increased production rates with more sustainable practices, which enable them to stay competitive in the rapidly changing industrial scenario. With the increased prevalence and sophistication of various machine learning tools that help with analysis and prediction, deciding which tool to use is often challenging [11].

A major component of CPPS is the digital twin, which is basically a digital counterpart of the physical system. A DT is a virtual model that utilizes real-time data from a physical object to simulate its behavior [12]. Digital twins are very promising in systems where real-time monitoring is essential, as they assist in the supervision and analysis of manufacturing systems, facilitating effective fault diagnosis and process planning [13]. DTs significantly enhance the operator’s decision-making capabilities by centralizing and compiling relevant information about the system. They have the potential to accurately analyze the performance of a process or system based on real-world data, such as working conditions and real-time status of physical elements, captured through sensors that are attached to the manufacturing system. To ensure that the digital twin closely aligns with the behavior of the physical system, periodic updates of the process parameters are necessary to minimize the variation between the actual and simulated behavior, thus improving accuracy [14]. These types of ‘time-maintained’ digital twins represent a key goal for smart manufacturing.

Data-driven digital twinning plays a crucial role in enhancing the reliability and performance of the manufacturing systems. Incorporating machine learning into this digital twin improves its functionality and potential. Advanced ML algorithms, trained on huge amounts of operational and real-time sensor data, allow for the precise analysis of system behavior and the accurate prediction of future outcomes [15]. This enables the DT to identify potential failures at an early stage, allowing for timely preventive measures to improve the overall efficiency and real-time performance of manufacturing systems.

Studies have demonstrated the effectiveness of such data-driven digital twinning frameworks across various manufacturing applications. For instance, in the cooling tower systems of automotive production plants, DTs have been utilized to optimize energy consumption and cooling efficiency using polynomial and linear regression techniques [16]. Similarly, in aluminum die-casting manufacturing, a real-time visual quality prediction system using XGBoost and deep learning-based defect detection achieved over 95% defect identification accuracy, reducing false detection rates and improving production reliability [17]. Akinsolu and Zribi [18] introduced a comprehensive framework which integrates 19 different ML algorithms, such as Random Forests, SVR, and Decision Trees, and evaluates and ranks model performance within a continuous-flow manufacturing environment. This framework assists in identifying the most suitable predictors for the accurate modeling of system variables like temperature and pressure [18]. Collectively, these studies highlight the successful application of DT in improving operational visibility, predictive capabilities, and system adaptability in industrial settings.

Combining AI–ML and big data, digital twinning is evolving rapidly, and there are potentially many application possibilities in the industry such as real-time equipment health monitoring, process optimization, and predictive maintenance scheduling in manufacturing systems [19,20]. For example, digital twins can track machine performance to anticipate failures, optimize cutting parameters in machining operations, and improve energy efficiency across production lines. These capabilities allow for real-time monitoring and data-driven decision-making, providing significant benefits in the predictive maintenance of physical systems [21,22]. Physics-based simulation and the digital twin concept can also be used to calculate the Remaining Useful Life (RUL) of manufacturing equipment, offering accurate RUL predictions that support maintenance planning, production scheduling, and operational decision-making processes [23].

Traditional maintenance methods are usually reactive, addressing issues after they occur, leading to increased downtimes, sudden production stops, and operational costs. Predictive Maintenance (PdM) can be more effective than traditional maintenance approaches, such as Reactive Maintenance (RM) and Preventive Maintenance (PM), in the context of Industry 4.0 [24]. PdM is a strategy used to forecast equipment failures, enabling preventive actions and maintenance to be planned before actual failure [25,26,27]. PdM has the potential to reduce unexpected downtimes and extend the lifespan of equipment, which in turn improves the efficiency and reliability of the physical system [28,29]. This approach uses inspection and diagnosis to detect anomalies in the equipment, facilitating early preventive actions [30,31]. In this way, PdM transforms unexpected failures into planned repairs, reducing the chances of production downtime and associated costs.

In some cases, simple regression models can be used for predictive modeling [32]. For example, different regression models based on correlation analyses can be used to effectively predict orbital welding time in single-item production systems. However, their effectiveness highly depends on process knowledge and the standardization of data.

The integration of DT technology with ML offers significant potential to enhance smart manufacturing systems, particularly in the field of predictive maintenance [33,34]. Although numerous ML algorithms exist for predictive maintenance, there is limited literature comparing their performance when applied within a DT framework for manufacturing processes [35]. This highlights a key research gap, as selecting the most appropriate ML algorithm is essential to ensuring accurate and reliable DT-based predictions. To aid algorithm selection, this study proposes and utilizes a data-driven digital twin framework to evaluate and compare the relative performance of various ML algorithms in predicting critical response parameters, particularly surface roughness and power consumption during CNC turning operations. The study aims to identify the most effective model for incorporation into a DT environment that enables real-time monitoring and preventive maintenance. The application area relates to a CNC turning process to predict energy usage and surface roughness to achieve more sustainable and cost-effective production. In the context of this application, an accurate prediction model will help achieve a better surface finish for high-quality and customized products.

2. Methods

2.1. Framework Development

2.1.1. Maintenance System Architecture

An architectural overview of the proposed digital twin framework, focused on predictive maintenance, is illustrated in Figure 1. This framework consists of two cycles, one for comparing and selecting the algorithms and the other for the implementation part, where the operator performs decision-making regarding maintenance, preventive actions, and process parameter control. In the first cycle, the data gathered through sensors attached to the physical system (a CNC turning machine in the present case) undergo data pre-processing to prepare it for implementing ML algorithms. These data are used to train and test various ML algorithms. The best algorithms that provide the most accurate predictions are utilized in the data-driven models to predict key response parameters. This may assist the operator in the predictive maintenance of manufacturing systems for improved performance and decision-making. Additionally, if another algorithm starts predicting better later in the system lifecycle, possibly because of user inputs, system degradation, etc., it becomes the algorithm of choice. The framework suggests that the system periodically evaluates the performance of multiple machine learning algorithms as new data are collected during operation. This process can be automated by using performance metrics to assess the effectiveness of each algorithm on the updated dataset. When a different algorithm consistently outperforms the current model based on the predefined metrics, the operator can switch to the parameter values suggested by the now better-performing algorithm. This adaptive mechanism enables continuous optimization of the system’s predictive capability over its lifecycle. Thus, this framework effectively assists in providing feedback to the operators, enabling them to take preventive actions on the physical system before any actual failure occurs. This subsequently allows for the continuous optimization of process parameters throughout the manufacturing process, resulting in sustained prediction accuracy despite process drift.

2.1.2. Implementation of the Proposed System Architecture

The framework of the proposed system offers a sequential procedure for developing a data-driven machine learning-based DT model for predicting critical response parameters. This framework involves four stages: data collection, data pre-processing, training and testing ML models, and selecting the best predictive models.

In this study, we have utilized the dataset based on the CNC turning process, derived from experimental observations conducted by Andre Dorigueto Canal and Andrerson Vincente Borille at the Competence Center in Manufacturing (CCM), a laboratory within the Aeronautics Institute of Technology (ITA), Brazil [36]. The dataset provided readings on surface roughness and force measurements that assist in evaluating machining performance.

These experiments were conducted on AISI H13 steel (mean hardness = 200 HV) using a ROMI E280 CNC lathe with a maximum rotational speed of 4000 rpm and a nominal power of 18.5 kW. A Sandvik Coromant cutting tool (ISO TNMG 16 04 04-PE 4425) with a tool shank (ISO MTJNL 2020K 16M1) was used for the machining process. Surface roughness was measured with a Mitutoyo Surftest (SJ-210) portable roughness tester. Additionally, cutting forces were measured using a Kistler Type (9265B) dynamometer connected to a Kistler Type charge amplifier and operated through Kistler Dynoware (2825A) data acquisition software.

After gathering the required data, the dataset was pre-processed to prepare it for use in machine learning models. Data pre-processing involved two steps: data cleaning and feature engineering.

Data Cleaning: The dataset was comprised of 27 columns and 324 samples. Two columns that contained inconsistent and invalid data were removed. These fields were “Run_ID”, an experimental identifier without any analytical relevance, and “C_TIME”, which had excessive missing values. All other columns were numeric type except for “Position” which was a categorical variable describing work piece placement. To enable numerical analysis, one-hot encoding was applied to “Position”, converting the categorical variable into a binary representation. Once all features were numeric, a correlation matrix was generated using the matplotlib and seaborn libraries to see which features correlated strongly with surface roughness. All of the features with a correlation coefficient between −0.5 and 0.5 were dropped from the dataset. This was subsequently analyzed using expert domain knowledge to identify the most relevant process parameters that effectively map with the response parameters and significantly influence the output or response parameters to be predicted. Based on the analysis, a normalized list of five key process parameters (velocity, feed, and depth of cut for roughness; force and velocity for power) was selected to train and test the ML algorithms.

Additionally, hyperparameter tuning was performed for each ML algorithm using Ridge Regression, Grid Search, and Randomized Search methods to identify the hyperparameters that maximize model performance. The results of the hyperparameter tuning, including the best parameters for each model, are illustrated in

Table 1. ML Hyperparameter Tuning Methods and Best Parameters for Models in Surface Roughness Prediction.

Model	Tuning Method	Best Parameters
Linear Regression	Ridge Regression	{‘alpha’: 1, ‘fit_intercept’: True, ‘solver’: ‘auto’}
XGB Regressor	Randomized Search	{‘colsample_bytree’: 0.78, ‘gamma’: 0.007, ‘learning_rate’: 0.29, ‘max_depth’: 8, ‘n_estimators’: 341, ‘subsample’: 0.75}
Random Forest Regressor	Grid Search	{‘max_depth’: None, ‘max_features’: ‘log2’, ‘min_samples_leaf’: 1, ‘min_samples_split’: 5, ‘n_estimators’: 100}
AdaBoost Regressor	Grid Search	{‘estimator__max_depth’: 3, ‘learning_rate’: 1.0, ‘loss’: ‘square’, ’n_estimators’: 50
SVR	Randomized Search	{‘C’: 1.03, ‘epsilon’: 0.105, ‘gamma’: ‘scale’, ‘kernel’: ‘rbf’
MLP	Randomized Search	{‘activation’: ‘tanh’, ‘alpha’: 0.00208, ‘batch_size’: 128, ‘hidden_layer_sizes’: (50, 50), ‘learning_rate_init’: 0.0619}

for surface roughness prediction and

Table 2. ML Hyperparameter Tuning Methods and Best Parameters for Models in Power Consumption Prediction.

Model	Tuning Method	Best Parameters
Linear Regression	Ridge Regression	{‘alpha’: 0.01, ‘fit_intercept’: True, ‘solver’: ‘svd’}
XGB Regressor	Randomized Search	{learning_rate = 0.29, max_depth = 8, n_estimators = 341, subsample = 0.75, colsample_bytree = 0.78, gamma = 0.007}
Random Forest Regressor	Grid Search	{‘estimator__max_depth’: 3, ‘learning_rate’: 1.0, ‘loss’: ‘square’, ‘n_estimators’: 200}
AdaBoost Regressor	Grid Search	{‘estimator__max_depth’: 3, ‘learning_rate’: 1.0, ‘loss’: ‘square’, ‘n_estimators’: 50}
SVR	Randomized Search	{‘C’: 98.96484985318548, ‘epsilon’: 0.03416347415306914, ‘gamma’: ‘auto’, ‘kernel’: ‘rbf’}
MLP	Randomized Search	{‘activation’: ‘logistic’, ‘alpha’: 0.010151348978007608, ‘batch_size’: 32, ‘hidden_layer_sizes’: (50, 50), ‘learning_rate_init’: 0.015226341829186325}

for surface power consumption prediction.

Out of the two target response parameters, the first one, the surface roughness of the machined workpiece, was already present in the dataset. The second response parameter, i.e., power consumption, was absent from the dataset. Fortunately, the dataset had values for cutting force and velocity. This assisted in estimating the missing parameter indirectly by using the available features.

Feature Engineering: Feature engineering was used to transform the available data into more meaningful features that enhanced the overall efficiency of the predictive model by using the fundamental relationships between input and output variables.

One such critical feature introduced was power consumption (P), a critical parameter for machine maintenance. Since power consumption is directly influenced by the cutting force (F) and the cutting velocity (V) during a machining operation, it was derived using the fundamental physical principle of mechanical power in translational motion expressed in Equation (1):

P = FV

(1)

The above relationship has been derived from Newtonian mechanics, where power is defined as the rate of work done, and work as the product of force and displacement. For a constant force applied in the direction of motion along with some displacement over time, power can be expressed as the product of force and velocity. This formulation has been used to estimate power consumption during machining operation.

Apart from this, no additional feature engineering steps were performed in this study.

2.1.3. Training and Testing of ML Models

After acquiring the relevant process parameters, the prediction models were selected, trained and evaluated. Various machine learning algorithms were used for evaluation. These ML algorithms were deployed in Python 3.13.3, with 80% of the dataset used for training and the remaining 20% for evaluating the accuracy on unseen data. This training and testing procedure was subsequently repeated for each ML model to determine their accuracy rates.

The performance of these algorithms was evaluated, and the best-performing ones were identified for potential use in the data-driven predictive models. These models enable precise predictions that support the predictive maintenance of manufacturing systems, leading to improved performance and better decision-making.

2.2. Data-Driven Predictive Models

The study focused on training data-driven predictive models using machine learning algorithms. Seven machine learning algorithms were selected for evaluation: Linear Regression, the XGB Regressor, the Random Forest Regressor, Average Ensemble, the AdaBoost Regressor, SVR, and MLP. The training and testing of all these models was done using the Scikit-learn Python library.

Models were selected which represent the whole spectrum of the machine learning models. The selected models are widely recognized in the literature for their prediction accuracy. Ensemble algorithms, namely the XGB (extreme gradient boosting) Regressor, the Random Forest Regressor, Average Ensemble, and the AdaBoost (adaptive boosting) Regressor, were selected due to their high prediction power and robustness relative to other algorithms [37]. Neural network algorithms, MLP (multilayer perceptron), and SVR (support vector regression) were evaluated due to their prevalence and universal approximation. Linear regression was chosen as a baseline model and for its interpretability [38]. Random Forest was selected as a complex and non-linear model. SVR represents a kernel-based flexible prediction model. XGBoost and AdaBoost were selected due to their high prediction power and robustness relative to other algorithms.

These ML algorithms were evaluated to test the effectiveness of prediction for surface roughness and power consumption. Based on the results of the evaluation, two individual predictive models, Random Forest and XGBoost, were identified as the most suitable ML algorithms due to their accuracy of prediction.

The first model was utilized to predict surface roughness, an important parameter of workpieces for high-quality and reliable products. This model used depth of cut, cutting speed, and feed rate as input parameters to successfully predict the surface roughness using machine learning algorithms.

The second model was used in the prediction of power consumption, which is crucial for cost reduction and energy-efficient production. The related data fields obtained from the dataset have not shown any correlation with power consumption. Thus, based on expert knowledge, feature engineering was used to derive power consumption using cutting force and cutting speed as input parameters, to build a meaningful predictive model that assists in the successful prediction of power consumption.

The data-driven predictive models thus formed assist in the predictive maintenance of the system in achieving a high surface finish with minimum energy usage for more efficient and economical production.

3. Results

After the training phase, the accuracy rate of each ML algorithm was analyzed by comparing the predicted value with the actual value of measurements. For the purpose of evaluating the prediction performance of every algorithm, various statistical measures, namely $R^2$ (R-squared), MAE (Mean Absolute Error), RMSE (Root Mean Square Error), and MAPE (Mean Absolute Percentage Error), were utilized as illustrated in Table 1 and Table 2.

$R^2$ indicates the quantity of total variation in the output variable which the model managed to reduce. A higher $R^2$ value signifies a better fit for the model [39].

$R^2$ is calculated as in Equation (2):

$$R^2=1-{\displaystyle \frac{{\sum }_{i=1}^n{\left(yi-\widehat{y_i}\right)}^2}{{\sum }_{i=1}^n{\left(yi-\overline{y}\right)}^2}}$$

(2)

MAE is a measure of the absolute value of the average difference between the predictions and the actual failures in the dataset. A small MAE value indicates better predictions for failures [40].

The formula for MAE is as shown in Equation (3):

$$MAE={\displaystyle \frac{{\sum }_{i=1}^n\left|y_i-\widehat{y_i}\right|}{n}}$$

(3)

In the above equation, $\widehat{y_i}$ is the predicted value and $y_i$ is the actual value.

RMSE represents the standard deviation of the prediction errors. A model’s prediction is considered more accurate if the RSME value turns out to be smaller [41].

RMSE is calculated as shown in Equation (4):

$$RMSE=\sqrt{{\displaystyle \frac{{\sum }_{i=1}^n{\left(y_i-\widehat{y_i}\right)}^2}{n}}}$$

(4)

MAPE is the mean of absolute percentage errors. Unlike MAE, MAPE measures the variation in percentage compared to the absolute values [42].

MAPE is calculated as shown in Equation (5):

$$MAPE={\displaystyle \frac{1}{n}}\sum _{i=1}^n\left|{\displaystyle \frac{A_i-P_i}{A_i}}\right|$$

(5)

In the above equation, $A_i$ denotes the i-th actual value, $P_i$ denotes the i-th predicted value, and n denotes the number of data points.

For surface roughness prediction, the performance results of ML algorithms were evaluated as illustrated in

Table 3. ML Algorithm Accuracy for Surface Roughness Prediction (Mean ± Standard Deviation).

ML Algorithm	$\mathbf{M}\mathbf{A}\mathbf{E}$	MAPE	RMSE	${\mathbf{R}}^2$
Linear Regression	0.020 ± 0.002	26.5% ± 3.4%	0.027 ± 0.003	86.0% ± 2.8%
XGB Regressor	0.020 ± 0.003	24.6% ± 4.8%	0.027 ± 0.004	85.0% ± 5.3%
Random Forest Regressor	0.011 ± 0.002	15.6% ± 4.0%	0.017 ± 0.003	94.2% ± 2.4%
Average Ensemble	0.016 ± 0.001	23.0% ± 3.8%	0.018 ± 0.002	93.1% ± 2.1%
AdaBoost Regressor	0.059 ± 0.003	56.2% ± 4.6%	0.067 ± 0.004	9.7% ± 18.9%
SVR	0.022 ± 0.005	29.4% ± 8.0%	0.027 ± 0.006	84.5% ± 8.5%
MLP	0.018 ± 0.002	24.7% ± 4.1%	0.023 ± 0.003	89.4% ± 2.0%

. As ML algorithms are stochastic in nature, each model was tested over 30 independent runs. To ensure the robustness and reliability of the evaluation, descriptive statistics including the mean and standard deviation of the performance metrics (MAE, MAPE, RMSE, and R²), were recorded. Based on the results of evaluation, the Random Forest Regressor achieved the best performance on the test dataset with the highest R² value (94.2% ± 2.4%), lowest MAE value (0.011 ± 0.002), lowest MAPE value (15.6% ± 4.0%), and lowest RMSE value (0.017 ± 0.003).

Additionally, a statistical significance analysis was performed to compare the Random Forest Regressor with other models (as illustrated in Figure 2). The results of this analysis, including the Z-scores and p-values, are illustrated in

Table 4. Statistical Significance Analysis Comparing the Random Forest Regressor with Other Models for Surface Roughness Prediction.

Comparison	Z-Score	p-Value	Significant (p < 0.05)
Random Forest vs. Linear Regression	12.0094	2.28606 × 10−17	True
Random Forest vs. XGB Regressor	8.25095	2.32836 × 10−11	True
Random Forest vs. Average Ensemble	8.20355	2.7945 × 10−11	True
Random Forest vs. AdaBoost	1.95375	0.055559	False
Random Forest vs. SVR	23.9245	9.76039 × 10−32	True
Random Forest vs. MLP	5.9513	1.63957 × 10−7	True

. The analysis showed that the Random Forest Regressor significantly outperformed other models, as indicated by low p-values (<0.05) for comparisons with Linear Regression, the XGB Regressor, Average Ensemble, SVR, and MLP (as illustrated in Table 4). The AdaBoost Regressor, however, did not show statistically significant improvement when compared to the Random Forest Regressor.

Moreover, the Q–Q (quantile–quantile) plot was also used to compare any deviation between the actual and predicted values in the various ML algorithms. A Q–Q plot is a graph depicting the quantiles of actual values set against the corresponding quantiles of predicted values (as illustrated in Figure 4). A perfect prediction would be represented by all the points on the plot following a linear pattern; any differences in the actual and predicted distributions are represented by non-linear relationships [43].

Based on the comparative analysis of the statistical metrics and the Q–Q plot results, the Random Forest Regressor was identified as the best ML algorithm for surface roughness prediction. In contrast, the AdaBoost Regressor showed the worst performance and proved insufficient in the prediction process.

Figure 3 and Figure 4 illustrate the comparative performance of all seven algorithms based on the test dataset. Undoubtedly, as shown by the visual representations, the Random Forest Regressor achieved better prediction results than other algorithms.

Similarly, for power consumption prediction (as illustrated in

Table 5. ML Algorithm Accuracy for Power Consumption Prediction (Mean ± Standard Deviation).

ML Algorithm	$\mathbf{M}\mathbf{A}\mathbf{E}$	MAPE	RMSE	${\mathbf{R}}^2$
Linear Regression	64.076 ± 3.503	11.5% ± 1.2%	87.650 ± 4.477	95.2% ± 0.4%
XGB Regressor	22.513 ± 4.424	3.0% ± 0.5%	42.650 ± 8.933	98.9% ± 0.5%
Random Forest Regressor	34.959 ± 4.336	4.7% ± 0.6%	52.650 ± 7.886	98.3% ± 0.5%
Average Ensemble	49.369 ± 3.001	7.8% ± 0.5%	58.650 ± 3.119	97.9% ± 0.3%
AdaBoost Regressor	29.339 ± 4.861	3.8% ± 0.8%	57.650 ± 11.564	97.9% ± 0.7%
SVR	75.970 ± 12.434	6.3% ± 1.0%	186.650 ± 21.157	78.5% ± 3.5%
MLP	34.231 ± 4.613	3.8% ± 0.5%	58.650 ± 8.667	97.8% ± 0.5%

), the best results were obtained for the XGB Regressor in the test dataset, with the highest R² value (98.9% ± 0.5%), lowest MAE value (22.513 ± 4.424), lowest MAPE value (3.0% ± 0.5%), and lowest RMSE value (42.650 ± 8.933).

The statistical significance analysis (as illustrated in Figure 5) performed to compare the XGB Regressor with other models showed that the XGB Regressor significantly outperformed other models, as indicated by low p-values (<0.05) for comparisons with Linear Regression, the Random Forest Regressor, Average Ensemble, the AdaBoost Regressor, SVR, and MLP (as illustrated in

Table 6. Statistical Significance Analysis Comparing XGB Regressor with Other Models for Power Consumption Prediction.

Comparison	Z-Score	p-Value	Significant (p < 0.05)
XGB Regressor vs. Linear Regression	29.9667	5.46433 × 10−37	True
XGB Regressor vs. Random Forest	4.6573	1.914 × 10−5	True
XGB Regressor vs. Average Ensemble	8.20355	2.7945 × 10−11	True
XGB Regressor vs. AdaBoost	7.78024	1.09453 × 10−13	True
XGB Regressor vs. SVR	5.63292	5.44446 × 10−7	True
XGB Regressor vs. MLP	31.2461	5.55743 × 10−38	True

).

Moreover, the XGB Regressor also showed the best Q–Q plot results among all seven algorithms (as illustrated in Figure 7). Although the Random Forest Regressor had a very close R² value (98.3% ± 0.5%), it slightly lagged behind the XGB Regressor. In contrast, the SVR algorithm showed the worst performance and proved insufficient for the prediction process.

Figure 6 and Figure 7 illustrate the comparative performance of all seven algorithms based on the test dataset. Undoubtedly, as shown by the visual representation, the XGB Regressor achieved better prediction results than other algorithms.

The results indicate that the Random Forest Regressor is the best surface roughness prediction model while the XGB Regressor is the best power consumption prediction model. Implementing these algorithms into the predictive models will assist in effective predictive maintenance by reducing energy usage and improving surface finish to achieve more sustainable and effective production. This way, the data-driven predictive models act as a digital twin, enabling better decision-making by enhancing the overall quality, productivity, and sustainability of the existing production system.

4. Discussion

This paper proposes a digital twin framework for smart manufacturing systems to compare different ML analysis techniques for predictive maintenance. The evaluation results show the effectiveness of the proposed framework in predicting surface roughness and power consumption during CNC turning operations. The framework offers a thorough approach for developing and comparing machine learning-based digital twin models for predictive maintenance in manufacturing systems, allowing for prediction and actionable feedback to operators for preventive actions. This enables predictive maintenance of the turning process by minimizing energy usage and assists in producing high-quality products with good surface finish, leading to more sustainable and cost-effective production. The findings indicate that among the seven ML models tested, the Random Forest Regressor achieved the best performance for surface roughness prediction, with the highest R² (94.2% ± 2.4%), lowest MAE (0.011 ± 0.002), lowest MAPE (15.6% ± 4.0%), and lowest RMSE (0.017 ± 0.003), while the XGB Regressor achieved the best performance for power consumption prediction, with the highest R² (98.9% ± 0.5%), lowest MAE (22.513 ± 4.424), lowest MAPE (3.0% ± 0.5%), and lowest RMSE (42.650 ± 8.933).

It is important to acknowledge that the current study is based on static experimental data and uses standard regression-based machine learning models, which do not capture temporal dependencies inherent in dynamic manufacturing environments. Future work should address this limitation by incorporating real-time data acquisition systems and time-series databases to develop more adaptive, temporally-aware predictive models. Additionally, future studies should consider applying the proposed framework to diverse, real-world industrial datasets, such as high-volume automotive parts machining, aerospace component manufacturing, or energy-efficient metal cutting applications, in order to further validate the models in different production settings.