Predicting Steam Turbine Power Generation: A Comparison of Long Short-Term Memory and Willans Line Model

Abstract

This study focuses on using machine learning techniques to accurately predict the generated power in a two-stage back-pressure steam turbine used in the paper production industry. In order to accurately predict power production by a steam turbine, it is crucial to consider the time dependence of the input data. For this purpose, the long-short-term memory (LSTM) approach is employed. Correlation analysis is performed to select parameters with a correlation coefficient greater than 0.8. Initially, nine inputs are considered, and the study showcases the superior performance of the LSTM method, with an accuracy rate of 0.47. Further refinement is conducted by reducing the inputs to four based on correlation analysis, resulting in an improved accuracy rate of 0.39. The comparison between the LSTM method and the Willans line model evaluates the efficacy of the former in predicting production power. The root mean square error (RMSE) evaluation parameter is used to assess the accuracy of the prediction algorithm used for the generator’s production power. By highlighting the importance of selecting appropriate machine learning techniques, high-quality input data, and utilising correlation analysis for input refinement, this work demonstrates a valuable approach to accurately estimating and predicting power production in the energy industry.

1. Introduction

Advancements in artificial intelligence (AI) techniques offer a new approach for predicting power generation in steam turbines, bypassing the need for traditional mathematical and thermodynamic models. The techniques presented in the field of learning from data can be classified into two categories: supervised learning and unsupervised learning. In supervised learning techniques, the model is trained using labelled data. Usually, a classification process is utilised for this purpose. On the other hand, for unsupervised learning, a model is trained through an existing technique such as clustering on the non-labelled data. In addition, the type of data examined is very influential in choosing a learning method. Some types of data do not have time continuity, and it is possible to shuffle the data for the test, train, and validation operations. However, in some cases, there is a temporal connection between each data line and the next line due to the time gap and the data being sequential. As such, if the data lines are moved, meaningful relational information is lost. In this case, especially where there is a time dependency between records, it is recommended to use special techniques such as recurrent neural networks (RNN), convolutional neural networks (CNN), and long short-term memory (LSTM) [1]. In this research, an attempt is made to forecast the power production of a two-stage back-pressure steam turbine generator using various input parameters, utilising the LSTM method. The organisation of this study can be outlined as follows:

• In Section 2, previous research studies that have utilised machine learning methods to predict industry-related parameters are examined and evaluated.
• Section 3 describes the framework and methodology employed in this study.
• Section 4 explains the Willans line model, which is used to estimate generator power by calculating enthalpy, entropy, and work transfer based on thermodynamic principles. The model relies on these calculations to predict power generation.
• In Section 5, a thorough assessment of the outcomes is conducted, including a comparison of the projections with the results obtained through the Willans line model.
• In Section 6, the findings are appraised, emphasising the significance of using machine learning models, particularly LSTM, within the context of various industries.

2. LSTM and Energy Systems Background

The previous work done in this research field can be examined from two perspectives. Firstly, the development of machine learning methods using the input data to create a model, and secondly, the use of that model for achieving the required goal. This section provides a summary of various research studies carried out in the past that have achieved different goals in the field of energy systems. These goals can be broadly categorised into system power output prediction, error detection, performance degradation and modelling, and environmental effects. The studies were conducted on different types of energy systems, such as photovoltaic solar panels, wind turbines, and gas and oil energy systems, with the majority focusing on the latter. Notably, most past research studies have employed the long-short-term memory (LSTM) method to predict the production capacity of energy systems. Some studies have also combined the LSTM method with other techniques, such as the convolutional neural network (CNN) method [2]. The effectiveness of these methods depends on the quality of the input data used; for instance, the time interval of the input data can significantly impact the performance of the system.

To provide further insights into the different techniques used in these studies, a selection of relevant articles has been chosen for comparison and in-depth examination. Table 1 presents a summary of these studies, including the year of publication, the specific methods employed, the type of case study, and the primary aims, such as power and electricity prediction, fault detection, and environmental impact. As depicted in the table, the energy systems examined can be classified into two overarching categories: oil and gas energy production systems, which include the use of steam turbines within this category, and the second category comprising renewable energy systems. Notably, the research conducted within the first group exhibits a well-balanced distribution and primarily focuses on predicting the lifespan errors and degradation patterns and ultimately forecasting the output power of these system types. On the other hand, with respect to renewable energy systems, as indicated in Table 1, given the inherent instability of such resources, the primary emphasis has been on accurately forecasting the production capacity. Throughout these research endeavors, the predominant methods employed have been LSTM, followed by the combination of LSTM with various other techniques, and ultimately, the utilization of the multi-layer perceptron (MLP) [3] method.

However, one persistent challenge that remains in this field is the intelligent selection of appropriate input variables and suitable architectural configurations to achieve predictions with minimal errors. This necessitates an ongoing effort to refine the choice of input features and the architectural design in order to enhance the accuracy and reliability of predictions in the realm of energy system analysis.

Previous research has utilised a range of software platforms, including MATLAB software in some cases [4], WEKA software in a few cases [5], and predominantly the Python platform [2]. Furthermore, in the context of the research described in this paper, the Python platform was employed for implementing both the LSTM and Willans line models [6].

Table 1. Summary of most recent learning models focusing on LSTM in energy systems.

Reference	Year	Method	Type of Case Study			Aim
			Oil/Gas Systems	Wind Energy Systems	Solar Energy Systems	Power Prediction	Fault Detection	Performance Modelling	Environmental Impact
Nikpey et al. [7]	2013	Multi-Layer Perceptron (MLP)	√					√
Wu et al. [8]	2016	LSTM		√		√
Roni et al. [4]	2017	MLP	√						√
Talaat et al. [9]	2018	MLP	√					√
Zhang et al. [10]	2018	LSTM	√				√
Cao et al. [11]	2018	LSTM		√		√
Zhang et al. [12]	2019	Gaussian Mixture Model (GMM) + LSTM		√		√
Jung et al. [13]	2019	LSTM			√	√
Gao et al. [14]	2019	LSTM			√	√
Adams et al. [15]	2020	SVM	√						√
Arriagada et al. [16]	2020	MLP	√				√
Yongsheng et al. [17]	2020	Extreme Learning Machine (ELM) + LSTM			√	√
Wang et al. [18]	2020	Time correlation modification + LSTM			√	√
Mei et al. [19]	2020	LSTM			√	√
Yuvaraju et al. [20]	2020	LSTM	√			√
Cao et al. [21]	2021	LSTM	√					√
Bai et al. [22]	2021	LSTM	√				√
Alsabban et al. [23]	2021	LSTM	√			√
Dong et al. [24]	2021	LSTM+ Gaussian mixture mode (GMM)		√		√
Zhang et al. [25]	2021	CNN + LSTM		√		√
Zhou et al. [26]	2021	Lower and upper bound estimation (LUBE) + LSTM		√		√
Kong et al. [27]	2022	LSTM	√			√
Ijaz et al. [28]	2022	LSTM	√			√
Zhang et al. [29]	2023	LSTM		√		√
Kabengele et al. [30]	2023	MLP	√					√

Table 1. Summary of most recent learning models focusing on LSTM in energy systems.

Reference	Year	Method	Type of Case Study			Aim
			Oil/Gas Systems	Wind Energy Systems	Solar Energy Systems	Power Prediction	Fault Detection	Performance Modelling	Environmental Impact
Nikpey et al. [7]	2013	Multi-Layer Perceptron (MLP)	√					√
Wu et al. [8]	2016	LSTM		√		√
Roni et al. [4]	2017	MLP	√						√
Talaat et al. [9]	2018	MLP	√					√
Zhang et al. [10]	2018	LSTM	√				√
Cao et al. [11]	2018	LSTM		√		√
Zhang et al. [12]	2019	Gaussian Mixture Model (GMM) + LSTM		√		√
Jung et al. [13]	2019	LSTM			√	√
Gao et al. [14]	2019	LSTM			√	√
Adams et al. [15]	2020	SVM	√						√
Arriagada et al. [16]	2020	MLP	√				√
Yongsheng et al. [17]	2020	Extreme Learning Machine (ELM) + LSTM			√	√
Wang et al. [18]	2020	Time correlation modification + LSTM			√	√
Mei et al. [19]	2020	LSTM			√	√
Yuvaraju et al. [20]	2020	LSTM	√			√
Cao et al. [21]	2021	LSTM	√					√
Bai et al. [22]	2021	LSTM	√				√
Alsabban et al. [23]	2021	LSTM	√			√
Dong et al. [24]	2021	LSTM+ Gaussian mixture mode (GMM)		√		√
Zhang et al. [25]	2021	CNN + LSTM		√		√
Zhou et al. [26]	2021	Lower and upper bound estimation (LUBE) + LSTM		√		√
Kong et al. [27]	2022	LSTM	√			√
Ijaz et al. [28]	2022	LSTM	√			√
Zhang et al. [29]	2023	LSTM		√		√
Kabengele et al. [30]	2023	MLP	√					√

This present study focuses on the prediction of the power output of a two-stage back-pressure steam turbine generator used in the paper production industry. To this end, a simple and computationally efficient LSTM method has been utilised, contrasting with the hybrid deep learning method reported in [4]. The use of machine learning algorithms is particularly relevant in this study, as thermodynamic models used to estimate production power are often time-consuming and require numerous constants for calculation. The predicted results are compared with those obtained from the thermodynamic model, providing an evaluation of the LSTM method’s performance and potential advantages.

3. Resources and Techniques

This section delves into three key aspects: the framing of the problem in this case study, the exploration of data necessary for developing a solution, and the presentation of the technique utilised for predicting power production.

3.1. Description of an Industrial Process

For this research, a two-stage steam turbine at a Kraft Pulp and Paper Mill has been used as a case study (see Figure 1). The power production of this steam turbine is approximately 40 MW. An hourly time-series dataset covering 22 months has been obtained from the industry in order to estimate the output power with high accuracy, low computational complexity, and low execution overhead compared to the Willans line model. Figure 1 shows the inputs and outputs of the model.

Data have been recorded at 10-min intervals, but there are gaps, with some samples missing. To address this issue, linear interpolation was utilised to fill these gaps. To improve the accuracy of predicting generator active power, the system employs an LSTM model.

3.2. Materials and Methods

Hochreiter and Schmidhuber introduced the LSTM neural networks in 1997 [1], and the concept has since undergone numerous improvements by various individuals. Although LSTM networks are a relatively old technique, they are used in a wide range of applications and are still popular, as shown in Figure 2, which illustrates the utilisation of artificial intelligence techniques in a multitude of research studies conducted in the period spanning from 2016 to 2022. It is evident that in this timeframe, there has been a notable prevalence of LSTM as the primary technique employed, closely followed by the gated recurrent unit (GRU) [31] technique during the initial stages. These two techniques have been employed more frequently in comparison with other methodologies, as shown by the graph.

LSTM neural networks are categorised as a specific type of recursive neural network (RNN), designed to effectively capture and model long-term temporal dependencies within the output data. By incorporating memory cells and gating mechanisms, LSTMs can retain and selectively update information over extended sequences, enabling them to effectively handle and exploit the intricacies of time-dependent patterns present in the data. This distinguishing feature sets LSTMs apart from traditional RNNs, allowing them to excel in tasks such as time series analysis, where capturing long-range dependencies is crucial for accurate predictions and insightful analysis.

An indication of the range of applications where LSTM methods have been utilised is shown in Figure 3. Primarily, this approach has been prominent for time series data analysis, where the data exhibit temporal interdependencies. Further, LSTM models have also found application in the domains of language modelling and sentiment analysis.

The research described here considers different inputs as shown in Figure 4, with an architecture comprising 50 neurons in LSTM layers having an activation “Relu” [33] function. The “ADAM” optimizer [34] has been included in the model to minimise errors. The model was trained using 40% of the available data, and the remaining 60% were utilised to test the model.

The selection of evaluation parameters is also an important issue in the application of machine learning. When the purpose of machine learning is to predict the value of a specific variable, the evaluation parameters that are usually used are mean absolute percentage error (MAPE), root mean square error (RMSE), mean percentage error (MPE), and absolute percentage error (APE) [35]. The MPE measures the average difference between predicted and true values over N samples using Equation (1) below. The RMSE measures the square root of the average of the squared differences between predicted and true values over N samples using Equation (2).

The APE of each sample is calculated using Equation (3), which measures the absolute percentage difference between predicted and true values. The MAPE is then calculated by averaging the APEs over N samples using Equation (4). By selecting appropriate evaluation parameters, machine learning models can be optimised to improve the accuracy of their predictions.

$$\mathrm{M}\mathrm{P}\mathrm{E}=\frac{1}{\mathrm{N}}{\sum }_{\mathrm{i}=1}^{\mathrm{N}}({\hat{\mathrm{y}}}_{\mathrm{i}}-{\mathrm{y}}_{\mathrm{i}})$$

(1)

$$\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}=\sqrt{\frac{1}{\mathrm{N}}{\sum }_{\mathrm{i}=1}^{\mathrm{N}}{({\hat{\mathrm{y}}}_{\mathrm{i}}-{\mathrm{y}}_{\mathrm{i}})}^2}$$

(2)

$$\mathrm{A}\mathrm{P}{\mathrm{E}}_{\mathrm{i}}=\left|\frac{({\hat{\mathrm{y}}}_{\mathrm{i}}-{\mathrm{y}}_{\mathrm{i}})}{{\mathrm{y}}_{\mathrm{i}}}\right|\times 100\%$$

(3)

$$\mathrm{M}\mathrm{A}\mathrm{P}\mathrm{E}=\frac{1}{\mathrm{N}}{\sum }_{\mathrm{i}=1}^{\mathrm{N}}\left(\mathrm{A}\mathrm{P}{\mathrm{E}}_{\mathrm{i}}\right)$$

(4)

3.3. Correlation Analysis

One of the key aspects of correlation analysis is exploring the interdependence between different parameters and their influence on the output. Correlation analysis is a crucial tool in several fields, including economics, as it can offer significant insights into the connection between variables. This analysis involves calculating the correlation variable using a specific relationship between the parameters, as shown in Equation (5), resulting in a value between zero and a positive one. When the value is closer to one, this implies that the target variable is significantly affected by other variables under consideration, and vice versa. A zero coefficient denotes an absence of correlation between the variables.

Considering the mathematical relationship shown in Equation (5), a thorough evaluation and calculation of the correlations was conducted across all parameters. This assessment aimed to discern the extent of interdependence and mutual influence between the parameters. In Figure 5, the correlation between each pair of parameters is displayed using distinct colors. To provide a better understanding of the intricate relationships among various parameters, Figure 6 depicts the magnitude of influence that each parameter exerts on the others if the correlation factor between them is greater than 0.8. Upon careful examination of this diagram, it becomes apparent that when the correlation parameter surpasses a value of 0.8, a linear equation is deemed suitable for capturing the underlying relationship. This line of best fit is employed to describe the connection between the two relevant parameters, and the corresponding equation is then presented for reference and analysis.

$$corr(X,Y)=\frac{\frac{1}{N}\sum _{i=1}^N(X_i-\overline{X})(Y_i-\overline{Y})}{{\sigma }_X\times {\sigma }_Y}$$

(5)

Furthermore, in the scope of this research, the focal point revolves around the generator’s production power, which is considered the ultimate output. Considering this, a careful selection process was carried out to identify pertinent inputs for the second model under evaluation. Specifically, as shown originally in Figure 6, four parameters—steam flow to turbine, pass-out steam pressure, turbine pass-out steam flow compressor, and turbine exhaust steam flow compressor—were found to exhibit a correlation parameter exceeding 0.8 when compared with the output parameter. Consequently, these four parameters were designated as inputs for the second model in this study, as their influential impact on the output power warranted further investigation and analysis.

4. Willans Line Model and Equations for Power Generation Estimation

The Willans line, a concept in steam turbine engineering, serves as a guiding principle in the evaluation of the two-stage back-pressure steam turbine. Developed by Robert Willans, this graphical representation provides essential insights into the efficiency and performance of the turbine across varying operating conditions [36]. The line establishes an idealised correlation between power output and steam consumption for an isentropic turbine. By comparing the actual power generated with mass flow to this theoretical line, a clear understanding of the turbine’s real-world efficiency is gained [36]. The Willans line also aids in identifying the most efficient operating range, where the turbine achieves optimal power output while conserving steam. It is a critical tool for ensuring our steam turbine operates at its highest efficiency, minimising resource consumption, and maximising power generation.

The thermodynamic model employed in this study utilises the Willans line to estimate the generated power of the generator. The calculations are performed using the “CoolProp” library [37], which provides functions to calculate properties such as enthalpy and entropy of the working fluid. The model considers multiple stages of the turbine, including the high-pressure stage (HPS) and the low-pressure stage (LPS).

In the implemented approach, the “PropsSI” function is used to calculate the enthalpy and entropy of the working fluid based on the given temperature and pressure values. These calculations are performed for the HPS stage, and the resulting values are stored. Similarly, the enthalpy of the medium-pressure stage is derived from the HPS entropy using the entropy–enthalpy relationship, and the entropy of the MPS stage is calculated directly. The work transfer between stages is then computed by multiplying the mass flow rates with the differences in enthalpy. This results in the calculation of the work done in the HPS stage and the LPS stage. The total generated power is obtained by summing the individual work transfers.

To optimise the regression of the raw data, the “GEKKO” library [38] is utilized. It provides capabilities for regression analysis and optimization. The implemented model performs regression analysis using the “GEKKO” library to find the best-fit parameters for the power-mass flow relationship. The resulting parameters are used to plot the regression lines.

Figure 7 shows the different turbine stages, specifically comparing the modelled Willans line with the raw data. These plots show the performance of the turbine under varying operating conditions, as well as how closely the model matches the raw data. For both the higher-pressure stage (a) and the lower-pressure stage (b), there is a close correlation at higher mass flows, but this begins to deviate away at lower pressures. While this type of Willans line can be useful for estimating the work generation under various loads, it is difficult to validate the outputs of these individual stages, as often multi-stage turbines operate with single, common generators, meaning that individual power values for stages are not available but rather only the overall power.

5. Results

This section presents the results obtained from the implementation of the model, providing a comprehensive understanding of its performance. Figure 8 depicts the outcomes, revealing that a significant proportion of the predicted points generated by the model, whether from the testing data or the training data, coincide with the actual data points. This alignment suggests a minimal level of error between the model’s predictions and the real data, thus indicating the efficacy of the model.

Prior to arriving at this stage, an extensive investigation was conducted using sensitivity analysis to develop two models, each comprising four inputs. To facilitate comparative analysis, an additional model was constructed based on Willans line equations.

Figure 8 provides insight into the construction process of these models, shedding light on the various steps involved. Remarkably, the measured real power points, symbolised by blue dots, exhibit a striking alignment with both the green points of the training data and the red points representing the testing data. This alignment serves as a demonstration of the performance of the initial model, which consists of nine inputs. Supporting this observation,

Table 2. Root Mean Square Error (RMSE) factor for output of the model for testing and training data.

DATA	RMSE	Data Set Size
Training	0.36	37,817
Testing	0.47	56,725

presents the evaluation parameter values for both the training and testing data, with the training data showing a lower RMSE value, aligning with the anticipated outcomes. The fact that these numbers are less than 0.5 shows the better performance of this model than other research with the same purpose [21,30,39,40]. Upon comparing the two outcomes, it was found that the accuracy of the model is better for the training data than the testing data. The reason for this is that the model was trained using this training data and will naturally perform better with this data.

The following section provides a comparative analysis of the results obtained from the three models, which are presented in Figure 9. This graph illustrates the predicted power in relation to the actual power. An ideal model would exhibit a linear relationship with a slope of one and a width of zero, signifying a perfect correspondence between the predicted and actual power values. Considering Figure 9, it becomes evident that both the four-input and nine-input models surpass the performance of the Willans line model. Furthermore, a detailed examination comparing the four-input and nine-input models reveals that the former displays a narrower dispersion, indicative of its superior performance and precision. As illustrated in Figure 9, the line produced by the Willans line model shows a marginal variance from the lines generated by the deep learning model. This discrepancy arises from the fact that approximately 80% of the input data exhibits a power output surpassing 22 MW, exerting a substantial impact on the accuracy of the model derived from the Willans line. Hence, the resultant line is represented as a dashed line in regions where the power output is less than 22 MW.

Further insights into the predicted power levels, taking into consideration the dates of data collection, are provided in Figure 10. Notably, the predicted powers depicted in the three charts demonstrate a high degree of consistency, with minimal disparities observed among them. To augment these findings, Figure 11 portrays the individual error magnitudes for each model, serving as additional evidence to reinforce the earlier explanations. This diagram further substantiates the notion that the four-input model outperforms both the nine-input model and the Willans line model, firmly establishing its favourable performance ranking.

6. Conclusions and Perspectives

The study described here demonstrates that the use of long short-term memory (LSTM), a machine learning algorithm, shows superior performance in predicting the power production of steam turbines than the conventional thermodynamic models. The results obtained from the comparison of LSTM and Willans line models, evaluated using the root mean square error (RMSE) metric, highlight the superior forecasting accuracy of the LSTM approach. Initially employing 9 inputs, the LSTM method achieved an RMSE of 0.47; however, by leveraging correlation analysis and refining the input set to 4 inputs, an even lower RMSE of 0.39 was attained. This study provides evidence for the efficacy of using machine learning algorithms and artificial intelligence to accurately predict desired system performance in various industries.

Moreover, the study highlights the importance of choosing the appropriate machine learning technique based on the type of data examined. In cases where there is time dependence between data lines, techniques such as LSTM that consider the time dependence of input data should be utilised for accurate predictions. The study also highlights the significance of the quality of input data used in machine learning models, as the time interval of the input data can significantly impact the performance of the system.

Overall, the study contributes to the growing body of literature on the use of machine learning algorithms in the energy production industry. It presents a case study on the prediction of production power generated by a two-stage back-pressure steam turbine used in the paper production industry. The results obtained through the LSTM method demonstrate the superior accuracy of this approach compared to previous algorithms used in the field. The study provides insight into the potential of machine learning algorithms to improve the accuracy and efficiency of system performance predictions in various industries.