Switched Auto-Regressive Neural Control (S-ANC) for Energy Management of Hybrid Microgrids

Abstract

Switched model predictive control (S-MPC) and recurrent neural networks with long short-term memory (RNN-LSTM) are powerful control methods that have been extensively studied for the energy management of microgrids (MGs). These methods ease constraint satisfaction, computational demands, adaptability, and comprehensibility, but typically one method is chosen over the other. The S-MPC method dynamically selects optimal models and control strategies based on the system’s operating mode and performance objectives. On the other hand, integration of auto-regressive (AR) control with these powerful control methods improves the prediction accuracy and the adaptability of the system conditions. This paper compares the two control approaches and proposes a novel algorithm called switched auto-regressive neural control (S-ANC) that combines their respective strengths. Using a control formulation equivalent to S-MPC and the same controller model for learning, the results indicate that pure RNN-LSTM cannot provide constraint satisfaction. The novel S-ANC algorithm can satisfy constraints and deliver comparable performance to MPC, while enabling continuous learning. The results indicate that S-MPC optimization increases power flows within the MG, resulting in efficient utilization of energy resources. By merging the AR and LSTM, the model’s computational time decreased by nearly 47.2%. In addition, this study evaluated our predictive model’s accuracy: (i) the R-squared error was 0.951, indicating a strong predictive ability, and (ii) mean absolute error (MAE) and mean square error (MSE) values of 0.571 indicate accurate predictions, with minimal deviations from the actual values.

1. Introduction

Model predictive control (MPC) is a control approach that is widely utilized in many industries, including chemical, electrical, and mechanical engineering. It is well-suited to microgrids (MGs) because it deals with restrictions and optimizes performance over time [1,2,3,4,5]. MPC entails formulating and solving an optimization problem at each time step, to determine the optimal control inputs for the next step. A MPC was described in [6,7,8,9] for effective MG optimization, and mixed integer linear programming (MILP) was employed to solve the problem posed. An MPC-inspired energy management (EM) system employing a neuro-fuzzy method, which accounted for renewable energy sources (RES’s) intermittent nature in grid-connected MG with loads and photovoltaic (PV) sources, was reported in [10]. Ref. [11] presented scenario-based stochastic programming with a rolling horizon strategy for minimizing the operating expenses of MGs when the wind speed is unknown. Rolling horizon or MPC techniques are reactive-based methodologies that modify or update data using deterministic approaches. A scenario-based MPC was developed in [12] to reduce operating expenses and overall emissions. To achieve inexpensive and flexible operation, Ref. [13] provided an MPC-based optimum management for renewable energy MGs with hybrid energy storage systems (ESSs), such as hydrogen, batteries, and capacitors. A hierarchical MPC-based technique for islanded AC MGs addressed power-quality and unbalanced power-sharing difficulties [14]. Despite this, the traditional MPC cannot control a MG in various operational modes.

In contrast, switched model predictive control (S-MPC) is a variant of MPC that employs multiple models, each representing a unique mode of operation or scenario of the system. S-MPC selects the optimal model and associated control strategy based on the current system state and the desired performance goals. This makes it possible for S-MPC to handle systems with mode-dependent dynamics. MPC is distinguished from S-MPC in using a single model to predict the system’s future behavior [15]. S-MPC employs multiple models and switches between them, based on the system’s current state. S-MPC can provide better performance and robustness than MPC, especially for complex systems with multiple modes or operating conditions [16,17]. Another novel study presented a hybrid MG model that incorporated two switched receding horizon control laws. This strategy reduces the overall energy expenses and maximizes the efficient utilization of RESs for expansive business establishments, while accommodating fluctuations in grid connectivity [18]. Moreover, Ref. [19] outlined the process of designing and applying a S-MPC to wind turbine systems, intending to manage the intricate nature and nonlinearity inherent in wind turbine systems. The system employs qpOASES as an integrated solver for optimum online control. It incorporates a cyber-physical real-time emulator for utility-scale wind turbines, with variable-speed and variable-pitch capabilities. The study showcased the viability and efficacy of S-MPC in attaining control objectives for wind turbine systems in real-time, utilizing brief control periods. In addition, the study in [20] presented a novel technique for enhancing wind turbine control by introducing a S-MPC framework. The proposed approach aimed to solve the limitations of the conventional continuous control-based MPC algorithm. The results of the comparative analysis indicated that the proposed algorithm exhibited superior performance compared to the existing MPC in various aspects, including computational efficiency, load mitigation, and dynamic response. Ref. [21] presented a novel S-MPC method specifically tailored to discrete-time nonlinear systems. The simulation outcomes emphasized its superiority over a conventional MPC technique regarding computational efficacy and control effectiveness. Another study presented a novel S-MPC methodology for power converters. During transient periods, the system utilized horizon-one nonlinear finite control set MPC to steer the system towards the intended reference [22].

On the other hand, S-MPC performance is highly vulnerable to model mismatch. In other words, it must select a suitable system model. Furthermore, the increased complexity of S-MPC impacts the stability and maintainability of MG control [23,24]. These challenges lead to accuracy issues with S-MPC methods. In addition, the computational time of S-MPC is much longer, because of the prediction horizon and various steps. Many authors have studied machine learning (ML) techniques to increase the accuracy of MG systems.

When improving the scheduling effectiveness in networked microgrids (NMGs), the main goal is minimizing the effects of electricity outages.This paper presents a framework consisting of three stages, to evaluate power transactions, manage renewable energy and market price risks, and tackle uncertainties. This framework is formulated as a mixed-integer linear programming problem [25,26,27,28]. On the other hand, ref. [29] introduced a novel approach utilizing the Internet of things (IoT) to dynamically optimize and regulate power loads in citizen energy communities. This technique is compared to the conventional direct load control (DLC) method. This technique aims to enhance power use efficiency using programmable appliances and dynamic demand response. In order to model the behavior of RESs, such as wind and solar, auto-regressive moving-average (ARMA)-based scenario generation was implemented. Large industries could receive direct assistance from storage and demand-side management systems, to reduce energy costs [30]. The other work employed an ARMA model to forecast solar PV, wind power generation, and electricity demand. Second, an optimal generation scheduling procedure is intended to reduce system operating expenses. The simulation results indicated that the optimal generation scheduling could minimize operating expenses under the worst-case scenario [31]. In [32], combining two models, the ARMA and nonlinear auto regressive with exogenous input (NARX), a novel method was presented for predicting solar radiation. The decision was made to utilize the benefits of both models, to produce more accurate prediction results. The simulation results validated this hybrid model’s ability to predict weekly solar radiation averages. Although the previous solar radiation forecasting techniques, particularly ARMA models, are effective for particular uses, they are unsuitable for other uses requiring a high forecasting precision. Several researchers have proposed hybrid models to improve the precision of solar radiation forecasting. Moreover, there is still not a proper plant model or prediction horizon, so the computational time of the model is still to high [32].

There have been numerous studies on using ML methods in place of AR models. For instance, ref. [33] thoroughly investigated the prediction performance of several recurrent neural network (RNNs) designs, such as a long short-term memory (LSTM), gated recurrent unit (GRU), and bidirectional LSTM. Using local weather forecasts and historical weather data, ref. [34] proposed a LSTM-based next-day forecasting model of hourly global horizontal irradiance (GHI). Refs. [35,36] suggested LSTM-based models using only the next day’s weather forecast as input. The studies by [37,38] used similar LSTM-based techniques. Refs. [39,40] validated the performance of hybrid deep learning models built on convolutional neural networks (CNNs) and LSTM for day-ahead GHI forecasting. In addition to RNN-based approaches, there have been studies evaluating the performance of other statistical and ML models for solar irradiance forecasting, such as coupled AR and a dynamic system by [41], a Markov switch model [42], and a support vector machine (SVM) by [43]. Ref. [44] reported an LSTM-based model for hour-ahead solar irradiance forecasting. The inputs, which included historical GHI and meteorological data from the preceding 24 h, were utilized to forecast the GHI for the next hour [44]. The results revealed that the LSTM-based model outperformed the other models, such as auto-regressive integrated moving average (ARIMA) and CNN [44]. Ref. [45] investigated the performance of LSTM and GRU. Refs. [46,47] published hybrid CNN-LSTM models for hour-ahead GHI forecasting. Their study showed that incorporating external weather information considerably increased the prediction accuracy. Unlike day-ahead irradiance forecasting methods, hour-ahead forecasting algorithms create projections for the following hour only using historical data.

On the other hand, RNNs are a form of ML technology widely employed for time series prediction and the modeling of dynamic systems [48,49]. RNNs are artificial neural networks (ANN) that are particularly useful for modeling time-series data and may be used to anticipate future MG behavior [50,51]. RNNs may learn and adapt to system dynamics by learning the temporal dependencies in data. RNNs have been used to solve various MG control challenges, including load forecasting, renewable energy integration, and demand response management [52,53,54]. RNNs have been applied to various systems, including power systems [55,56], with promising prediction accuracy and flexibility results.

In summary, both control families have benefits and drawbacks, and their complementarity is evident. On the one hand, S-MPC struggles with system complexity and long-term prediction horizons, whereas the combination of AR and LSTM (AR-LSTM) can deal with complex systems and infinite prediction horizons. AR-LSTM, conversely, has difficulty in satisfying constraints and lacks interpretability, whereas S-MPC can provide safety guarantees and understandability.

Although there is a clear potential for a synergy between the two families of methods, there have been few attempts to combine their relative advantages. This research deficiency is not limited to applying EM for MG. The control and ML communities evolve independently, adopting radically different notations to formulate the same problem. In spite of the parallel developments, several authors [57,58,59] have suggested that a collaboration between the two groups could result in potential advantages. Combining these methodologies represents a powerful method for integrating robust control theory methods with ML approaches, to exploit additional information from real-time data [60,61].

As shown in

Table 1. Comparison of Control Methods.

Control Method	Optimality	Computational Time [s]	Multiple Models	Adaptability	Constraints
MPC [62,63]	✓	>${10}^4$ [High]	×	Good	✓
S-MPC [15,16,20,64]	✓	≈${10}^4$ [Moderate]	✓	Outstanding	✓
DLC [29]	✓	≈${10}^4$ [Moderate]	×	Good	✓
AR [32]	×	≈${10}^4$ [Moderate]	×	Poor	×
CNN [65,66]	×	<${10}^4$ [Low]	×	Poor	×
RNN-LSTM [67,68]	✓	<${10}^4$ [Low]	×	Poor	×
S-ANC	✓	$<{10}^4$ [Low]	✓	Outstanding	✓

, each control method has its strengths and limitations. MPC and S-MPC offer robust optimality and constraint handling but may have computational challenges. AR and RNN-LSTM are efficient in computation but may not manage complex constraints effectively. S-ANC combines AR models with neural control, balancing optimality and computational efficiency. The choice of control method depends on the specific application and trade-offs between these criteria.

Contributions and Research Questions

This paper was motivated by how AR-LSTM and S-MPC could collaborate in applying the EM of MGs. While there is a consensus that combining the two algorithms may yield benefits, little has been done to develop methods that involve the two algorithms working together. In addition, works have investigated how these controllers can collaborate with the algorithms working with different control designs and modes. No previous research has compared and combined S-MPC and AR-LSTM for the same optimal control problem formulation in EM for MGs.

The second objective of this paper was to propose a novel method known as switched auto-regressive neural control (S-ANC), which merges S-MPC and AR-LSTM synergistically. The development and formulation of this new S-ANC algorithm were motivated by the conceptual and practical comparison of S-MPC and AR-LSTM. In contrast to comparable approaches, our method combines the S-MPC objective function and constraints with the AR-LSTM optimization and prediction function. This practice ensures interoperability between the two methods and enables the truncation of the S-MPC optimization problem, which can become highly complex, even for relatively simple MG structures. Finally, a flexible hybrid MG case is used to describe and evaluate this new algorithm.

Consequently, the primary contribution of this paper is the introduction of S-ANC, a control algorithm that combines techniques from the communities of control theory and ML. This algorithm is evaluated, and a new standard framework is generated for EM of hybrid MGs. In addition, the proposed S-ANC algorithm applies to various applications and domains, such as complex industrial processes and energy markets. This study also combines control theory and ML by comparing and disentangling the key distinctions between S-MPC and AR-LSTM.

2. Identifying the Distinctions between S-MPC and AR-RNN-LSTM

Optimal control determines the actions that optimize a performance objective by solving a sequential decision-making problem. The preceding section highlighted the need to compare S-MPC and AR-LSTM, the two primary approaches for optimal control applied to EM for hybrid MG control, and the possibility of combining them. Both methods utilize certain components, while others are more controller-specific. These formulation differences make it difficult to compare and combine the two approaches, necessitating a conceptual analysis. This assists in identifying the primary methods of optimal control and establishes a common ground for a comprehensive classification. The sections that follow detail the most important aspects of these control methods.

2.1. Strategy

There are typically two ways to approach an optimal control problem: by employing the S-MPC-inherent receding horizon principle or formalizing the problem as an AR-LSTM.

S-MPC is a control strategy that involves using a mathematical model of the system being controlled to predict the system’s future behavior and optimize a control signal over a finite time horizon. At each time step, the control signal is updated based on the current state of the system and the predictions made by the model. This is widely used in industrial control applications, such as process control, automotive control, and robotics, where it is important to consider the system dynamics being controlled and to optimize performance over a prediction horizon. At each time step k in S-MPC, switching logic controls multiple modes for the accumulators, which fully describe the controller model at the current time.Then, the trajectories of the future state x and input u are optimized for a prediction horizon $N_P$ based on the explicit representation of an objective function J and a controller model F. J is the minimization of the imported energy and maximization of the exported energy. The constraints H are also introduced explicitly in the optimization problem. The objective function, model, and constraints may also depend on the model outputs y and time-invariant parameters p. In addition, $r\left(k\right)$ is the reference variable representing the PV, load data, and zero along the prediction horizon $N_P$. $w_x\left(k\right)$ and $w_u\left(k\right)$ are weighting coefficients reflecting the relative significance of $x\left(k\right)$ and penalizing relatively large variations in $u\left(k\right)$, respectively. Only the initial control input from the optimized trajectory [16] is Implemented. Figure 1a depicts the full S-MPC procedure.

$$\begin{array}{c}J\left(k\right)=\sum _{k=0}^{N_P}{w}_x\left(k\right){(r\left(k\right)+x\left(k\right))}^2+\sum _{k=0}^{N_P}{w}_u\left(k\right)\Delta u{\left(k\right)}^2+\sum _{k=0}^{N_P}{w}_y\left(k\right)y{\left(k\right)}^2+p^2\end{array}$$

(1)

In the application of S-MPC to EM of MGs, the state vector x represents the state of charge of the accumulators ($SOAcc$), such as the battery, fuel tank, and water tank, and the model output y illustrates the imported and exported energy, such as a grid to the load $G{R}_{LD}$ and PV to the grid, and the battery ($P{V}_{GR}$ + $P{V}_{BAT}$). Depending on whether or not the controller model employs physical insights, the set of time-invariant parameters p may or may not represent the physical properties of the MG.

In contrast to RNN-LSTM, AR models are not neural network architectures. On the contrary, they are statistical models that identify dependencies and patterns within a time series, based on its own lagged values. The AR model predicts the future values of a variable based on its historical values and the estimated coefficients during model training. In other words, AR models are a statistical modeling technique that assumes a variable’s current value is a function of its previous values. They are frequently utilized for time series analysis and forecasting. Therefore, AR models can be viewed as a linear regression, in which the predictors are the values of the same variable at a prior time [32]. AR models can be used to model the system’s dynamics within the context of control systems or reinforcement learning. A model can predict future states or observations by estimating the AR coefficients. These predictions can then be fed into control algorithms or reinforcement learning agents, in order to optimize control signals or decision-making. Unlike neural network architectures, AR models are not adaptive by nature. The estimation of AR coefficients requires training on historical data, and their performance may degrade if the underlying dynamics of the system change significantly over time.

The following equation can mathematically represent an AR model of order q [32]:

$$\begin{array}{c}X\left(k\right)=c+\phi \left(1\right)X(k-1)+\phi \left(2\right)X(k-2)+\cdots +\phi \left(q\right)X(k-q)+\epsilon \left(k\right)\end{array}$$

(2)

where $X\left(k\right)$ represents the value of the time series at time k in this equation. c is a constant term or an intercept. $\phi$ terms represent AR model coefficients. The coefficients or weights associated with the previous values of the time series are denoted by 1, 2, …, q. $X(k-1)$, and $X(k-2)$, …, $X(k-q)$ represent the lagged values of the time series at time points $k-1$, $k-2$, …, $k-q$, respectively. $\epsilon \left(k\right)$ is the error term or random noise at time k, representing the data portion the model cannot explain.

RNN-LSTM is a neural network type ideally suited to processing sequential data. Unlike feed-forward neural networks, it has loops that allow information to be passed from one sequence step to the next. The approach for employing RNN-LSTM includes selecting an appropriate network architecture, an optimization algorithm for training the network, and an appropriate set of hyperparameters. RNN-LSTM is an extension of a feed-forward neural network with internal memory. RNN-LSTM is recurrent because it performs the same function for each data input, while the output of the current input is dependent on the previous computation. After the output has been generated, it is duplicated and sent back into the recurrent network [69]. For decision-making, it considers both the current input and the output from the previous input it learned. As shown in Figure 1b, the input vector of an LSTM network is $u(k-1)$ at time step k. $y\left(k\right)$ represents the output vectors passed through the network between time steps k and $k+1$. Three gates update and control the cell states in an LSTM network: the forget gate, input gate, and output gate. The gates are activated by hyperbolic tangent and sigmoid functions. Given new information that has entered the network, the forget gate determines which cell state information to forget. Given new input information, the input gate determines what new information will be encoded into the cell state. Using the output vector $y\left(k\right)$, the output gate controls what information encoded in the cell state is sent to the network as input in the subsequent time step.

In the mathematical modeling of RNN-LSTM, the current state can be expressed mathematically as

$$\begin{array}{c}x\left(k\right)=f\left(x\right(k-1),u(k\left)\right)\end{array}$$

(3)

where $x\left(k\right)$ represents the current state, $x(k-1)$ represents the previous state, and $u\left(k\right)$ is the current input. Because the input neuron would have applied the transformations to the previous input, we now have the state of the previous input rather than the input itself. Each successive input is, therefore, referred to as a time step.

Considering the simplest form of RNN-LSTM, where the activation function is tanx, the weight at the recurrent neuron is $W_{xx}$, and the weight at the input neuron is $W_{ux}$, we can write the equation for the state at time k as follows [69]:

$$\begin{array}{c}x\left(k\right)=tanx(W_{xx}x(k-1)+W_{ux}u\left(k\right))\end{array}$$

(4)

In this instance, the recurrent neuron only considers the previous state. The equation may involve multiple such states for longer sequences. After calculating the final state, the output can be generated. Once the current state has been computed, we can then calculate the output state as follows [69]:

$$\begin{array}{c}y\left(k\right)=W_{xy}x\left(k\right)\end{array}$$

(5)

where $y\left(k\right)$ is the output state and $W_{xy}$ is the weight at the output state. This process is represented in Figure 2.

First, it extracts $u\left(0\right)$ from the input sequence and then outputs $y\left(0\right)$, which, along with $u\left(1\right)$, is the input for the subsequent step. Therefore, $y\left(0\right)$ and $u\left(1\right)$ are the inputs for the subsequent step. Similarly, $y\left(1\right)$ from the subsequent step is the input for $u\left(2\right)$ for the subsequent step, and so on. Consequently, it remembers the context throughout training.

A cost function quantifies “how well” a neural network performs with respect to the training sample and the expected output. This may also depend on factors such as weights and biases. This is a single value, not a vector, because it evaluates the overall performance of the neural network. The objective of the cost function is to evaluate the network’s performance, to minimize its value during training. The cost function for a typical RNN-LSTM is the sum of losses at each time step [70].

$$\begin{array}{c}J\left(\theta \right)=\sum _{k=1}^{T}L(\widehat{y}\left(k\right),y\left(k\right))\end{array}$$

(6)

where $\theta$ represents the parameters of the RNN, T represents the length of the input sequence, ${\widehat{y}}_t$ represents the predicted output, and $y_t$ represents the actual output at time step k. L is the loss function quantifying the difference between the predicted and actual output. The RNN’s training parameters are adjusted to minimize the cost function using gradient descent or a comparable optimization algorithm. The objective is to identify the parameters that minimize the loss over all time steps, resulting in an RNN that can accurately predict the output for a given input sequence.

2.2. Problem-Solving Method

By analyzing the control processes illustrated in Figure 1a,b, it is possible to identify a number of expressions with total or partial equivalence between the two methods.

S-MPC can be solved implicitly by performing switching logic, forecasting, and resolving a dynamic optimization problem at each time step or explicitly by learning a control policy from data generated by a S-MPC with any function approximation. Consequently, S-MPC has a higher online computational cost, because every control step requires estimation of the states and dynamic optimization. Typically, the optimization problem in S-MPC is solved using numerical optimization techniques, such as nonlinear programming or quadratic programming (QP) (in this paper, QP has been used), to solve the optimization problem. The solution to the optimization problem over the prediction horizon provides the optimal control signal. At each time step, the first component of the optimal control signal is applied to the system, and the process is repeated with the updated state and prediction horizon values. S-MPC requires the solution of an optimization problem at each time step, which can be computationally expensive for large systems.

The training process for AR-LSTM involves back-propagation through time (BPTT), a variation in the back-propagation algorithm that considers temporal dependencies in the data. The RNN is unrolled throughout the training for a predetermined number of time steps, and gradients are calculated at each step. The RNN’s weights are then updated based on the gradients accumulated across all time steps. The most prevalent optimization algorithm for training RNNs is gradient descent, which involves updating the weights iteratively in the direction of the loss function’s negative gradient [69]. However, the standard gradient descent algorithm is susceptible to issues such as vanishing gradients, in which the gradients become extremely small and the weights do not update. Several variants of gradient descent, such as the adaptive gradient descent algorithms AdaGrad, RMSProp, and Adam, have been developed to address this issue [71].

2.3. Peak Performance

In S-MPC, the quality of the optimization solution depends on the controller model’s precision, which is frequently simplified for computational purposes. Stability and practicability are intrinsically ensure for S-MPC, whereas there is only an immature theory for these issues in AR-LSTM [72]. The absence of safety guarantees in AR-LSTM results from the constraints not being directly imposed in the formulation of the solution method. The optimality of the S-MPC solution depends on the accuracy of the model used to predict the system’s behavior and the optimization algorithm’s ability to find the optimization problem’s global optimum. If the model is inaccurate or the optimization algorithm fails to find the global optimum, the performance of the S-MPC controller may not be optimal.

The optimality of AR-LSTM relies on several factors, including the network’s architecture, the training optimization algorithm, and the complexity of the task being performed. AR-LSTM is capable of achieving high levels of performance on a wide variety of sequential data processing tasks, such as language modeling, machine translation, and speech recognition. AR-LSTM is able to model complex temporal dependencies in sequential data, which is one of its main advantages. The ability of AR-LSTM to incorporate feedback loops enables them to capture long-term dependencies that would be challenging to represent using other models, such as a gated recurrent unit (GRU). In addition, the ability to incorporate memory into the network via mechanisms improves the performance of RNNs in tasks requiring long-term memory. Nonetheless, several factors can restrict the optimality of AR-LSTM. One difficulty is the issue of vanishing and exploding gradients, which can hinder the network’s ability to discover long-term dependencies. This issue can be mitigated by employing specialized units, such as LSTM and GRU, and optimization algorithms designed to deal with these issues. Another issue is overfitting, which can occur when the model becomes excessively complex and begins to fit the noise in the data rather than the underlying patterns. This can be remedied by employing regularization techniques such as early stopping and dropout [73].

2.4. Calculational Effort

S-MPC can require significant computational effort, especially for large-scale systems. S-MPC necessitates the solution of an optimization problem in each time step, which can be computationally costly. Moreover, a significant disadvantage of S-MPC is the need to solve an optimization problem online, which can be complex and involve many optimization variables. Consequently, controller models for S-MPC are commonly simplified at the expense of optimality, and gains in optimization solver efficiency are highly desired. Moreover, switching logic and prediction must be performed at each control step. Nonetheless, several techniques have been developed to reduce the computational effort required for S-MPC, such as online optimization and ML techniques that update the optimization problem as the system evolves.

The computational effort required for training and utilizing AR-LSTM can be substantial, especially for large-scale problems with many time steps and/or parameters. BPTT is the primary computational bottleneck, because it is required to compute the gradients of the loss function with respect to the network parameters. Considering that the computational complexity of BPTT scales linearly with the number of time steps, training AR-LSTM on lengthy sequences can be computationally expensive. In addition, the number of network parameters can contribute to the computational complexity, as larger networks require more computation to update weights during training and to make predictions during inference. Several techniques have been developed to ease these computational challenges, including mini-batch training, which involves updating the weights based on a subset of the training data at each iteration, and gradient clipping, which involves capping the magnitude of gradients to prevent gradients from exploding during training [69].

3. Switched Auto-Regressive Neural Control (S-ANC)

This section introduces the specifics of the proposed novel S-ANC algorithm. The objective is to learn from the architecture of RNN-LSTM, while satisfying constraints. Switching logic, dynamic optimization, and learning are the elements from the control and ML communities that are effectively combined to achieve this objective. First, Section 3.1 introduces the hybrid MG structure. Section 3.2 provides an overview of how S-MPC and AR-LSTM are merged logically. Then, Section 3.3 formally describes the S-ANC algorithm.

3.1. Hybrid MG Description

This is a case study of a system constructed in Xanthi, Greece [74]. As depicted in Figure 3, the hybrid MG is comprised of a 15 kW PV array; a battery (BAT), a water tank (WT), and a fuel tank (FT) serving as energy storage systems (ESSs); an electrolyzer (EL); and a fuel cell (FC), as well as the utility grid (GR). The PV can be utilized in the hybrid MG as the primary energy source. If the PV cannot provide sufficient power, the BAT or the FC will meet the load. The GR will provide energy if the battery is depleted and no hydrogen is available. Alternatively, when the BAT is full and there is an excess, the EL will be utilized if there is space in the WT and the FT. The energy will then be sent to the GR.

3.2. Simple Definition of the Proposed Method

To comprehend the concept underlying S-ANC, one must first comprehend the distinctions between MPC and S-MPC when solving the QP and the two main learning methods of RNN: AR and LSTM. The S-MPC controller must be capable of selecting the appropriate model and control strategy based on the system’s current state, which necessitates additional computational resources and algorithmic complexity. In this paper, for instance, the system dynamics change significantly as the state of each accumulator in the hybrid MG changes; consequently, S-MPC can use different models for various states. This requires creating and validating multiple models, and the S-MPC controller must be able to switch between these models based on the current state.

The construction of S-MPC is challenging and intricate, particularly for a hybrid MG, which must accommodate many operating modes and complex switching conditions. This complexity is caused by a number of factors, including

• Model development: S-MPC requires the creation of multiple models that represent the system’s behavior in different operating modes. This requires an efficient system architecture and behavior;
• Mode detection: The S-MPC controller must be able to detect the current mode of operation of the system, which can be difficult in certain circumstances;
• Switching logic: The S-MPC controller must select the appropriate model and control strategy based on the current operating mode and desired performance objectives. This necessitates the designing of switching logic that maps the system’s current state to the appropriate model and control strategy (a mode’s objective function and an operational mode’s objective function may differ).

The S-MPC solution method takes information from the hybrid MG, such as PV and load data and accumulator parameters, including their charging and discharging efficiencies. Then, the input u, state x, and output vectors y are defined. Based on the controller model, the objective function J is inferred at each control step using this method. After that, the state vector is converted to an AR model $X\left(k\right)$, in order to predict the value at the subsequent time step. This is a straightforward concept that can produce accurate forecasts for various time series problems. Nevertheless, the AR model needs a plant model and a prediction horizon, so the computational time of the model is still high. Therefore, the current state x, input u, and output vectors y are updated using the AR-LSTM method.

S-ANC employs time series value-based AR-LSTM to estimate the value of being in a particular output vector $\widehat{y}$, as determined by S-MPC, with a prediction horizon of only one control step. By doing so, in S-ANC, the S-MPC method is truncated with the predicted output vector $\widehat{y}$ and optimized by the hybrid MG system during k steps ahead through employing the AR-LSTM method.Consequently, the principal components of the S-MPC, namely the reference, predictor, and switching logic, remain active in S-ANC; however, the time series value function is utilized to shorten the nonlinear program and enable learning. The interaction of S-ANC’s primary components is depicted in a diagram in Figure 4. The merging of S-MPC and AR-LSTM in the S-ANC algorithm is intuitively depicted in Figure 4.

3.3. Formal Definition

Initially, the system state, control, and output vectors are defined for the hybrid MG system in the S-MPC:

The system-state vector of the MG is as follows:

$$\begin{array}{c}x\left(k\right)=\left[SOAc{c}^l\left(k\right)\right]\end{array}$$

(7)

where $l\in \{BAT,FT,WT\}$. $SOAc{c}^{BAT}\left(k\right)$, $SOAc{c}^{FT}\left(k\right)$, and $SOAc{c}^{WT}\left(k\right)$ are the state of the accumulators for the battery, hydrogen tank, and water tank, respectively.

The system-control (input) vector of the MG is defined as follows:

$$\begin{array}{c}u\left(k\right)=[P_3\left(k\right);P_4\left(k\right);\dots P_{11}\left(k\right)]\end{array}$$

(8)

The system-output vector of the MG is defined as follows:

$$\begin{array}{c}y\left(k\right)=[P_1\left(k\right);P_2\left(k\right)]\end{array}$$

(9)

Consider the discrete-time linear state-space system:

$$\begin{array}{c}\mathbf{X}(k+1)={\mathbf{A}}_{x}x\left(k\right)+{\mathbf{B}}_{u}u\left(k\right)\end{array}$$

(10)

where $k=0,1,2,\dots ,N_P-1$ symbolizes the discrete-time instant.

By defining the following matrices:

$$\begin{array}{c}{\mathbf{A}}_x=\left[\begin{array}{c}I\\ A\\ A^2\\ ⋮\\ A^{N_P};\end{array}\right]{\mathbf{B}}_u=\left[\begin{array}{cccc}0& 0& \cdots & 0\\ B& 0& \cdots & 0\\ AB& B& \cdots & ⋮\\ ⋮& ⋮& \ddots & 0\\ A^{N_P-1}B& A^{N_P-2}B& \cdots & B\end{array}\right]\end{array}$$

(11)

where

$$\begin{array}{c}A=\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right]B^T=\left[\begin{array}{ccc}0& 0& 0\\ 0& 0& 0\\ {\eta }_{ch}& 0& 0\\ -{\eta }_{dis}& 0& 0\\ {\eta }_{ch}& 0& 0\\ -{\eta }_{dis}& 0& 0\\ 0& {\eta }_{ch}{,}_{H_2}& 0\\ 0& -{\eta }_{dis}{,}_{H_2}& 0\\ 0& 0& {\eta }_{ch}{,}_{H_{2}O}\\ 0& 0& -{\eta }_{dis}{,}_{H_{2}O}\end{array}\right]\end{array}$$

(12)

The linear state-space equation can be stated depending on the battery, fuel tank, and water tank equations as follows [75]:

$$\begin{array}{c}SOAc{c}^l(k+1)=SOAc{c}^l\left(k\right)+\frac{P_{a\to b}^j\left(k\right)-P_{b\to a}^j\left(k\right)}{C_l}\end{array}$$

(13)

where j is the energy flows, so $j\in \{Power,Hydrogen,Water\}$. $a\to b$ represents the energy flows between accumulators and converters; for example, $P_4$ is the power from the PV to the battery.

Define the constraints for the hybrid MG: Energy flows from the PV, GR, BAT, FT, EL, FC, and WT are positive and subject to maximum values.

$$\begin{array}{c}\begin{array}{cc}\hfill 0\le P_1& \left(k\right)=P_{LD}\left(k\right)-y_a\left(k\right)\le P_1^{max}\hfill \\ & 0\le P_m^j\left(k\right)\le P_m^{j^{max}}\hfill \end{array}\end{array}$$

(14)

where $P_m^{j^{max}}(m=1,2,\dots ,11)$ imply the maximum values of energy/matter flows.

 The sum of PV energy supplied directly for the load $\left(P_2\left(k\right)\right)$ and the battery for the charging $\left(P_4\left(k\right)\right)$ should be smaller than the energy flow from the PV array, $\left(P_{PV}\left(k\right)\right)$.

$$\begin{array}{c}{P}_2\left(k\right)+P_4\left(k\right)\le P_{PV}\left(k\right)\end{array}$$

(15)

The $SOAc{c}^l$ is restricted between the minimum and maximum values [16].

$$\begin{array}{c}SOAc{c}^{l^{min}}\le SOAc{c}^l\le SOAc{c}^{l^{max}}\end{array}$$

(16)

  Define the reference matrix $\left(\mathbf{R}\right)$ for the hybrid MG system:

$$\begin{array}{c}\mathbf{R}\left(k\right)=[w_{xx}(P_{LD}\left(k\right);P_{PV}\left(k\right);0);\dots ;P_{LD}(k+N_p-1);P_{PV}(k+N_p-1);0)]\end{array}$$

(17)

Design and control the multiple models (converting MPC to S-MPC) depending on several parameters as follows:

$$\begin{array}{c}\mathrm{if}{P}_{a\to b}^j<0;SOAc{c}^l>SOAc{c}^{l^{min}}\mathrm{and}SOAc{c}^l<SOAc{c}^{l^{max}}\end{array}$$

(18)

$$\begin{array}{c}\mathrm{if}{P}_{a\to b}^j>0;SOAc{c}^l>SOAc{c}^{l^{max}}\mathrm{and}SOAc{c}^l<SOAc{c}^{l^{min}}\end{array}$$

(19)

where i = 1, 2, …, 11.

Regarding the AR-LSTM formulation, if Equations (2) and (4) are merged, the new state vector will be as follows:

$$\begin{array}{c}\begin{array}{c}\hfill \mathbf{X}\left(k\right)=c+\phi \left(1\right)tanx(W_{xx}x(k-1)+W_{ux}u\left(k\right))+\phi \left(2\right)tanx(W_{xx}x(k-2)\\ \hfill +W_{ux}u(k-1))+\cdots +\phi \left(q\right)tanx(W_{xx}x(k-N_P)+W_{ux}u(k-N_P+1))+\epsilon \left(k\right)\end{array}\end{array}$$

(20)

The objective function of the hybrid MG system using S-ANC (the combination of Equations (1) and (6)):

$$\begin{array}{c}\begin{array}{c}\hfill J\left(k\right)=\sum _{k=0}^{N_P}{W}_{xx}\left(k\right){(\mathbf{R}\left(k\right)+\mathbf{X}\left(k\right))}^2+\sum _{k=0}^{N_P}{W}_{ux}\left(k\right)\Delta u{\left(k\right)}^2\\ \hfill +\sum _{k=0}^{N_P}{W}_{yy}\left(k\right)y{\left(k\right)}^2+p^2+\sum _{k=0}^{N_P}\widehat{y}\left(k\right),y\left(k\right)\end{array}\end{array}$$

(21)

$$\begin{array}{c}\begin{array}{c}\hfill 0=F(\mathbf{X}\left(k\right),y\left(k\right),u\left(k\right),\widehat{y}\left(k\right),\mathbf{R}\left(k\right),p)\\ \hfill 0\le H(\mathbf{X}\left(k\right),y\left(k\right),u\left(k\right),\widehat{y}\left(k\right),\mathbf{R}\left(k\right),p)\end{array}\end{array}$$

(22)

The main advantage of employing the formulation presented by Equations (21) and (22) is that it imposes short-term safety constraints, while allowing for continuous empirical experience-based learning. In addition, reducing the prediction horizon of the dynamic optimization problem significantly simplifies the resulting nonlinear program. Notably, both optimization functions from Equation (21) must be jointly merged, such that the state $\mathbf{X}$ must be related to the expected optimization variables in $k+1$. This results in lower overheads than optimizing with longer prediction horizons that must be discretized over time.

Notably, domain knowledge is encoded in the controller model F for optimization and control vectors, providing the algorithm with understandability. Then, the constraints are implied for the hybrid MG system. The next step is to automatically convert the traditional MPC into S-MPC. The final steps in the S-MPC are to solve the cost function and obtain “optimal decision variables”, as shown in Algorithm 1. After that, the hybrid AR-LSTM method is initiated by configuring the controller model F. The current state $\mathbf{X}$ is found using Equation (20), before training the “optimal control decisions”. Finally, the control variable $\mathbf{U}$ and $\widehat{y}$ are solved by utilizing updated reference $\mathbf{R}$ and Equations (21) and (22).

To begin, design a model of the MG system. The system reads some MG specifications, such as PV and load data, accumulator data, and the maximum values of power flows among the components of the hybrid MG. Following that, the MPC controller is implemented, which will state the optimization problem and solve it at each time step, to obtain the optimal control inputs for the next time step. However, the MPC is converted into the S-MPC before it is applied. The optimization problem should consider the objectives and constraints given in this paper’s methodology section. Implement an AR-LSTM model and train it on past data to increase the accuracy of the predictive model utilized by the S-MPC controller. Based on the present and previous system conditions, the AR-LSTM should be able to anticipate future MG behavior. The prediction should be input into the S-MPC controller’s optimization problem. Finally, as indicated in the methodology section of this paper, the S-MPC and AR-LSTM controllers in a closed-loop control system are combined. The proposed method can test the control strategy under various operating situations and evaluate its performance using the provided performance criteria (cost functions).

Algorithm 1: Switched Auto-regressive Neural Control (S-ANC)

More specifically, to implement our proposed method in operation, initially, model the MG system and the S-MPC and AR-LSTM controllers, and then combine these models into a closed-loop control system. Here are the detailed steps that need to be followed, as illustrated in Figure 5:

• Initiate the system specifications and operational conditions from the MG operator;
• Solve the systematic generation of the control problem, employing the MPC with the QP;
• Using switching logic, automatically convert the MPC into the S-MPC;
• The optimal control decisions are obtained;
• The optimal control decisions are employed as input data for the AR method;
• The data preparation is initiated. This step has several parameters, such as data cleaning, extracting features, and merging the input data and PV constraints;
• The AR model is implemented to increase the accuracy of our proposed method;
• After that, multivariate time series are employed;
• Then, the training and test data are selected and evaluated;
• To move the LSTM layer to after the RNN, a sequential network of an input LSTM layer is produced;
• In this step (implementation of LSTM), several parameters are defined, including the batch size, epoch number, and type of optimizer;
• Before moving the calculation to the model accuracy, the scaling for the forecast and actual data are inverted;
• The model accuracy is calculated using various methods, along with the mean directional accuracy, $R^2$ method, and so on;
• Integrate the S-MPC and AR-LSTM controllers into a closed-loop control system by connecting the RNN output to the MPC controller’s input, and the MPC controller’s output to the MG system’s input;
• Then, the optimal control decisions and references are updated. In other words, $\mathbf{X}$, $\mathbf{U}$, and $\mathbf{R}$ are re-evaluated depending on the model accuracy;
• If this accuracy is unreasonable, the S-MPC is reapplied using the updated control decisions.

4. Results and Discussion

4.1. Case 1: Implementation of S-MPC

The non-optimal and optimal control (S-MPC) were compared for 96 h (four days) in this case. In other words, in case 1, the emphasis was on the optimization of S-MPC and its effect on the power flows of the system (Figure 6a) and the SOC of the battery (Figure 6b). Non-control methods employ simplistic control strategies or heuristic rules, disregarding the system’s dynamic nature. In addition, they do not have any constraints, so there are disadvantages associated with this strategy, such as poor SOC management, potential deviations from desired SOC levels, and an inability to adapt to changing system conditions. As shown in Figure 6b, the SOC of the battery went below the critical value (20%), since the non-optimal method had no constraints. In contrast, the S-MPC method is an alternative to the non-optimal control method. S-MPC’s ability to dynamically select the appropriate model and control strategy based on the system’s current state is one of its key advantages. S-MPC optimizes control actions to achieve the desired performance objectives, particularly in effectively managing the SOC of a battery.

The S-MPC controller was designed to select the optimal model and control strategy based on the current operating mode and performance objectives. S-MPC enables enhanced power flow control and EM by effectively adapting to the changing dynamics of the hybrid MG, utilizing distinct models for each state. Case 1’s implementation of the S-MPC controller successfully optimized the hybrid MG system’s power flows. It substantially reduced energy imports and increased energy exports, resulting in a more efficient use of resources and enhanced energy flow management. The improved control strategy enabled the MG to operate closer to its optimal performance, enhancing its dependability and reducing the operational costs. However, it is important to note that developing and implementing the S-MPC controller for the hybrid MG system presented obstacles due to the complex switching conditions and multiple operating modes. To guarantee the selection of the optimal model and control strategy, the switching logic had to be meticulously designed. The controller’s increased complexity required more computational resources than traditional MPC methods. The model’s computational time was almost 405 s.

4.2. Case 2: Implementation of the Merged S-MPC and AR

Case 2 investigated the integration of the AR model with S-MPC. AR models accurately predict future time steps by capturing the time series behavior of the system. Various analyses, including variations, cross-validation (CV) iteration-time series behavior for training and validation, CV iteration-training data for each CV iteration, and predictions ordered by test prediction number were used to evaluate the performance of the AR models.

Figure 7a shows a visualization designed to provide insight into the behavior of the lagged target feature over time. We can identify patterns, trends, and correlations within the lagged target data by examining the plot. Understanding the characteristics of the lagged target can aid in developing and optimizing an AR linear regression model that uses this characteristic for prediction. By displaying the lagging feature of the target, we can observe its values across multiple time steps. This lets us determine whether the lagged target exhibits specific patterns, trends, or variations over time.

As illustrated in Figure 7b, the variations in the AR models illustrate their capacity to capture and model the system’s complex dynamics. By analyzing the CV iteration-time series behavior, the adaptability of the AR models to various patterns and tendencies in the training and validation datasets was assessed. This analysis shed light on how the models learn and generalize from the available data, enabling accurate predictions for various time series problems.

Case 2’s successful integration of AR models with S-MPC illustrated the importance of incorporating time series behavior and forecasting capabilities into a control system. Combining S-MPC and AR models permitted enhanced adaptation to system dynamics and improved the prediction accuracy, thereby enhancing the MG’s overall control performance.

4.3. Case 3: Implementation of the S-ANC

Case 3 examined the combination of S-MPC and AR-LSTM models. Integrating these advanced models aimed to enhance the predictive capabilities of the control system. The S-ANC predicted the last month of the year using the first eleven months as training data. Using metrics such as train–test (Figure 8a) and grid consumption prediction with AR regression and S-ANC (Figure 8b), the performance of this merged approach was evaluated. This integration (S-MPC and AR-LSTM) increased the precision of forecasting and the precision of power flow optimization. The enhanced prediction capabilities of the AR-LSTM models allowed the control system to anticipate future energy requirements and adjust the operation of the MG accordingly. In Case 3, the S-MPC controller was improved by combining it with AR and RNN-LSTM models. This integration aimed to improve the precision of predictions and the overall performance of the control system. Two primary figures were generated for analysis: a comparison between the training and the test, and a prediction of grid consumption using AR regression and S-ANC.

The train–test comparison diagram visually represents the AR-LSTM models’ capacity to generalize effectively for unobserved data. It compares the predicted grid consumption values during the testing phase with the actual values, indicating the AR-LSTM models’ ability to capture the hybrid MG’s complex patterns and dynamics (Figure 8b). The diagram depicts the performance of the combined S-MPC, AR, and RNN-LSTM models on the training and testing datasets (Figure 8a). According to our simulation, the model could generalize well for new data and the integration strategy’s effectiveness. Moreover, the prediction of grid consumption using AR regression and S-ANC illustrated the ability of the combined method to optimize power flows, while accurately predicting future power demands. By leveraging the predictive capabilities of AR-LSTM models within the S-MPC framework, the control system could more precisely estimate grid consumption, allowing for more effective EM and enhancing the MG’s adaptability to load demands and renewable energy generation fluctuations. The grid consumption forecast graph (Figure 8b) depicts the projections generated by the AR and S-ANC models. This enables a comparison of the two methods and highlights the advantages of the S-ANC method, which employs the AR-LSTM model for accurate predictions, while reducing computational time. The computational time of the model was reduced by nearly 214 s.

4.4. Calculation of Model Accuracy

To comprehensively evaluate the performance of the S-ANC prediction model, three unified evaluation indices, including R-squared score, mean absolute error (MAE), and mean square error (MSE) [76,77] were selected in this paper. The evaluation indexes have the following mathematical definitions:

R-squared Score:

$$R^2=1-\frac{{\sum }_{i=1}^n{(y_i-{\widehat{y}}_i)}^2}{{\sum }_{i=1}^n{(y_i-\overline{y})}^2}$$

(23)

Mean Absolute Error:

$$MAE=\frac{{\sum }_{i=1}^n|y_i-{\widehat{y}}_i|}{n}$$

(24)

Mean Squared Error:

$$MSE=\frac{{\sum }_{i=1}^n{(y_i-{\widehat{y}}_i)}^2}{n}$$

(25)

where $y_i$ is the output vector using the S-MPC; ${\widehat{y}}_i$ represents the predicted value of the output vector by employing the S-ANC; $\overline{y}$ represents the average value of output vector; n represents the total number of samples.

In this study, we assessed the performance of our predictive model using various accuracy metrics. The R-squared error, which measures the proportion of the variance in the dependent variable that can be predicted from the independent variables, was 0.951. This suggests that our model accounted for approximately 95.1% of the data variance, indicating a strong predictive ability. In addition, we determined that the MAE was 0.571. The MAE is the mean absolute difference between observed (from the MPC) and predicted values. A smaller MAE indicates that the predicted and observed values correspond more closely. In our case, the relatively low MAE indicates that our model’s predictions deviated from the true values, on average, by approximately 0.571%. Likewise, we determined the MSE to be 0.571. The MSE measures the average squared deviation between predicted and observed values. Similarly to the MAE, a lower MSE indicates greater precision. The MSE value of our model indicated that the squared differences between the predicted and observed values were, on average, 0.571 units. Overall, the results show that our predictive model was effective. The relatively low MAE and MSE values of 0.571 indicated precise predictions, with minimal deviations from the actual values.

5. Conclusions

Our findings show the efficacy and advantages of the S-ANC method for the intelligent control and management of hybrid MGs. The optimization of S-MPC improves energy management and power flow control, resulting in more efficient use of resources. The integration of AR and RNN-LSTM models improves the accuracy of predictions, allowing the control system to adapt to dynamic system conditions and to optimize the operation of the MG. The successful implementation of S-ANC significantly affects the dependability, sustainability, and cost-effectiveness of hybrid MG systems. We can achieve efficient control and management of complex energy systems by leveraging the capabilities of advanced modeling techniques within the S-MPC framework. These findings support the incorporation of hybrid MGs in future energy systems and contribute to developing intelligent control strategies. By combining the AR-LSTM, the computational time of the model was reduced by approximately 47.2%. In addition, this study assessed the accuracy of our predictive model. The R-squared error, which quantifies the amount of variance in the dependent variable that can be predicted from the independent variables, was 0.951. Our model predicted 95.1% of the variance in the data, indicating a high level of predictive ability. The MAE and MSE values of 0.571 indicated precise forecasts, with minimal deviations from the actual values. The focus of future research and development should be on validating larger-scale systems and incorporating additional advanced models. These developments will enhance the performance and applicability of the S-ANC methodology and contribute to the efficient operation and integration of hybrid MGs in future energy systems.