Machine Learning for Energy Systems Optimization

1. Introduction

This editorial overviews the contents of the Special Issue “Machine Learning for Energy Systems 2021” and review the trends in machine learning (ML) techniques for energy system (ES) optimization. This Special Issue focuses on reviewing severe challenges (e.g., the poor quality in data, underfitting, overfitting, or lack of training data), cutting-edge contributions (e.g., the optimization of ESs considering costs and grid operational constraints), and trends in ML for ESs. For this purpose, we collected several papers on future ESs that will inevitably exhibit increased complexities because of the increase in the capacity of distributed ESs as well as conventional generation plants enhanced with advanced technology (e.g., high-efficiency combined cycle gas turbines). Such an ES requires not only higher reliability and security but also the smooth integration of distributed ESs into the existing grid, without losing high functional improvement. This article summarizes the major findings and discussions of the Special Issue, which includes 13 research articles on ML techniques for ESs. In addition, this article details the challenges and problem-solving techniques for ES optimization, particularly those using ML techniques. We hope that this Special Issue that solves various optimization problems for ESs will be helpful to academics, industries, and other researchers who intend to improve the reliability and performance of ESs, develop ML techniques for any other application (e.g., thermal energy providing systems), including ESs, and examine the effect of the optimized ESs on their seamless integration into conventional systems.

Electric energy systems (ESs) are typically designed to provide reliable and safe electric energy services to customers. However, the installation of distributed generation (DG) resources or wind and photovoltaic (PV) resources, which intrinsically include uncertainty and variability in their outputs, increases the complexity of operating and controlling the electric power grid [1]. Additionally, energy storage systems such as pumped hydroelectric systems, compressed air, batteries (lithium-ion, lead-acid, lithium iron, flow battery, etc.), flywheels, and supercapacitors are deployed with DG resources to compensate for the variability in DG resources. Thus, most machine learning (ML) algorithms related to ESs attempt to deal with the optimal sizing, placing, scheduling, coordination, and selection of DG resources and energy storage systems.

Optimally allocated DG resources can have direct and indirect effects on the smooth integration of DG resources into electric power systems. The direct effects can be summarized as follows [2,3,4]: (1) improved ability to deliver energy via voltage support, (2) flexibility and reliability enhancement to meet load variations, (3) decrease in losses because of reverse power flow from DG resources, (4) more effective peak load reduction for expensive generation costs, and (5) islanding operations if the total generation of DG resources exceeds the total demand of a preset islanding zone and for well-coordinated protection [5]. The indirect effects can be summarized as follows: the reduction in electricity production costs [6]; technological advancement in power-processing equipment such as inverters, converters, rectifiers, storage, and any other controllers; and regulatory and political impacts [7].

Many studies have ensured the seamless, reliable, and safe integration of DG resources, including energy storage systems, into existing electric power grids by optimally determining their location, capacity, scheduling, or selection. Such an optimization problem (e.g., the allocation of the location and capacity, scheduling or unit commitment problems, or benefits evaluation) of the DG system was solved by various heuristic search methods (e.g., genetic algorithm [8], gradient-based search [9], particle swarm optimization (PSO) [10], Fibonacci tree [11], tabu search [12], and bee colony algorithm [13]), mathematical programming (e.g., dynamic programming [14], mixed integer nonlinear [15] or linear programming [16], and quadratic programming), statistical time-series analysis (e.g., autoregression moving average [17], probability models with Weibull and beta distributions [18], Monte Carlo simulation [19], and the big-M linearization method [20]), sensitivity analysis algorithms [21], mathematical formulation (e.g., Lagrangian methods [22]), and any other methods (e.g., alternating direction method of multipliers [23], game theories [24], and the general algebraic modeling system [25]).

These computer algorithms can be regarded as ML because they improve their objective function values by experience or the use of training data. Thus, a properly defined objective function plays an important role in the fast convergence or non-convergence to local minima after an appropriate number of iterations. Thus, many optimization studies on ESs have defined their objective functions in the following three categories:

Economic issues. In these studies, the cost models evaluate the effect of DG on economic costs, maximize annual profits, or minimize investment costs and expenses [26,27].

Environmental issues. The main objective is to minimize greenhouse gas emissions [28]. This objective is in agreement with the Paris COP21 policy for maintaining global warming below 2.0 °C by reducing greenhouse gases released by ESs [29].

Grid constraints. Various grid operational constraints are added to objective functions to operate and maintain the grid reliably and safely. The grid operational constraints include acceptable limits of voltage, line flow constraints, losses [30], stability, and frequency. Reliability constraints such as energy not supplied, the value of the lost load, and the loss of load expectation are also added to the objective functions [31]. Power quality constraints, such as harmonic distortion, are used in the objective function to maximize the network’s performance [32]. The reactive power [33,34] constraint and bus-type constraint [35] are also incorporated into the objective function.

The objective of this paper is to review the contents of the Special Issue “Machine Learning for Energy Systems 2021” and the trends in the latest ML techniques for ES optimization. In particular, this study reviews the latest ML algorithms with various objective functions. This paper is organized as follows. Section 2 introduces the main contributions of the papers published in the Special Issue. Section 3 reviews the various algorithm implementations, problem-solving methods, and objective functions of ML algorithms. Section 4 summarizes the major findings of this study.

2. Contributions of the Papers Published in This Special Issue

From the perspective of applying optimization and artificial intelligence (AI) to the field of electrical machines, refs. [36,37] were published in this Special Issue. In [36], researchers suggested a temperature prediction method for permanent magnet synchronous motors using deep neural networks so that faults caused by high temperature could be prevented. In addition, ref. [37] suggested an estimation method for transformer oil volume changes for stressed oil passages in major insulations. Because the oil volume change affects the rated capacity and voltage of the transformers, the accurate prediction of this parameter can be used to predict electrical equipment failure.

Refs. [38,39] suggested a combined heat and power (CHP) optimization method. Ref. [38] showed that an optimized CHP dispatch can decrease the electricity purchased from the grid and emission release. Moreover, ref. [39] showed that a microturbine-based CHP, without absorption chillers, was the most cost-effective type for metropolitan residential customers. The additional cases revealed that optimal CHP allocation with full-blast microturbines and absorption chillers effectively optimized energy consumption, cost, and emission reductions.

Refs. [40,41,42] showed that heuristic search-based DG allocation can be applied in the field. In [40], a PSO method could maintain the Volt/Var control scheme of DG. In [41], a PSO was also used to find the optimal dispatch place when minimizing the levelized cost of energy and fault current. In [42], a linear two-stage active and reactive power coordination optimization method was shown to improve voltage sensitivity through optimal DG allocation.

Trends in the applications of AI and optimization methods for power grids have been reviewed. In particular, ref. [43] focused on the resiliency and survivability of the grid and reviewed the trends that would affect the grid. In addition, ref. [44] suggested a flexible digital platform for digitalizing power grids.

In [45,46,47,48], prediction and forecasting methods were suggested for power system operation. In [45], bad data from residential loads were detected and customized for imputation with a high accuracy rate using the probabilistic forecasting method. In [46], an AI-based battery state prediction method, with high accuracy and low CPU occupation time, that can be used for online applications was presented. In [47], AI showed a high prediction accuracy in the PV energy harvesting amount despite the use of low-cost IoT devices. In [48], a non-intrusive load monitoring (NILM) method in a multi-agent architecture was shown to improve detection and classification scores compared to previous NILM algorithms.

3. Trends in Machine Learning Optimization for Energy Systems

The main benefits of DG resources that use renewable energy are summarized by their direct and indirect effects on environmental, technological, economic, grid operational, and regulatory aspects. In the last few decades, the optimization problems of ESs (e.g., the optimal capacity, location, number, type of ES and DG resources, their parameters, configuration, and the controlling and scheduling of ES and DG resources) have been solved to assist in sustainable energy system operation and control that may include various DG resources such as PV, wind, biomass, geothermal, hydropower, microturbines, electric vehicles, heating and cooling systems, and any other small-scale DG units. Such an optimization problem of ESs or an optimal power flow (OPF) problem can be solved by methods that fall in the following three categories: (1) conventional mathematical programming methods; (2) recent optimization algorithms that mimic not only the social behavior and insights of humans but also the responses and characteristics of animals and objects in order to achieve their objectives, various search algorithms (e.g., tabu search, harmony search, cuckoo search), and probability and stochastic methods (e.g., autoregressive models, Markov decision processes, Monte Carlo methods, Bayesian optimization, and any other uncertainty-solving techniques); and (3) ML approaches, including artificial neural networks and fuzzy logic. The first conventional method includes linear, nonlinear, and quadratic programming approaches; Newton’s method; and the interior point method [49]. Recent optimization algorithms include genetic algorithms, evolutionary programming, bacterial foraging, bee colony, ant colony, PSO, simulated annealing, differential evolution, honeybee mating, whale optimization, biogeography-based, firefly, imperialist competitive, plant growth simulation, shuffled bat, shuffled frog leaping, gradient descent, Pareto optimization, game theories, decision making, and agent-based algorithms. As the third approach, ML has recently been applied to solve optimization problems related to ESs. In other words, various ML implementations, management, and challenges have been intentionally introduced into the optimization of ESs. These ML algorithms have the following three characteristics:

(1)

Objective functions that define their goals (e.g., economic costs, annual profits, or fine-tuned cost functions);

(2)

A model that represents the behaviors and responses of the objective functions, which are evaluated in the optimization or training processes;

(3)

Repeated optimization and training procedures that enhance the efficiency and performance of the ML objective are attained by minimizing or maximizing each objective function value and their combination.

Training procedures can be classified as supervised, unsupervised, reinforcement, online, or transfer ML. If ML is a subset of AI, deep learning (DL) is a subset of ML [50]. To apply ML or DL techniques to the development and optimization of sustainable ESs, many studies have attempted to solve the following challenges:

(1)

Bidirectional power flow caused by the variable and intermittent outputs produced by DG resources, which should not violate thermal, voltage, and stability limits;

(2)

Increase in the complexity of conventional power grids caused by DG resources. For example, the optimal selection of DG resources is determined by economic, environmental, operational, or political issues;

(3)

New environments in the power market. For example, the power market can differ from a conventional vertically integrated monopolies to a fully competitive power market;

(4)

Various constraints that must be considered. The grid operational constraints (e.g., energy balance, power flow, peak load reduction, voltage drop, loss, overloading, harmonic distortion, power factor constraints, and frequency), reliability (e.g., failure rate and duration), stability, and protective (e.g., preventive, emergency, or restorative actions) coordination constraints (e.g., short-circuit current) should always be considered;

(5)

Reliability, robustness, and adaptability optimization of controllers and devices in ESs.

Thus, to solve the optimization problem of ESs subjected to the aforementioned equality and inequality constraints, the OPF method was introduced in 1962 [51]. However, an optimization problem solved by such an OPF method or various optimization and search algorithms can be slow, complex, or sometimes unsuitable. Therefore, the ML approach has become increasingly useful. For example, ML techniques have been applied for the assessment and control of multienergy systems from the viewpoint of reliability [52]. Recently, the dual variables of OPF were incorporated into ML by combining deep learning and Lagrangian duality [53]. These ML algorithms that have focused on the optimization of ESs aim to accelerate the conventional ES optimization models, decentralize the centralized optimization models, or speed them up by approximating the iteration processes and parameters related to optimization. In other words, a few ML algorithms have improved the optimization of current ES optimization models rather than been used in the development of new models. For example, a streaming algorithm that learns the optimization parameters from training samples was presented [54]. These ML algorithms improve the calculation and converge to the global optima of their objective function values through experience or the use of training data by reducing the complexity of the OPF problem.

While ML algorithms have improved conventional ES optimization models, new models using ML techniques have become increasingly important [55]. For example, ML approaches can be used to schedule and control large-scale (e.g., utility scale) or small-scale (e.g., microgrids) power grids or to maximize the energy usage of DG resources. ML techniques (e.g., neural networks) have been adopted to control microgrids [56]. Various applications of ML for demand response (DR) (e.g., optimally operating, scheduling, or choosing DR devices) have also been reviewed in [57]. A supervised ML framework that operates DG resources by learning inverter controllers with various OPF objectives was proposed in [58]. ML can maximize DG resource generation by applying it to the maximum power point tracking algorithm, layout configuration problem, or parameter optimization of PV and wind turbines.

Other ML approaches are related to the design of energy transaction markets. The optimal design of the electricity market relies on determining the optimal electricity prices and the Nash equilibrium in energy transactions or bidding [59]. These ML approaches were also employed to forecast prices [60].

Another set of ML techniques has been employed for various ES optimization problems. For example, conventional state estimation of the grid state has been enhanced by a deep neural network [61]. ML is also used to estimate the electricity supply and demand [62], and the power output of DG resources (e.g., solar PV and wind) [63]. ML has been used to decrease greenhouse gas emissions [64]. ML has also been applied to various predictive maintenance studies by detecting or forecasting a fault in ESs (e.g., conventional plants, DG resource generators, inverters, converters, rectifiers, transmission and distribution lines, ground and underground wires, cables, energy storage, and HVDC systems). For example, the optimal scheduling of cables was determined using an ML technique (e.g., Monte Carlo simulation) [65]. Planning, management, and political strategy formulation, as well as any other field of ESs, can be one of the application areas of ML approaches.

4. Conclusions

Currently, ML approaches, including supervised, unsupervised, reinforcement, online, transfer, deep learning, support vector machines, and decision trees, have been utilized to (1) enhance conventional optimization models and (2) to develop new robust and adaptive ML models. These two approaches are expected to become more complementary to each other to reliably, robustly, adaptively, and flexibly solve the optimization problem of ESs. This Special Issue article reviews the contents of the Special Issue, “Machine Learning for Energy Systems 2021” and the trends in ML techniques for ES optimization. We hope that the research papers published in this Special Issue and this review article will inform the readers about the developments in ML techniques for ES optimization in academia and industry.