Next Article in Journal / Special Issue
Algorithms for Optimal Power Flow Extended to Controllable Renewable Systems and Loads
Previous Article in Journal
Dynamical Recovery of Complex Networks under a Localised Attack
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Algorithms
Volume 14
Issue 10
10.3390/a14100275
algorithms-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

A Review of Parallel Heterogeneous Computing Algorithms in Power Systems

by
Diego Rodriguez
1,2,
Diego Gomez
2,
David Alvarez
1 and
Sergio Rivera
1,*
1
Electrical and Electronic Engineering, Universidad Nacional de Colombia, Sede Bogotá, Bogotá 111321, Colombia
2
GERS USA, Weston, FL 33331, USA
*
Author to whom correspondence should be addressed.
Algorithms 2021, 14(10), 275; https://doi.org/10.3390/a14100275
Submission received: 24 August 2021 / Revised: 17 September 2021 / Accepted: 21 September 2021 / Published: 23 September 2021
(This article belongs to the Special Issue Algorithms in Planning and Operation of Power Systems)
Browse Figures
Versions Notes

Abstract

:
The power system expansion and the integration of technologies, such as renewable generation, distributed generation, high voltage direct current, and energy storage, have made power system simulation challenging in multiple applications. The current computing platforms employed for planning, operation, studies, visualization, and the analysis of power systems are reaching their operational limit since the complexity and size of modern power systems results in long simulation times and high computational demand. Time reductions in simulation and analysis lead to the better and further optimized performance of power systems. Heterogeneous computing—where different processing units interact—has shown that power system applications can take advantage of the unique strengths of each type of processing unit, such as central processing units, graphics processing units, and field-programmable gate arrays interacting in on-premise or cloud environments. Parallel Heterogeneous Computing appears as an alternative to reduce simulation times by optimizing multitask execution in parallel computing architectures with different processing units working together. This paper presents a review of Parallel Heterogeneous Computing techniques, how these techniques have been applied in a wide variety of power system applications, how they help reduce the computational time of modern power system simulation and analysis, and the current tendency regarding each application. We present a wide variety of approaches classified by technique and application.

1. Introduction

The growth in energy dependence has led to increased complexity in the planning and operation of power systems [1]. In some scenarios, systems are observed operating at stressed conditions that, in the event of failures, can result in blackouts or cascading events resulting in large economic losses. For this reason, multiple studies have been carried out to evaluate the possible conditions that may appear on the network and avoid undesired power losses. As a result of yearly studies, operational criteria is generated to maintain system security under various conditions.
Operation considerations and study area limitations are not commonly updated during operation and short term planning, as a result of significant processing efforts and decision making requirements. In order to reduce execution times and to guarantee the optimal and secure operation of the network, different strategies have been implemented; some of the approaches reduce the size of a problem’s formulation, the number of impacted areas during the analysis, the number of decision variables, etc. Other options use the potential of current technologies, such as advanced computational structures.
The application of advanced computational structures during the operation and short-term planning of power systems has been integrated more frequently during the last fifteen years [2,3,4,5,6]. In particular, the application of structures focused on the parallel computation of burdensome tasks and the interactions among different technologies to solve power system problems have emerged as a possible solution.
One of the solutions involved is the use of High Performance Computing (HPC). HPC is a concept that has been long intertwined with Parallel and Distributed Computing (PDC). Many authors now even consider them to be inseparable [4,5,7]. This is mostly because a large part of today’s supercomputers use parallel, heterogeneous, and distributed architectures or systems [8]. However, HPC as a concept has generally referred to computers that highly outperform what is commercially available or what is considered normal performance [9]. Previous associations for the concept of HPC include Vector and Reduce Instruction Set Computer (RISC) processors, which, at one point, were considered high-end technology [9]; whereas, currently, some of these technologies may be taken for granted.
For the purpose of this review, the use of the terms Parallel Computing (PC) and Heterogeneous Computing (HC) will be preferred, as the term HPC is often considered too broad and often associated with many different definitions, as noted by [9]. The HPC definition is likely to change again in the future, hence, making the use of more explicit concepts referring to the type of technology used more appropriate. The necessity of precise language will increase if trends, such as Quantum Computing (QC), or completely new technologies continue to emerge and make their way into applied sciences and engineering applications.
The HC definition from [10] will be adopted for this review, which refers to systems that clearly and explicitly include processors of different types, such as Central Processing Units (CPUs), Graphics Processing Unit (GPUs), and Field-programmable gate arrays (FPGAs), interacting in on-premise or cloud environments.
During the last three decades, multiple works have applied Parallel Heterogeneous Computing (PHC) architectures in a wide variety of power system applications. In the beginning, CPU clusters connected through Message Passing Interface (MPI) were the first alternative. Then, technologies, such as fog and cloud computing were used not only to reduce simulation time but also as a solution to handle all the measured data of smart grids. Finally, during the last decade, researchers found, in FPGA and GPU architectures, a useful and optimized mechanism to parallelize tasks. FPGA architecture allows application of the Multiple-Instruction, Multiple-Data (MIMD) technique where different instructions can be executed on multiple data points simultaneously.
On the other hand, GPU architecture provides the possibility to use the Single-Instruction, Multiple-Data (SIMD) technique for common tasks where the same operation is performed on multiple data points at the same time. Due to the optimized GPU architecture for SIMD operations and developed toolkits and frameworks to program GPU instructions in non-graphic programming languages, such as C, C++, and FORTRAN, researchers have discovered how to use GPU in different power system applications where common instructions are recurrent. The literature review revealed more than 200 works using PHC in power system applications over the last three decades.
The review mainly focuses on research works where PC and HC are implemented in the following areas to limit the scope of the survey: power flow analysis, transient stability, contingency analysis, smart grids, Optimal Power Flow (OPF), electromagnetic transient simulation, renewable energy integration, dynamic state estimation, power quality, dynamic models, electrical vehicles, probabilistic power flow, Security Constrained-Optimal Power Flow (SCOPF), Transient Stability-Constrained Optimal Power Flow (TSCOPF), power system visualization, short circuit analysis, power system planning, power system reliability, electricity market, small-signal analysis, security-constrained economic dispatch, and hydrothermal scheduling.
The review classifies the works into two groups: works where GPU is used and works that use CPU clusters, fog or cloud computing, or FPGA in power system studies and applications. This classification helps to organize the review since GPU applications represent more than half of all the articles. However, GPU applications are the most recent works of PHC techniques in power systems since 80% of the articles published during the last five years correspond to GPU implementations.
This document is organized as follows. Section 2 presents an overview of the PHC techniques applied to power system studies and analysis. Section 3 organizes all works by technology and power system applications. Section 4 presents the conclusions, and Section 5 outlines the future work of PHC in power system applications.

2. Parallel Heterogeneous Computing

As defined in the previous section, PHC refers to systems that include processors of different types, such as CPUs, GPUs, and FPGAs interacting in on-premise or cloud environments, where the system architecture allowed processors to execute multitasks in parallel. The authors in ref. [8] categorized parallel computing based on how instructions and data are handled. Single-Instruction, Single-Data (SISD), SIMD, and MIMD represent the classification of parallel computing. SISD refers to serial computing where a single instruction is executed on a single data stream at a time.
SIMD refers to the processing of a single instruction concurrently on multiple data elements. Finally, MIMD defines a computing strategy where multiple processing units exist with their own instructions and data. This parallel computing technique typically employees MPI to communicate the processing units. MIMD is also known as multicomputers. This section presents different processing units used in PHC implementations in on-premise and cloud environments in power system applications.

2.1. Central Processing Unit

Multiple CPUs configured in workstations stand as the conventional PHC technique since they combine different processing units that perform tasks simultaneously. This technique requires a communication network to connect all nodes of the system. This technology is a clear example of the MIMD parallel technique. However, MIMD can be seen as SISD implementation in independent workstations connected by a communication network. As defined in ref. [8], the CPU is the fastest component of a computer that consists of a group of registers that contain the instructions to be executed. Those instructions are sent to the hardware to perform tasks, such as fetching, storing, or operating data. Registers are usually different for instructions, addresses, and operands. CPUs can contain components to accelerate the processing of floating-point numbers.

2.2. Fog and Cloud Computing

PHC techniques can run on-premise or in cloud environments. Most the CPU workstations run in on-premise environments where researchers grant not only the group of computers but also the communication network the ability to exchange instructions and data between the system nodes. Regarding cloud environments, fog and cloud computing have been used as distributed and centralized infrastructures that host and connect the processing units of PHC implementations. Running on cloud environments, nodes with CPU and GPU are configured to execute a wide variety of tasks. Multiple nodes represent the MIMD parallel computing technique.
Cloud computing refers to the on-demand provisioning of system services, such as multiple software, servers, data storage with the corresponding databases, and communication networks accessed remotely through the internet. Fog computing complements cloud computing giving the possibility of cloud applications to use on-premise resources to perform part of the application computation, storage, and communication. In other words, fog computing refers to an additional layer of cloud computing where on-premise resources are used. Within the services provided by this environment, cloud computing can integrate a wide variety of HC technologies and define schedulers to execute tasks in parallel.

2.3. Field-Programmable Gate Array

PHC systems can combine not only CPUs and GPUs but also integrated circuits, such as FPGAs. PHCs can take advantage of the high-performance programmable logic of Application-Specific Integrated Circuit (ASIC) designs. FPGAs are devices based on a matrix of logic blocks connected through programmable interconnections. After manufacturing, FPGAs are systems that can be reprogrammed to fit any application. This flexibility helps to parallelize the tasks of power system applications in HC systems with the MIMD approach. These devices can be integrated into on-premise or cloud environments as one of the provided services.

2.4. Graphics Processing Unit

The first decades in microprocessor advancements were focused mainly on serial workloads [11,12]; whereas more recently, CPUs have evolved to provide hardware that seizes parallelism through pipelining and even multi-core architectures. However, as pointed out by ref. [11], most of the circuitry on a CPU is devoted not to arithmetic, logic, and direct parallelism execution but to control complexity, such as caches, instruction decoders, and branch predictors, among others.
Instead, better results can be achieved with another paradigm, namely that of a GPU, which was developed using the general concept of vector processors. These vector processors allow the hardware to operate using registers that contain separate values [12]. Due in part to gaming and other consumer application demand over the last two decades, manufacturers, such as NVIDIA and AMD, now provide affordable GPU computing products on the market [11]. Some providing tools, such as CUDA, make scientific computing programming with GPUs an increasingly easier task.
GPUs are powerful co-processors designed to exploit data parallelism based on the SIMD approach. Modern GPU architectures embed hundreds to thousands of computing cores along with dedicated units. Several of the components are part of its own memory hierarchy, such as memory banks and caches. These elements are organized into so-called Streaming Multiprocessors (SM), where there is a warp/dispatch scheduler (more generally a thread scheduler), which distributes threads to be executed at different Scalar Processors (SPs), which can be either Integer SPs or Floating Point SPs to manage integer and float operations, respectively.
In a HC GPU structure, it is the host CPU system that is able to launch tasks in a device GPU system. This commonly involves data exchange (copy and paste operations) to and from the device’s global memory (normally RAM comprised by the graphics card). There is also a thread-block scheduler that distributes blocks of threads to different SMs for their execution.

3. PHC in Power Systems

Table 1 presents the review summary, specifying the classification and distribution of all the articles based on the power system application and the PHC technology. The summary also organizes the applications based on the number of references, leading to the present power flow analysis as the application where researchers have parallelized most of the power flow algorithm using heterogeneous computing. Figure 1 shows how PHC technologies have been used during the last three decades. In the beginning, CPU clusters, FPGAs, and fog and cloud computing were the most popular PHC alternatives. During the last ten years, vector processors, such as GPU, have had significant growth in power system applications since GPU represents 80% of all the articles included in the review and published during the last five years.

3.1. GPU

Figure 2 shows how GPU has been used in power system studies and applications. This technology uses the SIMD technique, and applications take advantage of this quality to vectorize parts of algorithms and studies.

3.1.1. Power Flow Analysis

As shown in Figure 2, power flow analysis is the application where GPU has been the most used ref. [6,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46]. Researchers have attempted to reduce the power flow convergence time by testing multiple numerical algorithms for the solution of linear equation systems, parallelizing the algorithm steps in the GPU. Depending on the iterative algorithm used to find the power flow solution, tasks, such as building the admittance and Jacobian matrices of large-scale power systems, have been parallelized taking advantage of the GPU architecture. In ref. [18], a multigrid-preconditioned conjugate gradient method was implemented in a GPU where the Direct Current (DC) analysis of power systems was accelerated, thus, improving the convergence rate of the conjugate gradient algorithm with the multigrid preconditioning method.
In ref. [28], the convergence rate of the conjugate gradient method was improved with a polynomial Chebyshev preconditioner integrated into a GPU-based conjugate gradient solver. In ref. [35], a fast decouple power flow algorithm integrated with the Inexact Newton method was solved with a GPU-based preconditioned conjugate gradient solver with a two-step preconditioner based on a diagonal Jacobi preconditioner and a polynomial Chebyshev preconditioner. In ref. [37], the Newton–Raphson and Gauss–Seidel power flow solvers were accelerated using a GPU where efficient data-oriented parallel primitives, such as map function, reduction, and scan, were used in different steps of both algorithms.

3.1.2. Transient Stability

Transient stability appears as the application where GPU is second-most used refs. [18,61,62,63,64,65,66,67,68,69,70,71,72,73,74]. In refs. [62,64], the nonlinear Differential-Algebraic Equations (DAEs) that represent the dynamic behavior of power systems were discretized and vectorized. Then, the slow coherency method was used to divide the system to solve the transient stability simulation in multiple processors at the same time. These modifications allow applying the SIMD computation to solve the power system DAES in parallel in a hybrid multi CPU-GPU with a parallel sparse matrix solver.

3.1.3. Electromagnetic Transient Simulation

Following transient stability, electromagnetic transient simulation is the third application where GPUs have been widely used refs. [164,165,166,167,168,169,170,171,172,173,174,175,176,177]. In ref. [168], linear passive elements, transmission lines, and electrical machines are represented by discrete-time models (lumped models, Universal Line Models (ULM), and Unified Machine Models (UMM)). These models are vectorized and linearized to be solved in parallel with a GPU.
In ref. [171], the electromagnetic simulation includes large-scale control systems dividing the algorithm into two parts: heterogeneous computing that computes control signals and a Norton equivalent circuit for non-linear electrical components and homogeneous computing for the electrical components represented by linear and time-discrete Norton equivalent circuits. The currents are computed in a GPU based on a a Layered Directed Acyclic Graph (LDAG) that sequentially links parallel-primitives and fused multiply-add that performs an add operation for the linear models. Finally, node voltages are calculated with the impedance matrix of the system.

3.1.4. Renewable Energy Integration

Renewable energy integration is the next application where GPU architecture is widely used refs. [1,178,179,180,181,182,183,184,185,186,187,188,189,190]. GPU has been used for solar potential estimation using light detection and ranging data [179] and for solving the OPF problem, including uncertainties from renewable generators [1].

3.1.5. Smart Grids

Smart grids appear as the next application where GPUs are the most used refs. [124,125,126,127,128,129,130,131,132,133,134]. The authors in ref. [125] presented a survey with the applications and trends of HPC for electric power systems where GPU emerged as one of the technologies for real-time and off-line smart grid simulation and visualization. An adaptive dispatch for smart grids was presented in ref. [131] where a wavelet recurrent neural network was implemented in a cloud-distributed GPU architecture to predict the optimal dispatch.

3.1.6. Contingency Analysis

Following smart grids, contingency analysis is the next application where GPU is used to reduce the simulation time of power flow analysis [99,100,101,102,103,104,105,106,107,108]. Depending on the simulation time reduction achieved, hybrid CPU-GPU solutions can evaluate a high number of contingencies and scenarios to find an optimal performance of power systems. The authors in ref. [102] presented a strategy to accelerate DC contingency screening where the following tasks are parallelized in a GPU: calculation of the node voltages and detection of overloaded elements after an outage of branches and generators.

3.1.7. Optimal Power Flow

As seen for contingency analysis, GPU is used for OPF to reduce the simulation time of power flow analysis [1,147,148,149,150,151,152,153,154,155]. OPF has been implemented using hybrid CPU-GPU algorithms where common instructions are parallelized using the SIMD architecture of GPUs. Metaheuristic methods and the Newton–Raphson algorithm are two of the strategies most often parallelized for OPF application. The metaheuristic methods are used to find the power system’s optimal operating point, considering the state and control variables and the optimization constraints (node voltages and generation limits).
All tasks performed in the GPU are constituted by common instructions that fit the SIMD technique. Data transfer tasks between the CPU and GPU generate a bottleneck since the communication channel’s bandwidth between both processing units is low. One of the challenges of CPU-GPU algorithms is minimizing the number of data transfer tasks to avoid bottlenecking as much as possible for the communication channel.
The authors in ref. [150,155] implemented an OPF with a metaheuristic method and the Newton–Raphson algorithm using a CPU-GPU platform. Particle Swarm Optimization (PSO) is used to optimize the total generation cost, transmission losses, and pollutant emissions of large-scale power systems. The initialization of the particle position and velocity, calculation of the fitness of all particles, the particle’s best position update, the swarm’s best position update, and the movement of all particles for all PSO iterations are parallelized in a GPU.

3.1.8. Dynamic State Estimation, Power Quality, and Dynamic Models

After OPF applications, GPU usage in dynamic state estimations [191,192,193,194,195,196,197,198,199], power quality [202,203,204,205,206,207,208,209], and dynamic models [210,211,212,213,214,215] appear. Related to the dynamic state estimation of power systems, a lateral two-level dynamic state estimator based on the extended Kalman Filter method is implemented in a CPU-GPU platform [194]. The parameter identification, state prediction, and state filtering of the extended Kalman Filter are parallelized in the GPU as well as the computation of the Jacobian matrix and the solution of the system of linear equations with the state variables to estimate.
Regarding power quality, [208] presented an optimized method for recognizing power quality disturbances based on the modified S transform and parallel stacked sparse auto-encoder. Finally, parameter identification of the dynamic models of electrical components is accelerated using GPU. The authors in ref. [211] presented an accelerated parameter identification of permanent magnet synchronous machines using a PSO algorithm parallelized in a GPU.

3.1.9. Other Applications

The rest of the applications appear with less than five references with the use of GPU has been used (electrical vehicles [217,218,219,220], probabilistic power flow [221,222,223], power system visualization [232,233], electricity market [240], small signal analysis [241], transient stability-constrained optimal power flow [229], and short circuit analysis [234]).
GPU has helped with the following tasks: to optimize the large-scale design of electric vehicles [217], to accelerate probabilistic power flow computation based on the Monte-Carlo simulation with simple random sampling [223], to visualize real-time power system contouring based on a power grid digital elevation model [233], to forecast power system demand improving the data training of an artificial neural network with a multi-layer perceptron architecture using the Levenberg–Marquardt learning method [240], to approximate the solution of large differential-algebraic equations for small-signal stability with four methods (Chebyshev discretization, time integration operator discretization, linear multistep, and Padé approximants) [241], and to speed up the short-circuit current calculation of large-scale power systems with a batch solution where the admittance matrix inverse, short-circuit current of the specified node, node voltages, and branch currents are calculated in parallel with the SIMD technique [234].
Regarding TSCOPF, ref. [229] presented the parallelization of an OPF evaluating steady-state and transient-state constraints where fuel costs were optimized. The transient-state constraints for each contingency were computed in the GPU. The rest of the algorithm ran in the CPU. For SCOPF application, no reference was found where a hybrid CPU-GPU algorithm had been implemented to reduce the execution time of either power flow simulations or the contingency evaluation.

3.2. CPU Clusters, Fog and Cloud Computing, and FPGA

On the other hand, Figure 2 shows how PHC technologies, such as CPU clusters, fog and cloud computing, and FPGA, have been used in power system studies and applications. These technologies do not use the SIMD technique since the corresponding architectures change widely from the GPU architecture.

3.2.1. Transient Stability

In contrast to GPU applications, transient stability analysis is the application where the mentioned PHC technologies are most used ref. [75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98]. In ref. [75], the very dishonest Newton method and the successive over-relaxed Newton method were implemented in parallel to perform a transient stability analysis using the local memory of the Intel supercomputer iPSC/2 and the shared memory of an Alliant FX/8 computer system. The transient stability analysis was executed once the differential machine equations are discretized using the trapezoidal rule.
In ref. [82], a parallel transient stability simulation was performed using the Interlaced Alternating Implicit (IAI) algorithm, a multilevel partition scheme to divide the power systems into subsystems and a hierarchical block bordered diagonal form algorithm to independently solve the DAES of each subsystem. Each subsystem was solved in a processor. The authors in ref. [96] performed a dynamic simulation of large-scale power systems using the Schur complement domain decomposition method based on the shared memory parallel programming model. The decomposition method helps to divide power systems into reduced subsystems to solve each subsystem as an individual problem using the shared memory of multi-core computers and OpenMP.

3.2.2. Contingency Analysis

Following transient stability analysis, contingency analysis stands as the second application where PHC technologies are the most used ref. [94,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123]. The authors in ref. [117,121] showed how the master–slave asymmetric communication model was used to analyze thousands of contingencies in large-scale power systems. The master schedules the algorithm tasks among slaves to evaluate all contingencies using proactive task scheduling and stealing methodologies to optimize the load balancing of workers (slaves). This methodology allows for the start of a contingency evaluation as soon as one work is finished without waiting for any instruction from the master. The communication between master and workers is achieved using MPI, where workers run in multi-threads in different processors.

3.2.3. Power Flow Analysis

After contingency analysis, power flow analysis appears with more PHC references [47,48,49,50,51,52,53,54,55,56,57,58,59,60]. The authors in ref. [47] shows a parallel LU factorization and substitution algorithm to solve large sparse matrix equations using the shared memory of 20 parallel multi-processor computers. The parallel factorization and substitution are used to optimize the execution time of a power flow analysis based on Newton’s and Fast Decoupled methods. In ref. [52], a parallel power flow solution based on the Newton–Raphson algorithm was presented. The Jacobian matrix was divided to parallelize the LU factorization. The proposed algorithm was implemented in a multiprocessor on a system on a programmable chip computer board containing an FPGA.

3.2.4. Smart Grids

Smart grids are the next application with PHC references [135,136,137,138,139,140,141,142,143,144,145,146]. The authors in ref. [136] presented how cloud computing can manage the smart grid data measured from front-end intelligent devices. Cloud computing fits big data applications since it provides scalability, agility, and flexibility. A data security solution was proposed based on identity-based encryption, signature, and proxy re-encryption. In ref. [138], the day-ahead energy resource scheduling of a smart grid with high penetration of distributed generation and electrical vehicles was optimized to satisfy the demand for sensitive loads.
The optimization was developed with a multi-objective model constituted by a PSO and a deterministic technique based on Mixed-Integer Linear Programming. The objective functions are the distributed generator energy production costs, the external suppliers’ energy costs, demand response program costs, the energy storage system and electrical vehicle discharging costs, the non-supplied demand costs, and the generation curtailment power costs. Each optimization problem defined in the multi-objective model was solved in an independent computer core.

3.2.5. Optimal Power Flow and Security Constrained Optimal Power Flow

The next applications with usage of PHC technologies are OPF refs. [156,157,158,159,160,161,162,163] and SCOPF refs. [225,226,227,228]. The authors in ref. [156] presented a parallel OPF solution developed in a network of workstations (CPU-cluster). The power system and the OPF problem was divided into geographical regions. Transmission lines that interconnect regions were divided into two lines connected to a dummy bus. Active and reactive power flows and voltage magnitudes and angles were defined as variables in the OPF problem for each area. The objective function for each area neglects the rest of the system and includes the cost of each generator that belongs to the corresponding area. Variables corresponding to dummy buses are modeled with dummy generators. Each OPF is defined with an interior point method. The solution is implemented in a network of seven Sun UltraSparc workstations.
In ref. [227], the decomposition of a SCOPF problem using the Bender decomposition was implemented. The Bender decomposition divides the SCOPF into one master problem (OPF) and N subproblems where a total of N contingencies are evaluated. Subproblems check the solution feasibility for the master problem. The master solution corresponds to the overall problem solution if the subproblems are feasible, and all control provided by the master problem does not violate any constraint for the post-contingency state.
Since subproblems are independent, they are solved in parallel and minimize required adjustments of preventive controls. When the subproblem is infeasible, instead of preventive control adjustments, it passes cut constraints to the master problem. The feasibility cut is added to the master problem. If there is any violation in the master problem under any contingency, the control variables for the violation are sent to the subproblems to retrieve the corresponding Bender cut.

3.2.6. Other Applications

With less than four references per application, PHC technologies have been used in dynamic state estimation [94,200,201], power system planning [236,237], TSCOPF [230,231], power system reliability [238,239], electromagnetic transient simulation [91,92], dynamic models [2,216], short circuit analysis [235], security constrained economic dispatch [242], probabilistic power flow [224], and hydro thermal scheduling [243]. In ref. [201], a power system is divided into multiple areas, and the state estimation of each area is performed individually in each area processor. Areas exchange border information through the coordinator processor (central processor). The proposed method attempts to define areas with similar sizes to balance the workload of area processors as much as possible. The state estimation is implemented in a cluster of computers. The high-performance portable implementation of MPI (MPICH2) is used to communicate the coordinator and area processors.
The researchers in ref. [236] presented the implementation of a parallel genetic algorithm to optimize the integration of different types of generation units during different time intervals. The generation planning method was implemented on a cluster of transputers. The coarse-grain version of the parallel genetic algorithm was implemented, where distributed subpopulations were optimized into several processes, and information was exchanged between subpopulations if required.
In ref. [230], the implementation of a parallel Differential Evolution (DE) algorithm to improve TSCOPF computation time was presented. The developed DE algorithm combines the time-domain simulation and the Transient Energy Function (TEF) method. The objective function corresponds to the generating fuel cost of the power system. The algorithm evaluates the power system stability under the specified contingencies where the transient event during a fault condition and an acceptable steady-state operating condition are considered.
First, the time-domain simulation calculates the generator rotor angles. Then, the TEF method computes the transient energies to determine the system stability. The DE algorithm is implemented on a Beowulf CPU-cluster with one control node and 30 working nodes connected with the MPI protocol. The initial population is divided into subpopulations. Each working node performs an individual DE algorithm to the corresponding subpopulation where the load flow calculation, fitness evaluation, transient stability assessment, and the selection are executed. The control node oversees the initialization, reproduction, and update of the global best individual.
In ref. [238], a parallel metaheuristic TSCOPF was implemented to determine the minimal investment costs to satisfy power system reliability constraints. The authors in [91] analyzed and compared the instantaneous relaxation and direct method solvers for real-time transient stability simulation and variations of nodal and state-space solvers for real-time electromagnetic transient simulations. The tests were performed in multi-core and multi-processor computers and FPGA. In ref. [216], a parallel PSO was implemented to estimate the parameters of a wide variety of photovoltaic models. The parallel PSO was implemented in OpenCL. The evaluation of all particles was performed at the same time in multi-processor devices.
In ref. [235], the probability density curves of short-circuit levels were obtained based on running a Monte-Carlo simulation in parallel in multiple virtual machines. Regarding Security Constrained-Economic Dispatch, ref. [242] presented a multithread solution of a power system economic dispatch based on the Multi-Thread Interior Point Barrier algorithm considering the generator limitations, transmission losses, and nonlinear cost functions. The algorithm validates that the optimal dispatch satisfies line flow limits adding line flow constraints as security constraints.
The authors in ref. [224] presented an implementation of real-time probabilistic power flow based on a parallel Monte-Carlo simulation run in multicore CPUs. The method allows for the evaluation of uncertainties related to renewable energy resources. Finally, in ref. [243], a parallel differential evolution algorithm was implemented to optimize the short-term schedule of a hydrothermal generator unit considering power flow constraints. The method divided a large population into subpopulations where each subpopulation searched for the optimal solution individually in an exclusive processor.

4. Conclusions

In this paper, we presented a review of PHC techniques in power systems applications, such as power flow analysis, transient stability, contingency analysis, and smart grids, among others. The review covered more than 200 works where technologies, such as CPU clusters, FPGA, hybrid CPU-GPU platforms running on on-premise and cloud environments, have been used to improve modern power system planning, operation, studies, visualization, and analysis.
Power flow analysis is the application where PHC techniques have been most used, followed by transient stability and contingency analysis. Regarding power flow analysis, different technologies have been adopted to accelerate the tasks of the power flow algorithm, such as building the admittance matrix, or—regarding the Newton–Raphson method—computing the Jacobian matrix and updating the state variables. On the other hand, applications, such as electricity market analysis, small-signal analysis, security constrained-economic dispatch, and hydrothermal scheduling, are the situations where PHC techniques are not frequently used since they are the applications with the lowest number of references.
The review is organized by application and two groups of technologies: the first group corresponds to works that use GPU, and the second group refers to works where CPU clusters, FPGA, and fog and cloud computing are used. This classification shows that GPU is the most-used PHC technique in power system studies and analysis since works that use this technology are more than half of all the articles presented by the review. GPU stands as the current tendency since 80% of the articles published during the last five years correspond to GPU implementations.
This tendency shows that researchers are taking advantage of the GPU architecture designed to exploit data parallelism using the SIMD parallel computing approach. The development of toolkits and frameworks to program GPU instructions in non-graphic programming languages has helped to expand the role of the GPU in power system applications. Finally, the wide number of applications that have used PHC techniques present PHC as a proper alternative for studying and analyzing modern power systems, due to the flexibility and adaptability of these techniques.

5. Future Work

As presented in the review, one of the most important characteristics of PHC techniques is the flexibility and adaptability of the technologies to a wide variety of applications. However, PHC techniques are not exclusive to the technologies presented in this article (CPU, FPGA, and GPU in on-premise and cloud environments). This provides the possibility to adopt new technologies, such as quantum computing, which continue to arise and make their way into applied sciences and engineering applications, into PHC implementations.
Regarding parallel computing, the fact that vector processors can compute on several data streams with a single instruction represents a significant advantage over scalar processors. Consequently, technologies, like GPUs, which is the current tendency, represent the future trend for parallel computing. As shown in the review, GPUs can interact properly with CPUs, and thus hybrid CPU-GPU platforms bring significant advantages that have begun to help improve power system operation.
Regarding infrastructure cost and maintenance and considering cloud computing advantages, combining new computing tendencies, such as quantum computing and advances in vector processors (improvement in GPUs), in cloud environments allow researches to use a wide number of resources to not only improve the power system performance but also to find further applications where PHC can be used to grant better results.

Author Contributions

Conceptualization, D.R.; Data curation, D.R., D.G., D.A. and S.R.; Formal analysis, D.R., D.G. and D.A.; Funding acquisition, S.R.; Investigation, D.R. and S.R.; Methodology, D.R., D.G., D.A. and S.R.; Writing—original draft, D.R., D.G. and D.A.; Writing—review & editing, D.R. and S.R. All authors have read and agreed to the published version of the manuscript.

Funding

This research was supported by Universidad Nacional de Colombia, and by GERS USA.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Huang, S.; Dinavahi, V. Fast Batched Solution for Real-Time Optimal Power Flow With Penetration of Renewable Energy. IEEE Access 2018, 6, 13898–13910. [Google Scholar] [CrossRef]
  2. Mojlish, S.; Erdogan, N.; Levine, D.; Davoudi, A. Review of Hardware Platforms for Real-Time Simulation of Electric Machines. IEEE Trans. Transp. Electrif. 2017, 3, 130–146. [Google Scholar] [CrossRef]
  3. Marin, M. GPU-Enhanced Power Flow Analysis. Ph.D. Thesis, Universite de Perpignan Via Domitia-University College Dublin, Dublin, Ireland, 2016. [Google Scholar]
  4. Li, Y.; Zhang, Z. Parallel Computing: Review and Perspective. In Proceedings of the 2018 5th International Conference on Information Science and Control Engineering, ICISCE 2018, Zhengzhou, China, 20–22 July 2018; pp. 365–369. [Google Scholar] [CrossRef]
  5. Kyaw, L.Y.; Phyu, S. Scheduling Methods in HPC System: Review. In Proceedings of the 2020 IEEE Conference on Computer Applications, ICCA 2020, Yangon, Myanmar, 27–28 February 2020; pp. 1–6. [Google Scholar] [CrossRef]
  6. Yoon, D.H.; Han, Y. Parallel power flow computation trends and applications: A review focusing on GPU. Energies 2020, 13, 2147. [Google Scholar] [CrossRef]
  7. Duan, P.; Xu, S.; Chen, H.; Yang, X.; Wang, S.; Hu, E. High Performance Computing (HPC)for Advanced Power System Studies. In Proceedings of the 2nd IEEE Conference on Energy Internet and Energy System Integration, EI2 2018-Proceedings, Beijing, China, 20–22 October 2018; pp. 1–9. [Google Scholar] [CrossRef]
  8. Thijssen, J. Computational Physics; Cambridge University Press: Cambridge, UK, 2007. [Google Scholar]
  9. Marksteiner, P. High-performance computing—An overview. Comput. Phys. Commun. 1996, 97, 16–35. [Google Scholar] [CrossRef]
  10. Andrade, H.; Crnkovic, I. A Review on Software Architectures for Heterogeneous Platforms. In Proceedings of the Asia-Pacific Software Engineering Conference, Nara, Japan, 4–7 December 2018; pp. 209–218. [Google Scholar] [CrossRef] [Green Version]
  11. Glaskowsky, P.N. NVIDIA’s Fermi: The First Complete GPU Computing Architecture. Available online: https://slideplayer.com/slide/8599949/ (accessed on 23 September 2021).
  12. Page, D. A Practical Introduction to Computer Architecture; Springer: Berlin/Heidelberg, Germany, 2009; p. 823. [Google Scholar] [CrossRef]
  13. Feng, Z.; Li, P. Multigrid on GPU: Tackling Power Grid Analysis on parallel SIMT platforms. In Proceedings of the 2008 IEEE/ACM International Conference on Computer-Aided Design, San Jose, CA, USA, 10–13 November 2008; pp. 647–654. [Google Scholar] [CrossRef]
  14. Shi, J.; Cai, Y.; Hou, W.; Ma, L.; Tan, S.X.D.; Ho, P.H.; Wang, X. GPU friendly Fast Poisson Solver for structured power grid network analysis. In Proceedings of the 2009 46th ACM/IEEE Design Automation Conference, San Francisco, CA, USA, 26–31 July 2009; pp. 178–183. [Google Scholar]
  15. Dzafic, I.; Neisius, H.-T. Real-time power flow algorithm for shared memory multiprocessors for European distribution network types. In Proceedings of the 2010 Conference Proceedings IPEC, Singapore, 27–29 October 2010; pp. 152–158. [Google Scholar] [CrossRef]
  16. Garcia, N. Parallel power flow solutions using a biconjugate gradient algorithm and a Newton method: A GPU-based approach. In Proceedings of the IEEE PES General Meeting, Minneapolis, MN, USA, 25–29 July 2010; pp. 1–4. [Google Scholar] [CrossRef]
  17. Singh, J.; Aruni, I. Accelerating Power Flow studies on Graphics Processing Unit. In Proceedings of the 2010 Annual IEEE India Conference (INDICON), Kolkata, India, 17–19 December 2010; pp. 1–5. [Google Scholar] [CrossRef]
  18. Feng, Z.; Zhao, X.; Zeng, Z. Robust Parallel Preconditioned Power Grid Simulation on GPU With Adaptive Runtime Performance Modeling and Optimization. IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst. 2011, 30, 562–573. [Google Scholar] [CrossRef]
  19. Li, Z.; Donde, V.D.; Tournier, J.-C.; Yang, F. On limitations of traditional multi-core and potential of many-core processing architectures for sparse linear solvers used in large-scale power system applications. In Proceedings of the 2011 IEEE Power and Energy Society General Meeting, Detroit, MI, USA, 24–28 July 2011; pp. 1–8. [Google Scholar] [CrossRef]
  20. Tournier, J.C.; Donde, V.; Li, Z. Comparison of direct and iterative sparse linear solvers for power system applications on parallel computing platforms. In Proceedings of the 17th Power Systems Computation Conference, PSCC 2011, Stockholm, Sweden, 22–26 August 2011. [Google Scholar]
  21. Vilachá, C.; Moreira, J.C.; Míguez, E.; Otero, A.F. Massive Jacobi power flow based on SIMD-processor. In Proceedings of the 2011 10th International Conference on Environment and Electrical Engineering, Rome, Italy, 8–11 May 2011; pp. 1–4. [Google Scholar] [CrossRef] [Green Version]
  22. Ablakovic, D.; Dzafic, I.; Kecici, S. Parallelization of radial three-phase distribution power flow using GPU. In Proceedings of the 2012 3rd IEEE PES Innovative Smart Grid Technologies Europe (ISGT Europe), Berlin, Germany, 14–17 October 2012; pp. 1–7. [Google Scholar] [CrossRef]
  23. Guo, C.; Jiang, B.; Yuan, H.; Yang, Z.; Wang, L.; Ren, S. Performance Comparisons of Parallel Power Flow Solvers on GPU System. In Proceedings of the 2012 IEEE International Conference on Embedded and Real-Time Computing Systems and Applications, Seoul, Korea, 19–22 August 2012; pp. 232–239. [Google Scholar] [CrossRef]
  24. Wigington, A.; Min, L.; Li, C.; Murray, W.; Narayan, A. Advancing the adoption of advanced computing methods and technologies for real-time control center operations. In Proceedings of the 2012 IEEE Power and Energy Society General Meeting, San Diego, CA, USA, 22–26 July 2012; pp. 1–6. [Google Scholar] [CrossRef]
  25. Yang, M.; Sun, C.; Li, Z.; Cao, D. An improved sparse matrix-vector multiplication kernel for solving modified equation in large scale power flow calculation on CUDA. In Proceedings of the 7th International Power Electronics and Motion Control Conference, Harbin, China, 2–5 June 2012; Volume 3, pp. 2028–2031. [Google Scholar] [CrossRef]
  26. Kamiabad, A.A.; Tate, J.E. Polynomial Preconditioning of Power System Matrices with Graphics Processing Units. In High Performance Computing in Power and Energy Systems; Springer: Berlin/Heidelberg, Germany, 2013; pp. 229–246. [Google Scholar] [CrossRef]
  27. Li, X.; Li, F.; Clark, J.M. Exploration of multifrontal method with GPU in power flow computation. In Proceedings of the 2013 IEEE Power Energy Society General Meeting, Vancouver, BC, Canada, 21–25 July 2013; pp. 1–5. [Google Scholar] [CrossRef]
  28. Li, X.; Li, F. GPU-based power flow analysis with Chebyshev preconditioner and conjugate gradient method. Electr. Power Syst. Res. 2014, 116, 87–93. [Google Scholar] [CrossRef]
  29. Blaskiewicz, P.; Zawada, M.; Balcerek, P.; Dawidowski, P. An Application of GPU Parallel Computing to Power Flow Calculation in HVDC Networks. In Proceedings of the 2015 23rd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing, Turku, Finland, 4–6 March 2015; pp. 635–641. [Google Scholar] [CrossRef]
  30. Li, X.; Li, F. GPU-based two-step preconditioning for conjugate gradient method in power flow. In Proceedings of the 2015 IEEE Power Energy Society General Meeting, Denver, CO, USA, 26–30 July 2015; pp. 1–5. [Google Scholar] [CrossRef]
  31. Magaña-Lemus, E.; Medina, A.; Ramos-Paz, A. Periodic steady state solution of power systems by selective transition matrix identification, LU decomposition and graphic processing units. In Proceedings of the 2015 IEEE Power Energy Society General Meeting, Denver, CO, USA, 26–30 July 2015; pp. 1–5. [Google Scholar] [CrossRef]
  32. Jiang, H.; Chen, D.; Li, Y.; Zheng, R. A Fine-Grained Parallel Power Flow Method for Large Scale Grid Based on Lightweight GPU Threads. In Proceedings of the 2016 IEEE 22nd International Conference on Parallel and Distributed Systems (ICPADS), Wuhan, China, 13–16 December 2016; pp. 785–790. [Google Scholar] [CrossRef]
  33. Araujo, M.H.D.; Rios, A.M.; Lemus, E.M. Periodic Steady State of Power Networks Using Limit Cycle Extrapolation, Spline Interpolation and Parallel Processing Based on GPUs. In Proceedings of the 2017 International Conference on Computational Science and Computational Intelligence (CSCI), Las Vegas, NV, USA, 14–16 December 2017; pp. 1606–1611. [Google Scholar] [CrossRef]
  34. Huang, S.; Dinavahi, V. Performance analysis of GPU-accelerated fast decoupled power flow using direct linear solver. In Proceedings of the 2017 IEEE Electrical Power and Energy Conference (EPEC), Saskatoon, SK, Canada, 22–25 October 2017; pp. 1–6. [Google Scholar] [CrossRef]
  35. Li, X.; Li, F.; Yuan, H.; Cui, H.; Hu, Q. GPU-Based Fast Decoupled Power Flow With Preconditioned Iterative Solver and Inexact Newton Method. IEEE Trans. Power Syst. 2017, 32, 2695–2703. [Google Scholar] [CrossRef]
  36. Marin, M.; Defour, D.; Milano, F. Asynchronous Power Flow on Graphic Processing Units. In Proceedings of the 2017 25th Euromicro International Conference on Parallel, Distributed and Network-based Processing (PDP), St. Petersburg, Russia, 6–8 March 2017; pp. 255–261. [Google Scholar] [CrossRef] [Green Version]
  37. Roberge, V.; Tarbouchi, M.; Okou, F. Parallel Power Flow on Graphics Processing Units for Concurrent Evaluation of Many Networks. IEEE Trans. Smart Grid 2017, 8, 1639–1648. [Google Scholar] [CrossRef]
  38. Wang, M.; Xia, Y.; Chen, Y.; Huang, S. GPU-based power flow analysis with continuous Newton’s method. In Proceedings of the 2017 IEEE Conference on Energy Internet and Energy System Integration (EI2), Beijing, China, 26–28 November 2017; pp. 1–5. [Google Scholar] [CrossRef]
  39. Zhou, G.; Bo, R.; Chien, L.; Zhang, X.; Shi, F.; Xu, C.; Feng, Y. GPU-Based Batch LU-Factorization Solver for Concurrent Analysis of Massive Power Flows. IEEE Trans. Power Syst. 2017, 32, 4975–4977. [Google Scholar] [CrossRef]
  40. Liu, Z.; Song, Y.; Chen, Y.; Huang, S.; Wang, M. Batched Fast Decoupled Load Flow for Large-Scale Power System on GPU. In Proceedings of the 2018 International Conference on Power System Technology (POWERCON), Guangzhou, China, 6–8 November 2018; pp. 1775–1780. [Google Scholar] [CrossRef]
  41. Gnanavignesh, R.; Shenoy, U.J. GPU-Accelerated Sparse LU Factorization for Power System Simulation. In Proceedings of the 2019 IEEE PES Innovative Smart Grid Technologies Europe (ISGT-Europe), 29 September–2 October 2019; pp. 1–5. [Google Scholar] [CrossRef]
  42. Gnanavignesh, R.; Shenoy, U.J. Parallel Sparse LU Factorization of Power Flow Jacobian using GPU. In Proceedings of the TENCON 2019-2019 IEEE Region 10 Conference (TENCON), Kochi, India, 17–20 October 2019; pp. 1857–1862. [Google Scholar] [CrossRef]
  43. Qi, S.; Li, G.; Bie, Z. Hybrid Energy Flow Calculation for Electric-Thermal Coupling System Based on Inexact Newton Method. In Proceedings of the 2019 IEEE Sustainable Power and Energy Conference (iSPEC), Beijing, China, 21–23 November 2019; pp. 2443–2449. [Google Scholar] [CrossRef]
  44. Tang, K.; Fang, R.; Wang, X.; Dong, S.; Song, Y. Mass Expression Evaluation Parallel Algorithm Based on ‘Expression Forest’ and Its Application in Power System Calculation. In Proceedings of the 2019 IEEE Power Energy Society General Meeting (PESGM), Atlanta, GA, USA, 4–8 August 2019; pp. 1–5. [Google Scholar] [CrossRef]
  45. Zhou, G.; Feng, Y.; Bo, R.; Zhang, T. GPU-accelerated sparse matrices parallel inversion algorithm for large-scale power systems. Int. J. Electr. Power Energy Syst. 2019, 111, 34. [Google Scholar] [CrossRef]
  46. Su, X.; He, C.; Liu, T.; Wu, L. Full Parallel Power Flow Solution: A GPU-CPU-Based Vectorization Parallelization and Sparse Techniques for Newton–Raphson Implementation. IEEE Trans. Smart Grid 2020, 11, 1833–1844. [Google Scholar] [CrossRef]
  47. Wu, J.Q.; Bose, A. Parallel solution of large sparse matrix equations and parallel power flow. IEEE Trans. Power Syst. 1995, 10, 1343–1349. [Google Scholar] [CrossRef]
  48. Feng, T.; Flueck, A.J. A message-passing distributed-memory Newton-GMRES parallel power flow algorithm. In Proceedings of the IEEE Power Engineering Society Summer Meeting, Chicago, IL, USA, 21–25 July 2002; Volume 3, pp. 1477–1482. [Google Scholar] [CrossRef]
  49. Foertsch, J.; Johnson, J.; Nagvajara, P. Jacobi load flow accelerator using FPGA. In Proceedings of the 37th Annual North American Power Symposium, Ames, IA, USA, 25–25 October 2005; pp. 448–454. [Google Scholar] [CrossRef]
  50. Johnson, J.; Vachranukunkiet, P.; Tiwari, S.; Nagvajara, P.; Nwankpa, C. Performance analysis of load flow computation using FPGA. In Proceedings of the 15th Power Systems Computation Conference, Liège, Belgium, 22–26 August 2005. [Google Scholar]
  51. Yang, F.; He, M.; Tang, Y.; Rao, M. Inexact block Newton methods for solving nonlinear equations. Appl. Math. Comput. 2005, 162, 1207–1218. [Google Scholar] [CrossRef]
  52. Wang, X.; Ziavras, S.G.; Nwankpa, C.; Johnson, J.; Nagvajara, P. Parallel solution of Newton’s power flow equations on configurable chips. Int. J. Electr. Power Energy Syst. 2007, 29, 422–431. [Google Scholar] [CrossRef]
  53. Li, Y.; Li, F.; Li, W. Parallel power flow calculation based on multi-port inversed matrix method. In Proceedings of the 2010 International Conference on Power System Technology: Technological Innovations Making Power Grid Smarter, Hangzhou, China, 24–28 October 2010. [Google Scholar] [CrossRef]
  54. Nechma, T.; Zwoliński, M.; Reeve, J. Parallel sparse matrix solver for direct circuit simulations on FPGAs. In Proceedings of the 2010 IEEE International Symposium on Circuits and Systems (ISCAS), Paris, France, 30 May–2 June 2010; pp. 2358–2361. [Google Scholar] [CrossRef]
  55. Cui, T.; Franchetti, F. A multi-core high performance computing framework for distribution power flow. In Proceedings of the 2011 North American Power Symposium, Boston, TX, USA, 14–16 November 2011; pp. 1–5. [Google Scholar] [CrossRef] [Green Version]
  56. Dağ, H.; Soykan, G. Power flow using thread programming. In Proceedings of the 2011 IEEE Trondheim PowerTech, Trondheim, Norway, 19–23 June 2011; pp. 1–5. [Google Scholar] [CrossRef]
  57. Cui, T.; Franchetti, F. Optimized parallel distribution load flow solver on commodity multi-core CPU. In Proceedings of the 2012 IEEE Conference on High Performance Extreme Computing, Waltham, MA, USA, 10–12 September 2012; pp. 1–6. [Google Scholar] [CrossRef]
  58. Ge, X.; Zhu, H.; Yang, F.; Wang, L.; Zeng, X. Parallel sparse LU decomposition using FPGA with an efficient cache architecture. In Proceedings of the 2017 IEEE 12th International Conference on ASIC (ASICON), Guiyang, China, 25–28 October 2017; pp. 259–262. [Google Scholar] [CrossRef]
  59. Ahmadi, A.; Jin, S.; Smith, M.C.; Collins, E.R.; Goudarzi, A. Parallel Power Flow based on OpenMP. In Proceedings of the 2018 North American Power Symposium (NAPS), Fargo, ND, USA, 9–11 September 2018; pp. 1–6. [Google Scholar] [CrossRef]
  60. Yoon, D.H.; Kang, S.K.; Kim, M.; Han, Y. Exploiting coarse-grained parallelism using cloud computing in massive power flow computation. Energies 2018, 11, 2268. [Google Scholar] [CrossRef] [Green Version]
  61. Jalili-Marandi, V.; Dinavahi, V. Large-scale transient stability simulation on graphics processing units. In Proceedings of the 2009 IEEE Power Energy Society General Meeting, Calgary, AB, Canada, 26–30 July 2009; pp. 1–6. [Google Scholar] [CrossRef] [Green Version]
  62. Jalili-Marandi, V.; Dinavahi, V. SIMD-Based Large-Scale Transient Stability Simulation on the Graphics Processing Unit. IEEE Trans. Power Syst. 2010, 25, 1589–1599. [Google Scholar] [CrossRef]
  63. Baijian, W.; Guo, W.X.; Hu, J.Y.; Wang, F.Z.; Ye, J. GPU based parallel simulation of transient stability using symplectic Gauss algorithm and preconditioned GMRES method. In Proceedings of the 2012 Power Engineering and Automation Conference, Wuhan, China, 18–20 September 2012; pp. 1–5. [Google Scholar] [CrossRef]
  64. Jalili-Marandi, V.; Zhou, Z.; Dinavahi, V. Large-Scale Transient Stability Simulation of Electrical Power Systems on Parallel GPUs. IEEE Trans. Parallel Distrib. Syst. 2012, 23, 1255–1266. [Google Scholar] [CrossRef]
  65. Qin, Z.; Hou, Y. A GPU-Based Transient Stability Simulation Using Runge-Kutta Integration Algorithm. Int. J. Smart Grid Clean Energy 2013, 2, 32–39. [Google Scholar] [CrossRef]
  66. Yu, Z.; Huang, S.; Shi, L.; Chen, Y. GPU-based JFNG method for power system transient dynamic simulation. In Proceedings of the 2014 International Conference on Power System Technology, Chengdu, China, 20–22 October 2014; pp. 969–975. [Google Scholar] [CrossRef]
  67. He, K.; Tan, S.X.; Zhao, H.; Liu, X.X.; Wang, H.; Shi, G. Parallel GMRES solver for fast analysis of large linear dynamic systems on GPU platforms. Integration 2016, 52, 10–22. [Google Scholar] [CrossRef] [Green Version]
  68. Liao, X.; Wang, F. Parallel computation of transient stability using symplectic Gauss method and GPU. IET Gener. Transm. Distrib. 2016, 10, 3727–3735. [Google Scholar] [CrossRef]
  69. Yu, Z.; Chen, Y.; Song, Y.; Huang, S. Comparison of parallel implementations of controls on GPU for transient simulation of power system. In Proceedings of the 2016 35th Chinese Control Conference (CCC), Chengdu, China, 27–29 July 2016; pp. 9996–10001. [Google Scholar] [CrossRef]
  70. Liu, K.; Liao, X.; Li, Y. Parallel Simulation of Power Systems Transient Stability Based on Implicit Runge–Kutta Methods and W-transformation. Electr. Power Compon. Syst. 2017, 45, 2246–2256. [Google Scholar] [CrossRef]
  71. Vasquez, A.D.; Sousa, T. A Parallel Processing Approach to Stability Analysis Considering Transmission and Distribution Systems. In Proceedings of the 2019 IEEE Milan PowerTech, Milan, Italy, 23–27 June 2019; pp. 1–6. [Google Scholar] [CrossRef]
  72. Wang, Z.; Chakrabortty, A.; Kelley, C.T.; Feng, X.; Franzon, P. Improved Numerical Methodologies on Power System Dynamic Simulation Using GPU Implementation. In Proceedings of the 2019 IEEE Power Energy Society Innovative Smart Grid Technologies Conference (ISGT), Washington, DC, USA, 18–21 February 2019; pp. 1–5. [Google Scholar] [CrossRef]
  73. Zhou, Y.; Guo, Q.; Sun, H.; Yu, Z.; Wu, J.; Hao, L. A novel data-driven approach for transient stability prediction of power systems considering the operational variability. Int. J. Electr. Power Energy Syst. 2019, 107, 379–394. [Google Scholar] [CrossRef]
  74. Yang, H.; Qiu, R.C.; Shi, X.; He, X. Unsupervised feature learning for online voltage stability evaluation and monitoring based on variational autoencoder. Electr. Power Syst. Res. 2020, 182, 106253. [Google Scholar] [CrossRef] [Green Version]
  75. Chai, J.S.; Zhu, N.; Bose, A.; Tylavsky, D.J. Parallel Newton type methods for power system stability analysis using local and shared memory multiprocessors. IEEE Trans. Power Syst. 1991, 6, 1539–1545. [Google Scholar] [CrossRef]
  76. Ten Bruggencate, M.; Chalasani, S. Parallel Implementations of the Power System Transient Stability Problem on Clusters of Workstations. In Proceedings of the 1995 ACM/IEEE Conference on Supercomputing, San Diego, CA, USA, 3–8 December 1995; p. 34. [Google Scholar] [CrossRef]
  77. La Scala, M.; Sblendorio, G.; Bose, A.; Wu, J.Q. Comparison of algorithms for transient stability simulations on shared and distributed memory multiprocessors. IEEE Trans. Power Syst. 1996, 11, 2045–2050. [Google Scholar] [CrossRef]
  78. Aloisio, G.; Bochicchio, M.A.; La Scala, M.; Sbrizzai, R. A distributed computing approach for real-time transient stability analysis. IEEE Trans. Power Syst. 1997, 12, 981–987. [Google Scholar] [CrossRef]
  79. Hong, C.; Shen, C.M. Parallel transient stability analysis on distributed memory message passing multiprocessors. In Proceedings of the 1997 Fourth International Conference on Advances in Power System Control, Operation and Management, Hong Kong, China, 11–14 November 1997; Volume 1, pp. 304–309. [Google Scholar] [CrossRef]
  80. Hong, C.; Shen, C.M. Implementation of parallel algorithms for transient stability analysis on a message passing multicomputer. In Proceedings of the 2000 IEEE Power Engineering Society Winter Meeting, Singapore, 23–27 January 2000; Volume 2, pp. 1410–1415. [Google Scholar] [CrossRef] [Green Version]
  81. Lu, J.; Yu, L.; Zhu, Y.L. Application in Electric Power System Transient Stability Analysis with PC Cluster System. In Proceedings of the 2005 IEEE/PES Transmission Distribution Conference Exposition: Asia and Pacific, Dalian, China, 18 August 2005; pp. 1–3. [Google Scholar] [CrossRef]
  82. Shu, J.; Xue, W.; Zheng, W. A parallel transient stability simulation for power systems. IEEE Trans. Power Syst. 2005, 20, 1709–1717. [Google Scholar] [CrossRef]
  83. Wei, X.; Shu, J.; Wu, Y.; Zheng, W. Parallel algorithm and implementation for realtime dynamic simulation of power system. In Proceedings of the 2005 International Conference on Parallel Processing (ICPP’05), Oslo, Norway, 14–17 June 2005; pp. 137–144. [Google Scholar] [CrossRef]
  84. Xue, W.; Qi, S. Multilevel Task Partition Algorithm for Parallel Simulation of Power System Dynamics. In Computational Science–ICCS 2007; Springer: Berlin/Heidelberg, Germany, 2007; pp. 529–537. [Google Scholar]
  85. Ye, J.; Liu, Z.; Zhu, L. The Implementation of a Spatial Parallel Algorithm for Transient Stability Simulation on PC Cluster. In Proceedings of the 2007 2nd IEEE Conference on Industrial Electronics and Applications, Harbin, China, 23–25 May 2007; pp. 1489–1492. [Google Scholar] [CrossRef]
  86. Lin, J.; Tong, X.Y.; Wang, X.D.; Wang, W.C. Parallel simulation for the transient stability of power system. In Proceedings of the 2008 Third International Conference on Electric Utility Deregulation and Restructuring and Power Technologies, Nanjing, China, 6–9 April 2008; pp. 1325–1329. [Google Scholar] [CrossRef]
  87. Wang, X.Z.; Yan, Z.; Xue, W. An adaptive clustering algorithm with high performance computing application to power system transient stability simulation. In Proceedings of the 2008 Third International Conference on Electric Utility Deregulation and Restructuring and Power Technologies, Nanjing, China, 6–9 April 2008; pp. 1137–1140. [Google Scholar] [CrossRef]
  88. Lin, J.; Wang, X.D.; Tong, X.Y. Asynchronous parallel simulation of transient stability based on equivalence. In Proceedings of the 2009 International Conference on Sustainable Power Generation and Supply, Nanjing, China, 6–7 April 2009; pp. 1–5. [Google Scholar] [CrossRef]
  89. Meng, K.; Dong, Z.Y.; Wong, K.P.; Xu, Y.; Luo, F.J. Speed-up the computing efficiency of power system simulator for engineering-based power system transient stability simulations. IET Gener. Transm. Distrib. 2010, 4, 652–661. [Google Scholar] [CrossRef]
  90. Robbins, B.A.; Zavala, V.M. Convergence Analysis of a Parallel Newton Scheme for Dynamic Power Grid Simulations. In Proceedings of the First International Workshop on High Performance Computing, Networking and Analytics for the Power Grid, HiPCNA-PG ’11, Seattle, WA, USA, 13 November 2011; Association for Computing Machinery: New York, NY, USA, 2011; pp. 3–10. [Google Scholar] [CrossRef]
  91. Dufour, C.; Jalili-Marandi, V.; Bélanger, J.; Snider, L. Power system simulation algorithms for parallel computer architectures. In Proceedings of the 2012 IEEE Power and Energy Society General Meeting, San Diego, CA, USA, 22–26 July 2012; pp. 1–6. [Google Scholar] [CrossRef]
  92. Dufour, C.; Jalili-Marandi, V.; Belanger, J. Real-Time Simulation Using Transient Stability, ElectroMagnetic Transient and FPGA-Based High-Resolution Solvers. In Proceedings of the 2012 SC Companion: High Performance Computing, Networking Storage and Analysis, Salt Lake City, UT, USA, 10–16 November 2012; pp. 283–288. [Google Scholar] [CrossRef]
  93. Jiang, Q.; Jiang, H. OpenMP-based parallel transient stability simulation for large-scale power systems. Sci. China Technol. Sci. 2012, 55, 2837–2846. [Google Scholar] [CrossRef]
  94. Huang, Z.; Chen, Y.; Chavarría-Miranda, D. High-Performance Computing for Real-Time Grid Analysis and Operation. In High Performance Computing in Power and Energy Systems; Springer: Berlin/Heidelberg, Germany, 2013; pp. 151–188. [Google Scholar] [CrossRef]
  95. Khaitan, S.K.; McCalley, J.D. High Performance Computing for Power System Dynamic Simulation. In High Performance Computing in Power and Energy Systems; Springer: Berlin/Heidelberg, Germany, 2013; pp. 43–69. [Google Scholar] [CrossRef]
  96. Aristidou, P.; Fabozzi, D.; Van Cutsem, T. Dynamic Simulation of Large-Scale Power Systems Using a Parallel Schur-Complement-Based Decomposition Method. IEEE Trans. Parallel Distrib. Syst. 2014, 25, 2561–2570. [Google Scholar] [CrossRef] [Green Version]
  97. Jin, S.; Huang, Z.; Diao, R.; Wu, D.; Chen, Y. Comparative Implementation of High Performance Computing for Power System Dynamic Simulations. IEEE Trans. Smart Grid 2017, 8, 1387–1395. [Google Scholar] [CrossRef]
  98. Sullivan, B.; Shi, J.; Mazzola, M.; Saravi, B. Faster-than-real-time power system transient stability simulation using Parallel General Norton with Multiport Equivalent (PGNME). In Proceedings of the 2017 IEEE Power Energy Society General Meeting, Chicago, IL, USA, 16–20 July 2017; pp. 1–5. [Google Scholar] [CrossRef]
  99. Gopal, A.; Dagmar, N.; Suresh, V. DC Power Flow Based Contingency Analysis Using Graphics Processing Units. In Proceedings of the 2007 IEEE Lausanne Power Tech, Lausanne, Switzerland, 1–5 July 2007; pp. 731–736. [Google Scholar] [CrossRef]
  100. Chen, Y.; Jin, H.; Jiang, H.; Xu, D.; Zheng, R.; Liu, H. GPU-based Static State Security Analysis in Power Systems. In Advances in Services Computing; Springer International Publishing: Cham, Switzerland, 2015; pp. 258–267. [Google Scholar]
  101. Li, X.; Balasubramanian, P.; Sahraei-Ardakani, M.; Abdi-Khorsand, M.; Hedman, K.W.; Podmore, R. Real-Time Contingency Analysis With Corrective Transmission Switching. IEEE Trans. Power Syst. 2017, 32, 2604–2617. [Google Scholar] [CrossRef]
  102. Zhou, G.; Zhang, X.; Lang, Y.; Bo, R.; Jia, Y.; Lin, J.; Feng, Y. A novel GPU-accelerated strategy for contingency screening of static security analysis. Int. J. Electr. Power Energy Syst. 2016, 83, 33–39. [Google Scholar] [CrossRef]
  103. Chen, D.; Jiang, H.; Li, Y.; Xu, D. A Two-Layered Parallel Static Security Assessment for Large-Scale Grids Based on GPU. IEEE Trans. Smart Grid 2017, 8, 1396–1405. [Google Scholar] [CrossRef]
  104. Zhou, G.; Feng, Y.; Bo, R.; Chien, L.; Zhang, X.; Lang, Y.; Jia, Y.; Chen, Z. GPU-Accelerated Batch-ACPF Solution for N-1 Static Security Analysis. IEEE Trans. Smart Grid 2017, 8, 1406–1416. [Google Scholar] [CrossRef]
  105. Huang, S.; Dinavahi, V. Real-Time Contingency Analysis on Massively Parallel Architectures With Compensation Method. IEEE Access 2018, 6, 44519–44530. [Google Scholar] [CrossRef]
  106. Sommer, S.; Aabrandt, A.; Jóhannsson, H. Reduce–factor–solve for fast Thevenin impedance computation and network reduction. IET Gener. Transm. Distrib. 2019, 13, 288–295. [Google Scholar] [CrossRef] [Green Version]
  107. Tang, K.; Dong, S.; Zhu, B.; Ni, Q.; Song, Y. GPU-Based Real-time N-1 AC Power Flow Algorithm With Preconditioned Iterative Method. In Proceedings of the 2018 IEEE Power Energy Society General Meeting (PESGM), Portland, OR, USA, 5–10 August 2018; pp. 1–5. [Google Scholar] [CrossRef]
  108. Zhang, C.; Xu, Y.; Chen, Y.; Liu, Z.; Liu, S.; Su, D.; Wu, H. Batch Computing Method for Sensitivity Analysis of Large Power Grids Based on GPU Acceleration. In Proceedings of the 2019 IEEE 3rd International Electrical and Energy Conference (CIEEC), Beijing, China, 7–9 September 2019; pp. 1792–1797. [Google Scholar] [CrossRef]
  109. Balduino, L.; Alves, A.C.B. Parallel processing in a cluster of microcomputers with application in contingency analysis. In Proceedings of the 2004 IEEE/PES Transmision and Distribution Conference and Exposition: Latin America (IEEE Cat. No. 04EX956), Sao Paulo, Brazil, 8–11 November 2004; pp. 285–290. [Google Scholar] [CrossRef]
  110. Angeline Ezhilarasi, G.; Swarup, K.S. Parallel contingency analysis in a high performance computing environment. In Proceedings of the 2009 International Conference on Power Systems, Kharagpur, India, 27–29 December 2009; pp. 1–6. [Google Scholar] [CrossRef]
  111. Gao, W.; Chen, X. Distributed Generation Placement Design and Contingency Analysis with Parallel Computing Technology. J. Comput. 2009, 4, 347–354. [Google Scholar] [CrossRef]
  112. Huang, Z.; Chen, Y.; Nieplocha, J. Massive contingency analysis with high performance computing. In Proceedings of the 2009 IEEE Power Energy Society General Meeting, Calgary, AB, Canada, 26–30 July 2009; pp. 1–8. [Google Scholar] [CrossRef]
  113. Shi, L.; Guo, Z.; Ni, Y.; Yao, L.; Bazargan, M. Implementation of a distributed parallel computing architecture for transient stability constrained ttc evaluation. In Proceedings of the 2009 IEEE Power Energy Society General Meeting, Calgary, AB, Canada, 26–30 July 2009; pp. 1–6. [Google Scholar] [CrossRef]
  114. Chen, Y.; Huang, Z.; Chavarría-Miranda, D. Performance evaluation of counter-based dynamic load balancing schemes for massive contingency analysis with different computing environments. In Proceedings of the IEEE PES General Meeting, Minneapolis, MN, USA, 25–29 July 2010; pp. 1–6. [Google Scholar] [CrossRef]
  115. Jin, S.; Huang, Z.; Chen, Y.; Chavarría-Miranda, D.; Feo, J.; Wong, P.C. A novel application of parallel betweenness centrality to power grid contingency analysis. In Proceedings of the 2010 IEEE International Symposium on Parallel Distributed Processing (IPDPS), Atlanta, GA, USA, 19–23 April 2010; pp. 1–7. [Google Scholar] [CrossRef]
  116. Mittal, A.; Hazra, J.; Jain, N.; Goyal, V.; Seetharam, D.P.; Sabharwal, Y. Real Time Contingency Analysis for Power Grids. In Euro-Par 2011 Parallel Processing; Jeannot, E., Namyst, R., Roman, J., Eds.; Springer: Berlin/Heidelberg, Germany, 2011; pp. 303–315. [Google Scholar]
  117. Khaitan, S.K.; McCalley, J.D. Achieving Load-Balancing in Power System Parallel Contingency Analysis Using X10 Programming Language. In Proceedings of the Third ACM SIGPLAN X10 Workshop, X10 ’13, Seattle, WA, USA, 20 June 2013; Association for Computing Machinery: New York, NY, USA, 2013; pp. 20–28. [Google Scholar] [CrossRef]
  118. Khaitan, S.K.; McCalley, J.D. Dynamic Load Balancing and Scheduling for Parallel Power System Dynamic Contingency Analysis. In High Performance Computing in Power and Energy Systems; Springer: Berlin/Heidelberg, Germany, 2013; pp. 189–209. [Google Scholar] [CrossRef]
  119. Khaitan, S.K.; McCalley, J.D. Parallelizing power system contingency analysis using D programming language. In Proceedings of the 2013 IEEE Power Energy Society General Meeting, Vancouver, BC, Canada, 21–25 July 2013; pp. 1–5. [Google Scholar] [CrossRef]
  120. Khaitan, S.K.; McCalley, J.D.; Somani, A. Proactive task scheduling and stealing in master-slave based load balancing for parallel contingency analysis. Electr. Power Syst. Res. 2013, 103, 9–15. [Google Scholar] [CrossRef]
  121. Khaitan, S.K.; McCalley, J.D. SCALE: A hybrid MPI and multithreading based work stealing approach for massive contingency analysis in power systems. Electr. Power Syst. Res. 2014, 114, 118–125. [Google Scholar] [CrossRef]
  122. Zhou, G.; Zhao, D.; Zou, K.; Xu, W.; Lv, X.; Wang, Q.; Yin, W. The static security analysis in power system based on Spark Cloud Computing platform. In Proceedings of the 2015 IEEE Innovative Smart Grid Technologies-Asia (ISGT ASIA), Bangkok, Thailand, 3–6 November 2015; pp. 1–6. [Google Scholar] [CrossRef]
  123. Tuyet-Doan, V.N.; Nguyen, T.T.; Nguyen, M.T.; Lee, J.H.; Kim, Y.H. Self-attention network for partial-discharge diagnosis in gas-insulated switchgear. Energies 2020, 13, 2102. [Google Scholar] [CrossRef] [Green Version]
  124. Tournier, J.; Donde, V.; Li, Z. Potential of General Purpose Graphic Processing Unit for Energy Management System. In Proceedings of the 2011 Sixth International Symposium on Parallel Computing in Electrical Engineering, Luton, UK, 3–7 April 2011; pp. 50–55. [Google Scholar] [CrossRef] [Green Version]
  125. Green, R.C.; Wang, L.; Alam, M. Applications and Trends of High Performance Computing for Electric Power Systems: Focusing on Smart Grid. IEEE Trans. Smart Grid 2013, 4, 922–931. [Google Scholar] [CrossRef]
  126. Li, Z.; Zhu, J.; Yang, F. How far is the GPU technology from practical power system applications? In Proceedings of the 2014 IEEE PES General Meeting | Conference Exposition, National Harbor, MD, USA, 27–31 July 2014; pp. 1–5. [Google Scholar] [CrossRef]
  127. Sooknanan, D.J.; Joshi, A. GPU computing using CUDA in the deployment of smart grids. In Proceedings of the 2016 SAI Computing Conference (SAI), London, UK, 13–15 July 2016; pp. 1260–1266. [Google Scholar] [CrossRef]
  128. Zheng, C.; Li, D.; Xi, Y.; Wang, X. Hybrid modeling and optimization for Energy Management System of MicroGrid. In Proceedings of the 2016 35th Chinese Control Conference (CCC), Chengdu, China, 27–29 July 2016; pp. 10013–10018. [Google Scholar] [CrossRef]
  129. Celik, B.; Suryanarayanan, S.; Maciejewski, A.A.; Siegel, H.J.; Sharma, S.; Roche, R. A comparison of three parallel processing methods for a resource allocation problem in the smart grid. In Proceedings of the 2017 North American Power Symposium (NAPS), Morgantown, WV, USA, 17–19 September 2017; pp. 1–6. [Google Scholar] [CrossRef] [Green Version]
  130. Karimipour, H.; Dinavahi, V. On false data injection attack against dynamic state estimation on smart power grids. In Proceedings of the 2017 IEEE International Conference on Smart Energy Grid Engineering (SEGE), Oshawa, ON, Canada, 14–17 August 2017; pp. 388–393. [Google Scholar] [CrossRef]
  131. Capizzi, G.; Sciuto, G.L.; Napoli, C.; Tramontana, E. Advanced and Adaptive Dispatch for Smart Grids by Means of Predictive Models. IEEE Trans. Smart Grid 2018, 9, 6684–6691. [Google Scholar] [CrossRef]
  132. Li, Z.; Yang, F. Advanced Metering Infrastructure and Graphics Processing Unit Technologies in Electric Distribution Networks. In Electric Distribution Network Management and Control; Springer: Singapore, 2018; pp. 309–345. [Google Scholar] [CrossRef]
  133. Liao, C.L.; Lee, S.J.; Chiou, Y.S.; Lee, C.R.; Lee, C.H. Power consumption minimization by distributive particle swarm optimization for luminance control and its parallel implementations. Expert Syst. Appl. 2018, 96, 479–491. [Google Scholar] [CrossRef]
  134. Çavdar, I.; Faryad, V. New design of a supervised energy disaggregation model based on the deep neural network for a smart grid. Energies 2019, 12, 1217. [Google Scholar] [CrossRef] [Green Version]
  135. Perumalla, K.S.; Nutaro, J.J.; Yoginath, S.B. Towards High Performance Discrete-Event Simulations of Smart Electric Grids. In Proceedings of the First International Workshop on High Performance Computing, Networking and Analytics for the Power Grid, HiPCNA-PG ’11, Seattle, WA, USA, 13 November 2011; Association for Computing Machinery: New York, NY, USA, 2011; pp. 51–58. [Google Scholar] [CrossRef]
  136. Baek, J.; Vu, Q.H.; Liu, J.K.; Huang, X.; Xiang, Y. A Secure Cloud Computing Based Framework for Big Data Information Management of Smart Grid. IEEE Trans. Cloud Comput. 2015, 3, 233–244. [Google Scholar] [CrossRef]
  137. Feng, M.; Luo, X.; Frankie, Q.; Zhang, Q.; Litvinov, E. Cloud computing: An innovative IT paradigm to facilitate power system operations. In Proceedings of the 2015 IEEE Power Energy Society General Meeting, Denver, CO, USA, 26–30 July 2015; pp. 1–5. [Google Scholar] [CrossRef]
  138. Soares, J.; Ghazvini, M.A.F.; Vale, Z.; de Moura Oliveira, P.B. A multi-objective model for the day-ahead energy resource scheduling of a smart grid with high penetration of sensitive loads. Appl. Energy 2016, 162, 1074–1088. [Google Scholar] [CrossRef]
  139. Hashemi-Dezaki, H.; Hariri, A.M.; Hejazi, M.A. Impacts of load modeling on generalized analytical reliability assessment of smart grid under various penetration levels of wind/solar/non-renewable distributed generations. Sustain. Energy Grids Netw. 2019, 20, 100246. [Google Scholar] [CrossRef]
  140. Ramanan, P.; Yildirim, M.; Chow, E.; Gebraeel, N. An Asynchronous, Decentralized Solution Framework for the Large Scale Unit Commitment Problem. IEEE Trans. Power Syst. 2019, 34, 3677–3686. [Google Scholar] [CrossRef] [Green Version]
  141. Zhao, L.; Matsuo, I.B.M.; Zhou, Y.; Lee, W.J. Design of an Industrial IoT-Based Monitoring System for Power Substations. IEEE Trans. Ind. Appl. 2019, 55, 5666–5674. [Google Scholar] [CrossRef]
  142. Dileep, G. A survey on smart grid technologies and applications. Renew. Energy 2020, 146, 2589–2625. [Google Scholar] [CrossRef]
  143. Elgamal, M.; Korovkin, N.; Elmitwally, A.; Chen, Z. Robust multi-agent system for efficient online energy management and security enforcement in a grid-connected microgrid with hybrid resources. IET Gener. Transm. Distrib. 2020, 14, 1726–1737. [Google Scholar] [CrossRef]
  144. Na, U.; Lee, E.K. Fog BEMS: An agent-based hierarchical fog layer architecture for improving scalability in a building energy management system. Sustainability 2020, 12, 2831. [Google Scholar] [CrossRef] [Green Version]
  145. Wang, L.; Zheng, J.H.; Jing, Z.X.; Wu, Q.H. Individual-based distributed parallel optimization for operation of integrated energy systems considering heterogeneous structure. Int. J. Electr. Power Energy Syst. 2020, 118, 105777. [Google Scholar] [CrossRef]
  146. Zahmatkesh, H.; Al-Turjman, F. Fog computing for sustainable smart cities in the IoT era: Caching techniques and enabling technologies-an overview. Sustain. Cities Soc. 2020, 59, 102139. [Google Scholar] [CrossRef]
  147. Papadakis, S.E.; Bakrtzis, A.G. A GPU accelerated PSO with application to Economic Dispatch problem. In Proceedings of the 2011 16th International Conference on Intelligent System Applications to Power Systems, Hersonissos, Greece, 25–28 September 2011; pp. 1–6. [Google Scholar] [CrossRef]
  148. Rakai, L.; Rosehart, W. GPU-Accelerated Solutions to Optimal Power Flow Problems. In Proceedings of the 2014 47th Hawaii International Conference on System Sciences, Waikoloa, HI, USA, 6–9 January 2014; pp. 2511–2516. [Google Scholar] [CrossRef]
  149. de Oliveira, L.B.; Marcelino, C.G.; Milanés, A.; Almeida, P.E.; Carvalho, L.M. A successful parallel implementation of NSGA-II on GPU for the energy dispatch problem on hydroelectric power plants. In Proceedings of the 2016 IEEE Congress on Evolutionary Computation (CEC), Vancouver, BC, Canada, 24–29 July 2016; pp. 4305–4312. [Google Scholar] [CrossRef] [Green Version]
  150. Roberge, V.; Tarbouchi, M.; Okou, F. Optimal power flow based on parallel metaheuristics for graphics processing units. Electr. Power Syst. Res. 2016, 140, 344–353. [Google Scholar] [CrossRef]
  151. Yahyaoui, H.; Dekdouk, A.; Krichen, S. Graphic-based optimal network reconfiguration in CPU/GPU architectures using AGA-LS metaheuristics. In Proceedings of the 2016 IEEE International Conference on Power System Technology (POWERCON), Melaka, Malaysia, 28–29 November 2016. [Google Scholar]
  152. Belič, E.; Lukač, N.; Deželak, K.; Žalik, B.; Štumberger, G. GPU-Based Online Optimization of Low Voltage Distribution Network Operation. IEEE Trans. Smart Grid 2017, 8, 1460–1468. [Google Scholar] [CrossRef]
  153. Fioretto, F.; Yeoh, W.; Pontelli, E.; Ma, Y.; Ranade, S.J. A Distributed Constraint Optimization (DCOP) Approach to the Economic Dispatch with Demand Response. In Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems, São Paulo, Brazil, 8–12 May 2017. [Google Scholar]
  154. Huang, S.; Dinavahi, V. GPU-based parallel real-time volt/var optimisation for distribution network considering distributed generators. IET Gener. Transm. Distrib. 2018, 12, 4472–4481. [Google Scholar] [CrossRef]
  155. Araújo, I.; Tadaiesky, V.; Cardoso, D.; Fukuyama, Y.; Santana, Á. Simultaneous parallel power flow calculations using hybrid CPU-GPU approach. Int. J. Electr. Power Energy Syst. 2019, 105, 229–236. [Google Scholar] [CrossRef]
  156. Baldick, R.; Kim, B.H.; Chase, C.; Luo, Y. A fast distributed implementation of optimal power flow. IEEE Trans. Power Syst. 1999, 14, 858–864. [Google Scholar] [CrossRef]
  157. Lo, C.H.; Chung, C.Y.; Nguyen, D.H.M.; Wong, K.P. Parallel evolutionary programming for optimal power flow. In Proceedings of the 2004 IEEE International Conference on Electric Utility Deregulation, Hong Kong, China, 5–8 April 2004. [Google Scholar]
  158. Kim, J.; Jeong, H.M.; Lee, H.S.; Park, J.H. PC Cluster based Parallel PSO Algorithm for Optimal Power Flow. In Proceedings of the 2007 International Conference on Intelligent Systems Applications to Power Systems, Kaohsiung, Taiwan, 4–8 November 2007. [Google Scholar]
  159. Liu, K.; Li, Y.; Sheng, W. The decomposition and computation method for distributed optimal power flow based on message passing interface (MPI). Int. J. Electr. Power Energy Syst. 2011, 33, 1185–1193. [Google Scholar] [CrossRef]
  160. Kraning, M.; Chu, E.; Lavaei, J.; Boyd, S.P. Dynamic Network Energy Management via Proximal Message Passing. Found. Trends Optim. 2014, 1, 73–126. [Google Scholar] [CrossRef] [Green Version]
  161. Ye, C.; Huang, M.X. Multi-Objective Optimal Power Flow Considering Transient Stability Based on Parallel NSGA-II. IEEE Trans. Power Syst. 2015, 30, 857–866. [Google Scholar] [CrossRef]
  162. Grisales-Noreña, L.; Gonzalez Montoya, D.; Ramos-Paja, C.A. Optimal sizing and location of distributed generators based on PBIL and PSO techniques. Energies 2018, 11, 1018. [Google Scholar] [CrossRef] [Green Version]
  163. Grisales-Noreña, L.; Montoya, O.D.; Ramos-Paja, C.A. An energy management system for optimal operation of BSS in DC distributed generation environments based on a parallel PSO algorithm. J. Energy Storage 2020, 29, 101488. [Google Scholar] [CrossRef]
  164. Ujaldon, M. Using GPUs for accelerating electromagnetic simulations. Appl. Comput. Electromagn. Soc. J. 2010, 25, 294–302. [Google Scholar]
  165. Debnath, J.K.; Fung, W.K.; Gole, A.M.; Filizadeh, S. Simulation of large-scale electrical power networks on graphics processing units. In Proceedings of the 2011 IEEE Electrical Power and Energy Conference, Detroit, MA, USA, 24–28 July 2011. [Google Scholar]
  166. Kumar Debnath, J.; Fung, W.K.; Gole, A.M.; Filizadeh, S. Electromagnetic transient simulation of large-scale electrical power networks using graphics processing units. In Proceedings of the 2012 25th IEEE Canadian Conference on Electrical and Computer Engineering (CCECE), Montreal, QC, Canada, 29 April–2 May 2012. [Google Scholar]
  167. Song, Y.; Chen, Y.; Yu, Z.; Huang, S.; Chen, L. A fine-grained parallel EMTP algorithm compatible to graphic processing units. In Proceedings of the 2014 IEEE PES General Meeting | Conference Exposition, National Harbor, MD, USA, 27–31 July 2014. [Google Scholar]
  168. Zhou, Z.; Dinavahi, V. Parallel Massive-Thread Electromagnetic Transient Simulation on GPU. IEEE Trans. Power Deliv. 2014, 29, 1045–1053. [Google Scholar] [CrossRef]
  169. Debnath, J.K.; Gole, A.M.; Fung, W.K. Graphics-Processing-Unit-Based Acceleration of Electromagnetic Transients Simulation. IEEE Trans. Power Deliv. 2016, 31, 2036–2044. [Google Scholar] [CrossRef] [Green Version]
  170. Zhao, J.; Liu, J.; Li, P.; Fu, X.; Song, G.; Wang, C. GPU based parallel matrix exponential algorithm for large scale power system electromagnetic transient simulation. In Proceedings of the 2016 IEEE Innovative Smart Grid Technologies-Asia (ISGT-Asia), Melbourne, Australia, 28 November–1 December 2016. [Google Scholar]
  171. Song, Y.; Chen, Y.; Huang, S.; Xu, Y.; Yu, Z.; Marti, J.R. Fully GPU-based electromagnetic transient simulation considering large-scale control systems for system-level studies. IET Gener. Transm. Distrib. 2017, 11, 2840–2851. [Google Scholar] [CrossRef]
  172. Zhou, Z.; Dinavahi, V. Fine-Grained Network Decomposition for Massively Parallel Electromagnetic Transient Simulation of Large Power Systems. IEEE Power Energy Technol. Syst. J. 2017, 4, 51–64. [Google Scholar] [CrossRef]
  173. Song, Y.; Chen, Y.; Huang, S.; Xu, Y.; Yu, Z.; Xue, W. Efficient GPU-Based Electromagnetic Transient Simulation for Power Systems With Thread-Oriented Transformation and Automatic Code Generation. IEEE Access 2018, 6, 25724–25736. [Google Scholar] [CrossRef]
  174. Lin, N.; Dinavahi, V. Parallel High-Fidelity Electromagnetic Transient Simulation of Large-Scale Multi-Terminal DC Grids. IEEE Power Energy Technol. Syst. J. 2019, 6, 59–70. [Google Scholar] [CrossRef]
  175. Lin, N.; Dinavahi, V. Variable Time-Stepping Modular Multilevel Converter Model for Fast and Parallel Transient Simulation of Multiterminal DC Grid. IEEE Trans. Ind. Electron. 2019, 66, 6661–6670. [Google Scholar] [CrossRef]
  176. Yao, S.; Zhang, S.; Guo, W. Electromagnetic transient parallel simulation optimisation based on GPU. J. Eng. 2019, 2019, 1737–1742. [Google Scholar] [CrossRef]
  177. Shu, D.; Wei, Y.; Dinavahi, V.; Wang, K.; Yan, Z.; Li, X. Cosimulation of Shifted-Frequency/Dynamic Phasor and Electromagnetic Transient Models of Hybrid LCC-MMC DC Grids on Integrated CPU–GPUs. IEEE Trans. Ind. Electron. 2020, 67, 6517–6530. [Google Scholar] [CrossRef]
  178. DeLeon, R.; Felzien, K.; Senocak, I. Toward a gpu-accelerated immersed boundary method for wind forecasting over complex terrain. In Fluids Engineering Division (Publication); American Society of Mechanical Engineers: New York, NY, USA, 2012; Volume 1, pp. 1385–1394. [Google Scholar] [CrossRef] [Green Version]
  179. Lukač, N.; Žalik, B. GPU-based roofs’ solar potential estimation using LiDAR data. Comput. Geosci. 2013, 52, 34–41. [Google Scholar] [CrossRef]
  180. Bonanno, F.; Sciuto, G.L.; Napoli, C.; Pappalardo, G.; Tramontana, E. A novel cloud-distributed toolbox for optimal energy dispatch management from renewables in IGSs by using WRNN predictors and GPU parallel solutions. In Proceedings of the 2014 International Symposium on Power Electronics, Electrical Drives, Automation and Motion, Ischia, Italy, 18–20 June 2014; pp. 1077–1084. [Google Scholar] [CrossRef]
  181. Gao, H.; Chen, Y.; Xu, Y.; Yu, Z.; Chen, L. A GPU-based parallel simulation platform for large-scale wind farm integration. In Proceedings of the 2014 IEEE PES T D Conference and Exposition, National Harbor, MD, USA, 27–31 July 2014; pp. 1–5. [Google Scholar] [CrossRef]
  182. Morales-Aguilar, E.; Ramirez, A.; Matar, M. Multi-frequency sweeping method for periodic steady-state computations on the graphics processor unit. Electr. Power Syst. Res. 2015, 121, 295–301. [Google Scholar] [CrossRef]
  183. Liu, M.; Liu, G.; Zhao, G.; Li, Z.; Zhao, S.; Guo, C.; Chu, Z. A GPU-Based Lagrange Multiplier Optimization for Dynamic Economic Dispatch Aiming to Reduce Wind Power Curtailment. In IOP Conference Series: Materials Science and Engineering; IOP Publishing: Bristol, UK, 2018. [Google Scholar]
  184. Perera, A.T.D.; Wickramasinghe, P.U.; Scartezzini, J.L.; Nik, V.M. Integrating Renewable Energy Technologies into Distributed Energy Systems Maintaining System Flexibility. In Proceedings of the 2018 5th International Symposium on Environment-Friendly Energies and Applications (EFEA), Rome, Italy, 24–26 September 2018. [Google Scholar] [CrossRef]
  185. Abdelaziz, M.; Moradzadeh, M. Monte-Carlo simulation based multi-objective optimum allocation of renewable distributed generation using OpenCL. Electr. Power Syst. Res. 2019, 170, 81–91. [Google Scholar] [CrossRef]
  186. Kang, D.; Youn, C.H. Real-Time Control for Power Cost Efficient Deep Learning Processing With Renewable Generation. IEEE Access 2019, 7, 114909–114922. [Google Scholar] [CrossRef]
  187. Liang, L.; Hou, Y.; Hill, D.J. GPU-Based Enumeration Model Predictive Control of Pumped Storage to Enhance Operational Flexibility. IEEE Trans. Smart Grid 2019, 10, 5223–5233. [Google Scholar] [CrossRef]
  188. Lin, N.; Dinavahi, V. Exact Nonlinear Micromodeling for Fine-Grained Parallel EMT Simulation of MTDC Grid Interaction With Wind Farm. IEEE Trans. Ind. Electron. 2019, 66, 6427–6436. [Google Scholar] [CrossRef]
  189. Hu, T.; Wu, W.; Guo, Q.; Sun, H.; Shi, L.; Shen, X. Very short-term spatial and temporal wind power forecasting: A deep learning approach. CSEE J. Power Energy Syst. 2020, 6, 434–443. [Google Scholar] [CrossRef]
  190. Lin, N.; Cao, S.; Dinavahi, V. Comprehensive Modeling of Large Photovoltaic Systems for Heterogeneous Parallel Transient Simulation of Integrated AC/DC Grid. IEEE Trans. Energy Convers. 2020, 35, 917–927. [Google Scholar] [CrossRef]
  191. Cisneros-Magaña, R.; Medina, A.; Dinavahi, V. Parallel Kalman filter based time-domain harmonic state estimation. In Proceedings of the 2013 North American Power Symposium (NAPS), Manhattan, NY, USA, 22–24 September 2013. [Google Scholar] [CrossRef]
  192. Karimipour, H.; Dinavahi, V. Accelerated parallel WLS state estimation for large-scale power systems on GPU. In Proceedings of the 2013 North American Power Symposium (NAPS), Manhattan, NY, USA, 22–24 September 2013. [Google Scholar] [CrossRef]
  193. Karimipour, H.; Dinavahi, V. On detailed synchronous generator modeling for massively parallel dynamic state estimation. In Proceedings of the 2014 North American Power Symposium (NAPS), Pullman, WA, USA, 7–9 September 2014; pp. 1–6. [Google Scholar] [CrossRef]
  194. Karimipour, H.; Dinavahi, V. Extended Kalman Filter-Based Parallel Dynamic State Estimation. IEEE Trans. Smart Grid 2015, 6, 1539–1549. [Google Scholar] [CrossRef]
  195. Rahman, M.A.; Venayagamoorthy, G.K. Dishonest Gauss Newton method based power system state estimation on a GPU. In Proceedings of the 2016 Clemson University Power Systems Conference (PSC), Clemson, SC, USA, 8–11 March 2016; pp. 1–6. [Google Scholar] [CrossRef]
  196. Rahman, M.A.; Venayagamoorthy, G.K. Convergence of the Fast State Estimation for Power Systems. SAIEE Afr. Res. J. 2017, 108, 117–127. [Google Scholar] [CrossRef] [Green Version]
  197. Xia, Y.; Chen, Y.; Ren, Z.; Huang, S.; Wang, M.; L, M. State estimation for large-scale power systems based on hybrid CPU-GPU platform. In Proceedings of the 2017 IEEE Conference on Energy Internet and Energy System Integration (EI2), Beijing, China, 26–28 November 2017. [Google Scholar] [CrossRef]
  198. Cisneros-Magaña, R.; Medina, A.; Dinavahi, V.; Ramos-Paz, A. Time-Domain Power Quality State Estimation Based on Kalman Filter Using Parallel Computing on Graphics Processing Units. IEEE Access 2018, 6, 21152–21163. [Google Scholar] [CrossRef]
  199. Cao, Z.; Wang, Y.; Chu, C.C.; Gadh, R. Robust pseudo-measurement modeling for three-phase distribution systems state estimation. Electr. Power Syst. Res. 2020, 180, 106138. [Google Scholar] [CrossRef]
  200. Jeong, H.; Lee, H.S.; Park, J.H. Application of parallel particle swarm optimization on power system state estimation. In Proceedings of the 2009 Transmission Distribution Conference Exposition: Asia and Pacific, Seoul, Korea, 26–30 October 2009; pp. 1–4. [Google Scholar] [CrossRef]
  201. Korres, G.N.; Tzavellas, A.; Galinas, E. A distributed implementation of multi-area power system state estimation on a cluster of computers. Electr. Power Syst. Res. 2013, 102, 20–32. [Google Scholar] [CrossRef]
  202. Garcia, N.; Olmos, R.C. GPU-accelerated Poincaré map method for harmonic-oriented analyses of power systems. In Proceedings of the 2013 IEEE Power Energy Society General Meeting, Vancouver, BC, Canada, 21–25 July 2013; pp. 1–5. [Google Scholar] [CrossRef]
  203. Yang, K.; Li, H.; Huang, Y.; Zhang, Q.; Zhao, G. GPU-accelerated particle swarm optimization for selective harmonic elimination in multilevel converters with unequal DC levels. In Proceedings of the IECON 2017-43rd Annual Conference of the IEEE Industrial Electronics Society, Beijing, China, 29 October–1 November 2017; pp. 1186–1191. [Google Scholar] [CrossRef]
  204. Balouji, E.; Salor, Ö.; Ermis, M. Exponential Smoothing of Multiple Reference Frame Components With GPUs for Real-Time Detection of Time-Varying Harmonics and Interharmonics of EAF Currents. IEEE Trans. Ind. Appl. 2018, 54, 6566–6575. [Google Scholar] [CrossRef]
  205. Severoğlu, N.; Salor, Ö. Harmonic analysis in power systems using convolutional neural networks. In Proceedings of the 2018 26th Signal Processing and Communications Applications Conference (SIU), Izmir, Turkey, 2–5 May 2018; pp. 1–4. [Google Scholar] [CrossRef]
  206. Balouji, E.; Bäckström, K.; McKelvey, T.; Salor, Ö. Deep Learning Based Harmonics and Interharmonics Pre-Detection Designed for Compensating Significantly Time-varying EAF Currents. IEEE Trans. Ind. Appl. 2020, 56, 3250–3260. [Google Scholar] [CrossRef]
  207. Uz-Logoglu, E.; Salor, Ö.; Ermis, M. Real-Time Detection of Interharmonics and Harmonics of AC Electric Arc Furnaces on GPU Framework. IEEE Trans. Ind. Appl. 2019, 55, 6613–6623. [Google Scholar] [CrossRef]
  208. Qiu, W.; Tang, Q.; Liu, J.; Teng, Z.; Yao, W. Power Quality Disturbances Recognition Using Modified S Transform and Parallel Stack Sparse Auto-encoder. Electr. Power Syst. Res. 2019, 174, 105876. [Google Scholar] [CrossRef]
  209. Solis-Munoz, F.; Osornio-Rios, R.A.; Romero-Troncoso, R.J.; Jaen-Cuellar, A.Y. Differential evolution implementation for Power Quality Disturbances monitoring using OpenCL. Adv. Electr. Comput. Eng. 2019, 19, 13–22. [Google Scholar] [CrossRef]
  210. Ma, J.; Man, K.L.; Ting, T.O.; Zhang, N.; Guan, S.U.; Wong, P.W.H. Accelerating Parameter Estimation for Photovoltaic Models via Parallel Particle Swarm Optimization. In Proceedings of the 2014 International Symposium on Computer, Consumer and Control, Taichung, Taiwan, 10–12 June 2014; pp. 175–178. [Google Scholar] [CrossRef]
  211. Liu, Z.; Li, X.; Wu, L.; Zhou, S.; Liu, K. GPU-Accelerated Parallel Coevolutionary Algorithm for Parameters Identification and Temperature Monitoring in Permanent Magnet Synchronous Machines. IEEE Trans. Ind. Inform. 2015, 11, 1220–1230. [Google Scholar] [CrossRef]
  212. Liu, Z.; Wei, H.; Zhong, Q.; Liu, K.; Li, X. GPU Implementation of DPSO-RE Algorithm for Parameters Identification of Surface PMSM Considering VSI Nonlinearity. IEEE J. Emerg. Sel. Top. Power Electron. 2017, 5, 1334–1345. [Google Scholar] [CrossRef] [Green Version]
  213. Klimeš, L.; Mauder, T.; Charvát, P.; Štětina, J. Front tracking in modelling of latent heat thermal energy storage: Assessment of accuracy and efficiency, benchmarking and GPU-based acceleration. Energy 2018, 155, 297–311. [Google Scholar] [CrossRef]
  214. Wang, L.; Zhang, Z.; Huang, C.; Tsui, K.L. A GPU-accelerated parallel Jaya algorithm for efficiently estimating Li-ion battery model parameters. Appl. Soft Comput. 2018, 65, 12–20. [Google Scholar] [CrossRef]
  215. Yüzügüler, A.C.; Moga, A.; Franke, C. Towards Commoditizing Simulations of System Models Using Recurrent Neural Networks. In Proceedings of the 2018 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm), Aalborg, Denmark, 29–31 October 2018; pp. 1–6. [Google Scholar] [CrossRef]
  216. Ma, J.; Man, K.L.; Guan, S.U.; Ting, T.O.; Wong, P.W.H. Parameter estimation of photovoltaic model via parallel particle swarm optimization algorithm. Int. J. Energy Res. 2016, 40, 343–352. [Google Scholar] [CrossRef]
  217. Schwarzer, V.; Ghorbani, R. New opportunities for large-scale design optimization of electric vehicles using GPU technology. In Proceedings of the 2011 IEEE Vehicle Power and Propulsion Conference, Chicago, IL, USA, 6–9 September 2011; pp. 1–6. [Google Scholar] [CrossRef]
  218. Sánchez, L.; Otero, J.; Couso, I.; Blanco, C. Battery diagnosis for electrical vehicles through semi-physical fuzzy models. In Proceedings of the 2016 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Vancouver, BC, Canada, 24–29 July 2016; pp. 416–423. [Google Scholar] [CrossRef]
  219. Schambers, A.; Eavis-O’Quinn, M.; Roberge, V.; Tarbouchi, M. Route planning for electric vehicle efficiency using the Bellman-Ford algorithm on an embedded GPU. In Proceedings of the 2018 4th International Conference on Optimization and Applications (ICOA), Mohammedia, Morocco, 26–27 April 2018; pp. 1–6. [Google Scholar] [CrossRef]
  220. Roberge, V.; Tarbouchi, M.; Noureldin, A. Integrated Motor Optimization and Route Planning for Electric Vehicle using Embedded GPU System. In Proceedings of the 2019 5th International Conference on Optimization and Applications (ICOA), Kenitra, Morocco, 25–26 April 2019; pp. 1–6. [Google Scholar] [CrossRef]
  221. Abdelaziz, M. GPU-OpenCL accelerated probabilistic power flow analysis using Monte-Carlo simulation. Electr. Power Syst. Res. 2017, 147, 70–72. [Google Scholar] [CrossRef]
  222. Abdelaziz, M.M.A. OpenCL-Accelerated Probabilistic Power Flow for Active Distribution Networks. IEEE Trans. Sustain. Energy 2018, 9, 1255–1264. [Google Scholar] [CrossRef]
  223. Zhou, G.; Bo, R.; Chien, L.; Zhang, X.; Yang, S.; Su, D. GPU-Accelerated Algorithm for Online Probabilistic Power Flow. IEEE Trans. Power Syst. 2018, 33, 1132–1135. [Google Scholar] [CrossRef]
  224. Cui, T.; Franchetti, F. A multi-core high performance computing framework for probabilistic solutions of distribution systems. In Proceedings of the 2012 IEEE Power and Energy Society General Meeting, San Diego, CA, USA, 22–26 July 2012; pp. 1–6. [Google Scholar] [CrossRef] [Green Version]
  225. Borges, C.L.T.; Alves, J.M. Power System Real Time Operation based on Security Constrained Optimal Power Flow and Distributed Processing. In Proceedings of the 2007 IEEE Lausanne Power Tech, Lausanne, Switzerland, 1–5 July 2007; pp. 960–965. [Google Scholar] [CrossRef]
  226. Thitithamrongchai, C.; Eua-Arporn, B. Security-constrained Optimal Power Flow: A Parallel Self-adaptive Differential Evolution Approach. Electr. Power Compon. Syst. 2008, 36, 280–298. [Google Scholar] [CrossRef]
  227. Li, Y.; McCalley, J.D. Decomposed SCOPF for Improving Efficiency. IEEE Trans. Power Syst. 2009, 24, 494–495. [Google Scholar] [CrossRef]
  228. Kim, M.; Park, J.K.; Nam, Y.W. Market-clearing for pricing system security based on voltage stability criteria. Energy 2011, 36, 1255–1264. [Google Scholar] [CrossRef]
  229. Geng, G.; Jiang, Q.; Sun, Y. Parallel Transient Stability-Constrained Optimal Power Flow Using GPU as Coprocessor. IEEE Trans. Smart Grid 2017, 8, 1436–1445. [Google Scholar] [CrossRef]
  230. Cai, H.R.; Chung, C.Y.; Wong, K.P. Application of Differential Evolution Algorithm for Transient Stability Constrained Optimal Power Flow. IEEE Trans. Power Syst. 2008, 23, 719–728. [Google Scholar] [CrossRef]
  231. Tian, F.; Zhang, X.; Yu, Z.; Qiu, W.; Shi, D.; Qiu, J.; Liu, M.; Li, Y.; Zhou, X. Online decision-making and control of power system stability based on super-real-time simulation. CSEE J. Power Energy Syst. 2016, 2, 95–103. [Google Scholar] [CrossRef] [Green Version]
  232. Tate, J.E.; Overbye, T.J. Contouring for Power Systems Using Graphical Processing Units. In Proceedings of the 41st Annual Hawaii International Conference on System Sciences (HICSS 2008), Waikoloa, HI, USA, 7–10 January 2008; p. 168. [Google Scholar] [CrossRef]
  233. Chen, Z.; Shen, L.; Zhao, Y.; Yang, C. Parallel algorithm for real-time contouring from grid DEM on modern GPUs. Sci. China Technol. Sci. 2010, 53, 33–37. [Google Scholar] [CrossRef]
  234. Shang, B.; Xu, Y.; Zhang, C.; Chen, Y.; Liu, Z.; Lin, L.; Xu, C.; Yu, J. GPU-Accelerated Batch Solution for Short-Circuit Current Calculation of Large-scale Power Systems. In Proceedings of the 2019 IEEE 3rd International Electrical and Energy Conference (CIEEC), Beijing, China, 7–9 September 2019; pp. 1743–1748. [Google Scholar] [CrossRef]
  235. Sato, F.; Garcia, A.V.; Monticelli, A.; Alves, A.B. Distributed short-circuit analysis in heterogeneous computer networks. Int. J. Electr. Power Energy Syst. 2000, 22, 129–136. [Google Scholar] [CrossRef]
  236. Fukuyama, Y.; Chiang, H.D. A parallel genetic algorithm for generation expansion planning. IEEE Trans. Power Syst. 1996, 11, 955–961. [Google Scholar] [CrossRef]
  237. Ye, C.; Ding, Y.; Song, Y.; Lin, Z.; Wang, L. A data driven multi-state model for distribution system flexible planning utilizing hierarchical parallel computing. Appl. Energy 2018, 232, 9–25. [Google Scholar] [CrossRef]
  238. Rami, A.; Zeblah, A.; Hamdaoui, H.; Massim, Y.; Harrou, F. An efficient artificial immune algorithm for power system reliability optimisation. Int. J. Power Energy Convers. 2009, 1, 178–197. [Google Scholar] [CrossRef]
  239. Green, R.C.; Wang, L.; Alam, M.; Singh, C. Intelligent and parallel state space pruning for power system reliability analysis using MPI on a multicore platform. In Proceedings of the ISGT 2011, Anaheim, CA, USA, 17–19 January 2011; pp. 1–8. [Google Scholar] [CrossRef]
  240. He, T.; Meng, K.; Dong, Z.Y.; Oh, Y.T.; Xu, Y. Use of High-performance Graphics Processing Units for Power System Demand Forecasting. J. Electr. Eng. Technol. 2010, 5, 363–370. [Google Scholar] [CrossRef]
  241. Milano, F. Small-Signal Stability Analysis of Large Power Systems With Inclusion of Multiple Delays. IEEE Trans. Power Syst. 2016, 31, 3257–3266. [Google Scholar] [CrossRef]
  242. Zadeh, A.K.; Nor, K.M.; Zeynal, H. Multi-thread security constraint economic dispatch with exact loss formulation. In Proceedings of the 2010 IEEE International Conference on Power and Energy, Kuala Lumpur, Malaysia, 29 November–1 December 2010; pp. 864–869. [Google Scholar] [CrossRef]
  243. Zhang, J.; Lin, S.; Liu, H.; Chen, Y.; Zhu, M.; Xu, Y. A small-population based parallel differential evolution algorithm for short-term hydrothermal scheduling problem considering power flow constraints. Energy 2017, 123, 538–554. [Google Scholar] [CrossRef]
Algorithms 14 00275 g001 550
Figure 1. PHC implementations organized by technology and year.
Figure 1. PHC implementations organized by technology and year.
Algorithms 14 00275 g001
Algorithms 14 00275 g002 550
Figure 2. PHC applications in power systems.
Figure 2. PHC applications in power systems.
Algorithms 14 00275 g002
Table
Table 1. PHC applications in power system studies and analysis.
Table 1. PHC applications in power system studies and analysis.
ApplicationGPUCPU Clusters, Fog and CloudTotal
Computing, and FPGA
Power Flow Analysis[6,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46][47,48,49,50,51,52,53,54,55,56,57,58,59,60]49
Transient Stability[18,61,62,63,64,65,66,67,68,69,70,71,72,73,74][75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98]39
Contingency Analysis[99,100,101,102,103,104,105,106,107,108][94,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123]26
Smart Grids[124,125,126,127,128,129,130,131,132,133,134][135,136,137,138,139,140,141,142,143,144,145,146]23
Optimal Power Flow[1,147,148,149,150,151,152,153,154,155][156,157,158,159,160,161,162,163]18
Electromagnetic Transient Simulation[164,165,166,167,168,169,170,171,172,173,174,175,176,177][91,92]16
Renewable Energy Integration[1,178,179,180,181,182,183,184,185,186,187,188,189,190]-14
Dynamic State Estimation[191,192,193,194,195,196,197,198,199][94,200,201]12
Power Quality[202,203,204,205,206,207,208,209]-8
Dynamic Models[210,211,212,213,214,215][2,216]8
Electrical Vehicles[217,218,219,220]-4
Probabilistic Power Flow[221,222,223][224]4
Security Constrained Optimal Power Flow-[225,226,227,228]4
Transient Stability-Constrained Optimal Power Flow[229][230,231]3
Power System Visualization[232,233]-2
Short Circuit Analysis[234][235]2
Power System Planning-[236,237]2
Power System Reliability-[238,239]2
Electricity Market[240]-1
Small Signal Analysis[241]-1
Security Constrained Economic Dispatch-[242]1
Hydrothermal Scheduling-[243]1
Total14595240
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Rodriguez, D.; Gomez, D.; Alvarez, D.; Rivera, S. A Review of Parallel Heterogeneous Computing Algorithms in Power Systems. Algorithms 2021, 14, 275. https://doi.org/10.3390/a14100275

AMA Style

Rodriguez D, Gomez D, Alvarez D, Rivera S. A Review of Parallel Heterogeneous Computing Algorithms in Power Systems. Algorithms. 2021; 14(10):275. https://doi.org/10.3390/a14100275

Chicago/Turabian Style

Rodriguez, Diego, Diego Gomez, David Alvarez, and Sergio Rivera. 2021. "A Review of Parallel Heterogeneous Computing Algorithms in Power Systems" Algorithms 14, no. 10: 275. https://doi.org/10.3390/a14100275

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop