Neural Network Algorithm with Reinforcement Learning for Microgrid Techno-Economic Optimization

Abstract

Hybrid energy systems (HESs) are gaining prominence as a practical solution for powering remote and rural areas, overcoming limitations of conventional energy generation methods, and offering a blend of technical and economic benefits. This study focuses on optimizing the sizes of an autonomous microgrid/HES in the Kingdom of Saudi Arabia, incorporating solar photovoltaic energy, wind turbine generators, batteries, and a diesel generator. The innovative reinforcement learning neural network algorithm (RLNNA) is applied to minimize the annualized system cost (ASC) and enhance system reliability, utilizing hourly wind speed, solar irradiance, and load behavior data throughout the year. This study validates RLNNA against five other metaheuristic/soft-computing approaches, demonstrating RLNNA’s superior performance in achieving the lowest ASC at USD 1,219,744. This outperforms SDO and PSO, which yield an ASC of USD 1,222,098.2, and MRFO, resulting in an ASC of USD 1,222,098.4, while maintaining a loss of power supply probability (LPSP) of 0%. RLNNA exhibits faster convergence to the global solution than other algorithms, including PSO, MRFO, and SDO, while MRFO, PSO, and SDO show the ability to converge to the optimal global solution. This study concludes by emphasizing RLNNA’s effectiveness in optimizing HES sizing, contributing valuable insights for off-grid energy systems in remote regions.

1. Introduction

Many countries worldwide have shown interest in renewable energy conversion sources such as wind and solar photovoltaic (PV) due to the limited lifespan of fossil fuels, which is projected to be only a few more decades. Furthermore, around three billion individuals reside in geographically isolated and rural areas across the globe [1]. Remote areas lacking access to the grid are categorized as off-grid loads. The inherent uncertainty associated with renewable energy sources poses substantial challenges, making reliance on a single renewable energy conversion resource particularly difficult for the electrification of rural loads [2]. Hybrid energy systems (HESs) integrate several power resources, including solar photovoltaic (PV) modules/panels, wind turbine generators (WTs), diesel generators, fuel cells (FCs), and batteries. Hybrid energy systems offer a viable and dependable alternative for providing electricity to off-grid and distant locations. Additionally, they can help delay or eliminate the expenses associated with extending the power grid. HES sizing and design offer an extremely complex nonlinear optimization challenge, despite their numerous benefits, such as environmental friendliness, reduced storage needs, and low maintenance costs [3]. The optimal size of HESs is crucial for obtaining techno-economic benefits since they not only contribute to energy or cost reduction but also extend their functional lifespan [4]. Moreover, optimal sizing refers to a complex optimization design challenge that seeks to minimize one or several objectives while adhering to certain constraints. Efficient energy storage solutions pose an ongoing challenge in HES design [5]. The need to store surplus energy captured during peak times for use during low-generation periods requires identifying suitable and cost-effective energy storage technologies. Furthermore, balancing the economic aspects of HES design is a crucial challenge, considering the initial costs of components, long-term benefits, and operational costs. Ensuring the economic viability of HESs is vital for widespread adoption and sustained success. Moreover, community engagement and acceptance are critical, especially in off-grid and remote areas [6]. Designing HESs that align with local needs, preferences, and socioeconomic conditions is essential for garnering support and ensuring long-term sustainability.

Hence, the primary step to resolve this issue aims to identify the objective function of the system design, whereas the second crucial step aims to select the appropriate optimizer that will be used to address this problem, with the aim of designing microgrid/HES elements based on the predetermined objective function [7]. Currently, the process of determining the optimal sizing of HES components often involves transforming it into an optimization problem. Optimization problems may be solved using two distinct approaches: deterministic techniques and metaheuristic methods [8]. Due to the nonlinear nature and complicated objective functions involved in selecting the components of an HES, deterministic methods are seldom utilized to tackle these optimization issues because of the following factors [9]. Initially, these approaches are highly sensitive to the initial solutions, which can rapidly become stuck in the local optima. Furthermore, these strategies rely on rigorous mathematical assumptions and may be in need of gradient information. In contrast to deterministic approaches, metaheuristic approaches use simplicity and unpredictability to emulate natural processes. For instance, the particle swarm optimizer (PSO) is motivated by social behavior of bird swarming [10], whereas cuckoo search (CS) is motivated by the brood parasitism observed in some cuckoo species [11]. Metaheuristic approaches are more suited for addressing difficult optimization issues compared to deterministic methods due to their simplicity and unpredictability. Several metaheuristic techniques have been utilized to determine the appropriate size of HESs with different technical and/or economic objectives [12,13]. The technical objectives are primarily focused around assessing the reliability of the HES using various reliability parameters, including loss of load expected (LOLE), loss of power supply probability (LPSP), loss of energy expected (LOEE), loss of load hours (LOLH), deficiency of power supply probability (DPSP), equivalent loss factor (ELF), unmet load (UL), and renewable energy fraction (REF). The life cycle cost (LCC), cost of energy (COE), net present cost (NPC), and annualized system cost (ASC) are all included in the economic goals.

Table 1. Microgrid techno-economic optimization in the literature considering several design factors.

Ref.	Microgrid/HES Configurations	Optimization Technique	Technical Objectives and Constraints
[31]	PV/WTs/battery	CS, GA, PSO	ASC, seasonal load variation
[14]	PV/diesel	HS	ASC and CO2
[15]	PV/WTs/battery	GWO	ASC, power balance constraints
[16]	PV/WTs/battery	FA	COE, load dissatisfaction rate
[17]	PV/WTs/FC	ABS	ASC, LPSP
[32]	PV/WTs/diesel/battery	HOMER	NPC and CO2 emission
[18]	PV/WTs/diesel	GA	ASC, LPSP
[20]	PV/WTs/battery	PSO	ASC, full-load demand supply
[33]	WTs/PV/FC	PSO	ASC, LOLE, LOEE, ELF
[34]	PV/wind/battery	GA	ASC, LOLH
[19]	PV/WTs/battery/diesel	GA	COE and CO2 emission
[7]	PV/WTs/battery/diesel	GA	COE, REF with zero LPSP
[21]	PV/WTs/FC	PSO	NPC and LPSP
[22]	PV/wind/diesel/FC/battery	PSO	Multi-objective: UL, NPC, CO2
[23]	PV/WTs/FC	SA, TS, PSO, HS	ASC
[24]	PV/battery, WTs/battery, PV/WTs/battery	GA-PSO	NPC and LPSP
[25]	PV/WTs/FC	MBA	ASC
[26]	PV/WTs/tidal/battery	CSA	NPC, ELF
[27]	PV/FC/diesel	CSA	COE and LPSP
[28]	PV/WTs/diesel/battery	IFA	ASC and CO2
[35]	PV/Biomass/battery	FPA, ABC, HS, FA	NPC, LPSP, excess energy
[36]	PV/wind/battery	BBBC	NPC and UL
[37]	PV/diesel	HS	ASC and CO2
[29]	PV/WTs/FC	FPA	NPC, LOLE, LOEE
[30]	PV/WTs/battery/diesel	GOA	COE, DPSP
[38]	PV/WT/diesel/battery	MLUCA	TAC and GHG emissions minimization
[39]	PV/WT/diesel/battery	ACO	Minimize total annual cost (TAC)
[40]	PV/WT/diesel/FC/hydrogen tank	MBA	TAC
[41]	PV/WT/diesel/battery	PICEA	Minimize ACS, LPSP
[42]	PV/WT/bio gasifiers/diesel/battery	GWCSO	Minimize the total cost
[43]	PV/WT/battery	IEBSA	Minimizing losses and cost, optimizing the voltage profile, and enhancing ENS
[44]	PV/WT/diesel/battery	HHO	ASC minimization
[45]	PV/WT/hydrogen/battery	HSSA	Life cycle cost minimization
[46]	PV/WT/battery/grid	NN	PLPSP

provides an overview of the reviewed literature for this study. This table summarizes a diverse collection of research endeavors focused on optimizing HES configurations, each employing distinct methodologies and addressing unique technical objectives. In the realm of photovoltaic (PV) and diesel hybrid systems [14,15], the application of the Harmony Search (HS) algorithm aims at minimizing both the annualized system cost (ASC) and carbon dioxide (CO₂) emissions, showcasing a dual emphasis on economic and environmental sustainability. The Grey Wolf Optimizer (GWO) [15] was applied to optimize the annualized system cost (ASC) by including PV panels, WTs, and battery storage. This algorithm considers power balancing limitations and focuses on maintaining equilibrium between power generation and consumption. The authors in [16] introduced the Firefly Algorithm (FA) to address the conflicting objectives of minimizing the cost of energy (COE) and mitigating load dissatisfaction, offering insights into balancing economic efficiency and consumer satisfaction in hybrid energy systems. The inclusion of Artificial Bee Swarm Optimization (ABS) in the PV/WTs/FC configuration [17] underscores a focus on minimizing ASC with adherence to the loss of power supply probability (LPSP). Similarly, studies incorporating the Genetic Algorithms (GAs) approach [7,18,19] tackle challenges such as ASC minimization, LPSP constraints, and reducing CO₂ emissions, showcasing the versatility of GA in optimizing various aspects of hybrid systems. The particle swarm optimizer (PSO) is prevalent in optimizing PV/WTs/battery [20], PV/WTs/FC [21], and PV/wind/diesel/FC/battery [22] configurations. These studies delve into multi-objective optimization, considering factors such as unsupplied load (UL), net present cost (NPC), and CO₂ emissions. Additionally, a comparative analysis was conducted in [23] involving Tabu Search (TS), Simulated Annealing (SA), PSO, and HS, emphasizing PSO’s superior performance in ASC minimization. Furthermore, hybrid approaches, such as combining GA with PSO [24] and utilizing the CS algorithm alongside GA and PSO [2], demonstrate the potential synergies in optimizing NPC and LPSP. The integration of the Mine Blast Algorithm (MBA) [25] introduces a novel perspective in ASC minimization for PV/WTs/FC systems, showcasing its efficiency and reliability compared to other metaheuristic techniques. Studies involving the Crow Search Algorithm (CSA) [26,27] emphasize minimizing NPC while considering constraints like Energy Loss Fraction (ELF) and COE. Additionally, the Improved Firefly Algorithm (IFA) [28] addresses ASC and CO₂ emissions in a PV/WTs/diesel/battery configuration. The introduction of the Flower Pollination Algorithm (FPA) in PV/WTs/FC [29] prioritizes minimizing NPC, LOLE, and LOEE, providing a holistic view of system performance. The inclusion of the Grasshopper Optimization Algorithm (GOA) in PV/WTs/battery/diesel [30] explores the optimization of COE and DPSP, underlining the significance of addressing economic considerations and system reliability.

The Multi-objective Line-up Competition Algorithm (MLUCA) stands out for its adeptness in simultaneously minimizing greenhouse gas (GHG) emissions and total annual cost (TAC) in PV/WT/diesel/battery systems [38]. Ant Colony Optimization (ACO) emerges as a noteworthy contender, excelling in TAC minimization across configurations involving photovoltaic technology, wind turbine generators, diesel generators, and battery banks [39]. The Mine Blast Algorithm (MBA) showcases versatility by effectively addressing TAC reduction in PV/WT/diesel/FC/hydrogen tank systems [40]. The Preference-Inspired Co-Evolutionary Algorithm (PICEA) exhibits proficiency in simultaneously minimizing Annual Cost of Supply (ACS), LPSP, and fuel emissions in PV/WT/diesel/battery configurations [41]. Hybrid Grey Wolf with Cuckoo Search Optimization (GWCSO) is applied to determine the optimal size of solar PV, WT, biomass gasifiers, batteries, and a standby diesel generator based on the total annualized cost minimization [42]. The application of the Improved Escaping-Bird Search Algorithm (IEBSA) is employed to ascertain the optimal design for solar PV, WT, and batteries [43]. This optimization is based on the criteria of minimizing energy losses, optimizing the voltage profile, considering the associated system costs, and enhancing the Energy-Not-Supplied (ENS) index [43]. Harris Hawk optimization (HHO) was used to find the best design for the microgrid/HES based on ASC minimization and power network reliability [44]. The Improved Fruit Fly Algorithm (IFFA) demonstrates effectiveness in addressing cost and emissions considerations in PV/WT/diesel/battery systems [28]. Harmony-Search-Based Simulated Annealing (HSSA) emerges as a robust choice for minimizing the life cycle cost within PV/WT/hydrogen/battery systems [45]. Genetic Algorithm particle swarm optimization (GAPSO) garners recognition for its prowess in minimizing total present cost in PV/WT/battery systems [24]. Lastly, a neural network (NN) establishes its efficacy in reducing the potential loss of power supply probability (PLPSP) within PV/WT/battery/utility grid systems [46]. This comprehensive review elucidates the diverse landscape of optimization methodologies, offering valuable insights for scholars and practitioners navigating the intricate domain of HES optimization.

The variety and efficacy of artificial neural networks (ANNs) have been prominently demonstrated in numerous facets of our lives in recent years. The neural network algorithm (NNA), inspired by ANNs, has demonstrated exceptional global search capabilities [47]. However, its slower convergence rate and susceptibility to local minima in addressing nonlinear complex optimization problems have prompted further enhancements. This article presents the application of a new variant of NNA known as the reinforcement learning neural network algorithm (RLNNA) for the optimal sizing of standalone HESs [48]. RLNNA introduces key improvements to address NNA’s limitations. First, leveraging the principles of reinforcement learning, a modification factor adjustment approach is proposed to maintain the RLNNA population diversity. Additionally, the incorporation of a historical population in the transfer operator enhances the algorithm’s global search ability. A feedback operator containing both historical and present feedback terms is designed to accelerate convergence. The main contributions can be summarized and listed as follows:

▪

This study introduces an innovative soft computing/metaheuristic algorithm, the reinforcement learning neural network algorithm (RLNNA), strategically applied to optimize the annualized system cost (ASC) and enhance overall system reliability in an autonomous microgrid/hybrid energy system (HES). This research, focusing on an off-grid HES in a rural region of Saudi Arabia, integrates diverse energy sources—solar photovoltaic technology, wind turbine generators, batteries, and a diesel generator—demonstrating a comprehensive approach to improving sustainability and efficiency in energy management.

▪

In the validation phase, the proposed RLNNA is rigorously assessed against prominent algorithms such as PSO, GA, SDO, MRFO, and traditional NNA. The optimization criteria encompass minimizing the annualized system cost (ASC), achieving a 0% loss of power supply probability (LPSP), and maximizing the renewable energy fraction (REF), addressing economic, reliability, and sustainability considerations within the microgrid/HES framework.

▪

Remarkably, RLNNA surpasses the other five AI techniques in convergence time and global solution capture, positioning it as a promising and effective algorithm for optimization challenges within computational intelligence-based algorithms. Conversely, GA and NNA face challenges with local solutions and prolonged convergence times, potentially becoming trapped in local optima.

▪

In conclusion, the study underscores RLNNA’s efficacy in optimizing HES sizing, offering unparalleled performance relative to established soft-computing algorithms. These findings provide valuable insights for researchers and practitioners seeking efficient solutions for off-grid energy systems, particularly in remote locales.

The subsequent sections of this paper are thoughtfully organized to provide a comprehensive exploration of the research. Section 2 serves as an introduction to the site description and meteorological data, establishing the contextual foundation for this study. It describes the proposed HES configuration, shedding light on the details of the system design. This section discusses the integration of various energy sources, storage components, and the overall architecture aimed at fulfilling the energy demands of the off-grid rural area. Section 3 introduces the essential components of the optimization algorithm, specifically the RLNNA, which has been proposed for the optimal sizing of the microgrid/HES. This section elaborates on the unique features and mechanisms of RLNNA in tackling the challenges associated with HES optimization. Section 4 transitions into a detailed discussion of the simulation results. Finally, Section 5 provides the major conclusions drawn from this research.

2. System Description and Mathematical Modeling

2.1. System Description

The methodology applied in this paper encompasses several essential facets crucial to comprehensively understanding and optimizing the HES. The resource data, load profile, control techniques, and system component requirements are all covered in detail in this part.

2.1.1. Meteorological Data

Figure 1 presents a visual representation of the solar irradiance levels and monthly mean temperature on a horizontal surface of the area under study. The monthly wind speed is presented in Figure 2. Notably, the highest and lowest temperature and sun irradiance levels are observed during July and January and June and December, respectively. The generation of hydrogen for industrial uses and the use of electrical loads constitute the city’s energy consumption. Figure 3 illustrates the hourly variation in the load, showcasing the fluctuations in energy demand over the course of a day. Additionally, Figure 4 provides insight into the monthly load variations, demonstrating the changes in energy requirements throughout the months.

These visual representations offer a comprehensive understanding of the climatic conditions and energy consumption patterns within the area under study, which are fundamental factors influencing the design and optimization of the HES for this region. The selected load profile sample, depicted in Figure 3 and Figure 4, is chosen for its high variability in daily energy consumption. This diversity in load variations serves to test the system’s performance across a spectrum of electricity consumption levels. A minimum load demand of 210 kW is suggested, while a peak load demand of 2850 kW is created. The average electricity consumption is estimated at approximately 1070.1 kW. These datasets are critical as they simulate real-world conditions, allowing for comprehensive testing and evaluation of the HES’s performance across varying energy demands and wind conditions.

2.1.2. The Microgrid/HES Configuration

The wind turbine, PV system, diesel generator, converter, and batteries are the five main parts of the proposed microgrid/HES. Figure 5 provides a visual representation of the system configuration, offering a schematic illustration of how these components are interconnected and function collectively within the system. Furthermore, the detailed specifications and costs associated with each of these components are outlined in

Table 2. Capital, O&M, replacement costs, and lifetime of the microgrid components [49].

Component	Capital Cost	O&M Cost	Replacement Cost	Lifetime (Years)
Photovoltaic	1000 (USD/kW)	15 (USD/kW)	-	20
Wind turbine	1300 (USD/kW)	30.33 USD/kW)	-	20
Battery bank	200 (USD/kWh)	5 (USD/kWh)	200 (USD/kWh)	5
Diesel generator	300 (USD/kW)	0.012 (USD/kWh)	300 (USD/kW)	10
Converters	133 (USD/kW)	10 (USD/kW)	100 (USD/kW)	10

. This table elucidates the specific characteristics, technical details, and financial costs relevant to the wind turbine, PV system, diesel generator, converter, and batteries integrated within the HES configuration. This comprehensive breakdown is fundamental in understanding the individual contributions of each component to the overall system functionality and economic implications.

2.1.3. The PV/Wind/Diesel/Battery Dispatch Strategy

In HESs, the dispatch strategy plays a pivotal role, acting as a control technique governing the operation of both the battery bank and diesel generator in scenarios where renewable energy sources alone cannot meet the load demand. Optimizing HESs involves performing a techno-economic analysis, and this can be executed through various control techniques, each with distinct advantages and limitations. The proposed technique, developed by MATLAB, aims at offering a more applicable and efficient dispatch technique tailored for microgrid/HESs. This strategy divides its operation into two distinct cases:

Case 1: If the load demand aligns with the power generated from the wind turbines, the wind turbines exclusively supply the load without involving the battery. Consequently, no activation of the generator is required in this scenario.

Case 2: If the power captured by the wind turbines exceeds the load demand, the surplus power is directed towards charging the battery if it is not at full capacity. Should the battery be fully charged, the surplus power is dissipated. Throughout this scenario, the generator remains inactive. However, if the wind power alone cannot meet the load demand, there are further considerations:

▪

If the battery state of charge (SOC) is above the defined minimum SOC (SOCmin), the load requirements are fulfilled through discharging the battery.

▪

If the SOC falls below the minimum threshold (SOCmin), two sub-cases are encountered:

✓

In instances where the load demand falls beneath the minimum load capacity of the diesel generator, the generator functions at its prescribed minimum load ratio. This ratio represents the lowest permissible load on the diesel generator and is typically denoted as a percentage of its overall capacity. For instance, if the stipulated minimum load ratio is established at 40%, and the requisite electrical output from the diesel generator is 50% of its capacity, the diesel generator operates at 50%. Conversely, if the demand is 20%, the generator functions at the minimum load ratio of 40% to avert exceedingly low load operations.

✓

Should the minimum load of the generator be either less than or equivalent to the prevailing load demand, the generator is utilized to fulfil the load requirements without necessitating the involvement of the battery charging process.

The above comprehensive strategy showcases the decision-making process in allocating power from different sources, ensuring optimal utilization while addressing load demand and storage needs within the HES framework.

2.2. Mathematical Modeling

2.2.1. PV System

It is crucial to comprehend the performance of PV systems to accurately estimate their energy output. To compute the hourly output energy of the PV system, critical inputs such as solar irradiance, ambient temperature, and manufacturing information specific to the PV module are utilized within the PV panel. The hourly output power (P_PV, module) of the PV panel/module can be estimated using Equation (1) [50]. This equation sums up the relationship between these input factors and the resultant hourly output power generated by the PV panel:

$$P_{PV,module}=f_{PV}{P}_{R,mod}\left(\frac{\overline{G_T}}{\overline{G_{T,STC}}}\right)\left[1+{\alpha }_P\left(T_c-T_{c,STC}\right)\right]$$

(1)

where:

$P_{R,mod}$: Represents the rated power of the photovoltaic panel/module, measured in kilowatt (kW). This signifies the power output of the PV array under standard test conditions.

$f_{PV}$: denotes the photovoltaic derating factor, typically represented as a percentage (%).

$\overline{G_T}$: signifies the average solar irradiance incident on the PV array, measured in kW/m².

$\overline{G_{T,STC}}$: represents the incident irradiation in kW/m² under standard test conditions.

${\alpha }_P$: stands for the temperature coefficient of power, represented as a percentage change per degree Celsius (%/°C).

$T_c$: indicates the PV cell temperature, measured in degrees Celsius (°C).

$T_{c,STC}$: denotes the temperature of the PV cell under standard test circumstances, expressed in degrees Celsius (°C).

The temperature of a PV module, a critical factor affecting its electrical characteristics, exhibits a strong correlation with environmental conditions. While at night, it aligns with the ambient temperature, but in sunlight, the PV temperature can surpass the ambient temperature by 30 °C or above. As such, estimating the PV module temperature becomes crucial. The steady-state temperature of the PV panel/module can be estimated using Equation (2) [50], which takes into account factors such as the ambient temperature ($T_a$), the nominal operating cell temperature (NOCT), the solar irradiance at which NOCT is symbolized as ($G_{T,NOCT}$), and the ambient temperature at which NOCT is symbolized as ($T_{a,NOCT}$), as well as maximum power efficiencies under standard test conditions (${\eta }_{mp,STC}$). This equation aids in determining the PV module’s temperature under different environmental circumstances, ensuring accurate assessments of its performance.

$$T_c=\frac{T_a+\left(NOCT-T_{a,NOCT}\right)\left(\frac{\overline{G_T}}{G_{T,NOCT}}\right)\left[1-\frac{{\eta }_{mp}}{0.9}\right]}{1+\left(NOCT-T_{a,NOCT}\right)\left(\frac{\overline{G_T}}{G_{T,NOCT}}\right)\left(\frac{{{\alpha }_{P }\eta }_{mp,STC}}{0.9}\right)}$$

(2)

The maximum power point efficiency (${\eta }_{mp}$) is determined by the following formula:

$${\eta }_{mp}={\eta }_{mp,STC}\left(1-{\alpha }_P{T}_{c,STC}\right)$$

(3)

Hence, when the solar irradiance incident on the PV panel’s surface and the PV panel/module temperature are established, the hourly PV output power generated can be computed using Equation (1). This equation facilitates the determination of the PV system’s power output, considering these critical parameters.

2.2.2. Wind Turbine System Model

Three characteristic speeds—cut-in ($V_c$), rated ($V_r$), and cutoff ($V_f$)—as well as rated power ($P_r$) serve as the foundation for wind turbine models. The output power of wind turbines can be estimated using a variety of models, including quadratic, linear, and Weibull parameter-based models. A quadratic model is utilized in this study to approximate the wind turbine output power captured [49]. A wind turbine’s P_WT output power captured can be computed as follows:

$$P_{WT}(V)=P_r\times \left\{\begin{array}{c}0\\ \frac{V^2-V_c^2}{V_r^2-V_c^2}\\ 1\end{array}\right. \begin{array}{c} V<V_c or V>V_f\\ V_c\le V\le V_f\\ V_r\le V\le V_f\end{array}$$

(4)

In cases where data are typically available at standard heights like 10 m or 20 m, estimations for other heights are necessary based on measured wind speeds. One powerful method is the power law, articulated by equation [49]:

$$V_{hub}{=V}_{data}{\left(\frac{Z_{hub}}{Z_{data}}\right)}^{\alpha }$$

(5)

Here, $Z_{data}$ represents the measurement height, $Z_{hub}$ denotes the hub height at which wind speed is required, and $\alpha$ signifies the power law exponent. This formula is instrumental in estimating wind speeds at different heights using available data at specific standard heights.

2.2.3. Modeling of Diesel Generator

The diesel generator functions to cover the load when the collective energy produced from renewable sources (such as PV, wind turbine, and battery) is insufficient to meet the load requirement. The cost of diesel fuel for the generator is computed using the subsequent equation [50]:

$$C_{DG}=C_F\sum _{t=1}^{8760}F\left(t\right)$$

(6)

Here, $F\left(t\right)$ represents the hourly fuel consumption in liters per hour (L/h), contingent upon the load characteristics of the diesel generator, formulated as follows [50]:

$$F\left(t\right)=A\times P_{DG}\left(t\right)+B\times P_R$$

(7)

where:

$P_R$: the diesel generator’s rated power, measured in kilowatts (kW).

$P_{Diesel}$: the power produced by the diesel generator for a specific hour, measured in kilowatts (kW).

$C_F$: the fuel cost per liter, denoted in USD per litre (USD/L).

Coefficients A and B: these coefficients, representing the fuel curve, are set at A = 0.246 L/kWh and B = 0.0845 L/kWh, respectively.

The fuel consumption of a diesel generator is significantly influenced by both the rated power and the actual generated power. Consequently, it is imperative for the diesel generator to operate within a specific power range, avoiding operation at its minimum setting. Typically, diesel generator manufacturers provide guidance regarding the minimum operating setting. This operational range of the diesel generator is expressed mathematically as follows:

$$P_{Diesel,min}\le P_{Diesel}\left(t\right)\le P_{Diesel,max}$$

(8)

In the context of the proposed HES, an assessment of the total carbon dioxide (CO₂) emissions is conducted across the system’s annual operation. This evaluation involves a comparative analysis between the emissions generated by the proposed HES and those stemming from a scenario where the load demand is solely met by a diesel generator. The calculation of the total CO₂ emissions resulting from the fuel combustion of the generator involves considering the emission factor of CO₂ per kilowatt-hour (kWh) of electricity generated by the diesel generator. For this research, the emission factor is established at 0.699 kg of CO₂ emitted per kWh of electricity generated by the generator [51].

2.2.4. Battery Bank Model

Batteries serve as a critical component in the HES due to the sporadic nature of wind and solar power, influenced by varying weather conditions and the time of day. Battery storage is employed to bridge the energy supply–demand mismatch that may arise from these fluctuations. Selecting an appropriate battery size for the microgrid/HES necessitates a comprehensive analysis, considering various factors, including the maximum depth of discharge, temperature effects, nominal battery capacity, and battery lifetime considerations.

In this research, certain battery efficiency parameters are defined: The round-trip efficiency is used to determine the battery’s charge efficiency, while the discharge efficiency is considered as 1. Based on the operational mode of the system (charging or discharging), the state of charge (SOC) of the battery bank can be estimated using the following formulas [50]:

▪

Charging Mode (when energy generation exceeds load demand):

$$SOC(t)=SOC(t-1)(1-\sigma )+\left(E_{GA}(t)-\frac{E_L\left(t\right)}{{\eta }_{inv}}\right) {\eta }_{battery}$$

(9)

▪

Discharging Mode (when load demand exceeds energy generation):

$$SOC\left(t\right)=SOC\left(t-1\right)\left(1-\sigma \right)+\left(\frac{E_L\left(t\right)}{{\eta }_{inv}}-E_{GA}\left(t\right)\right)$$

(10)

where:

$SOC\left(t\right)$ and $SOC\left(t-1\right)$: represent the state of charge of the battery at the current time, t, and the previous time (t − 1), respectively.

$\sigma$: denotes the hourly rate of self-discharging, indicating the rate at which the battery loses charge when not in use.

$E_{GA}\left(t\right)$: signifies the total energy captured at time t by the renewable energy sources in the microgrid/HES.

$E_L\left(t\right)$: corresponds to the load demand at time t, i.e., the electricity required by the system at a specific moment.

${\eta }_{inv}$: represents the efficiency of the inverter used within the HES.

${\eta }_{battery}$: represents the battery bank’s charging efficiency, indicating how effectively the battery stores and releases energy.

To maintain the battery within safe operating conditions, two key constraints are imposed:

▪

SOC(t) should not fall below a minimum permissible energy level, denoted as ${SOC}_{min}$. This level ensures that there is always a certain minimum energy reserve within the battery bank.

▪

During the charging process, SOC(t) should not exceed a maximum permissible energy level, designated as ${SOC}_{max}$. This limit is often equivalent to the nominal capacity of the battery bank ($C_{Batt}$), ensuring that the battery does not become overcharged and damaged.

The relationship between ${SOC}_{min}$ and ${SOC}_{max}$ can be defined as the maximum depth of discharge (DOD). Specifically, ${SOC}_{min}$ is calculated as (1 − DOD)$C_{Batt}$, where $C_{Batt}$ is the battery bank’s nominal capacity. This relationship helps ensure that the battery operates within the specified range and longevity limits.

2.2.5. System Reliability Model

The LPSP serves as a key metric for evaluating the reliability of power systems within the HES. It seeks to account for the inherent stochastic characteristics of renewable energy sources, which exert a substantial influence on the dynamics of energy production. LPSP is crucial in the design phase to ensure system reliability. The LPSP is based on specific methodologies, summarizing the steps as follows:

▪

Surplus Energy Storage: if the energy production from renewables exceeds the current load demand, the excess energy is stored in the batteries unit, recalculating the state of charge (SOC) based on Equation (9) until it reaches the maximum limit.

▪

Load Demand Exceeds Renewable Generation: When the load demand surpasses the available energy from renewable energy sources, the battery bank is used to make up the difference. In such cases, SOC is recalculated using Equation (10).

At maximum SOC, the charging process halts as managed by the energy management system. The LPSP over a specific period T (typically a year in this context) is determined by the equation [49]:

$$LPSP=\frac{{\sum }_{t=0}^{T}Power Failure Time\left(PFT\right)}{T}$$

(11)

Power Failure Time (PFT) represents the duration when the load remains unsatisfied due to insufficient power generation from the HES, providing a measure of system reliability in handling energy demands.

2.2.6. Renewable Energy Fraction Model

The REF is a crucial metric used to quantify the proportion of energy delivered to the load that is generated from renewable sources. It elucidates the system’s reliance on renewable energy. The REF is calculated as follows [50]:

$$REF=\left(1-\frac{L_{DG}}{T_{Load}}\right)\times 100$$

(12)

Here, $L_{DG}$ denotes the percentage of load served by the diesel generator in kilowatt-hours (kWh). An REF value of 100% indicates a purely renewable energy system, while an REF of 0% signifies a system reliant solely on diesel power. REF values between these extremes reflect the microgrid/HES, representing the proportion of renewable energy integrated into the overall energy supply.

2.2.7. System Economics Model

The cost of energy (COE) is an essential metric for evaluating system cost analysis within an HES. It represents the average cost per kWh of useful electrical energy generated. The COE can be determined using the following formula [49]:

$$COE=\frac{\mathrm{A}\mathrm{S}\mathrm{C}}{T_{Load}}$$

(13)

Here, the total annualized cost of the system in USD per year is referred to as the ASC. The term $T_{Load}$ indicates the total electrical load served by the system over the course of a year, measured in kilowatt-hours per year. The annualized system cost (ASC) comprises three main components: the annualized capital cost ($C_{acap}$), the annualized replacement cost ($C_{arep}$), and the annualized maintenance cost ($C_{amain}$), which has been formulated as follows:

$$ASC=C_{acap}+C_{arep}+C_{amain}$$

(14)

The determination of the annualized replacement cost, the annualized capital cost, and the O&M cost is a crucial component for the overall system cost analysis in the HES. These costs are estimated based on the initial capital investments, system lifetime, and the unit costs of the system elements.

The $C_{acap}$ for each system component, including the PV array, wind turbine, battery banks, and diesel generator, can be determined using the formula:

$$C_{acap}=C_{cap}CRF\left(i,Y_{proj}\right)$$

(15)

Here, $C_{cap}$ represents the initial capital cost of each component in USD, while the capital recovery factor (CRF) accounts for the system life period in years ($Y_{proj}$).

The $C_{cap}$ is determined by the sum of the product of the total capacity ($E_x$) and the unit cost ($C_{unit,x}$) for each system component:

$$C_{cap}=\left(E_{PV}{\times C}_{unit,PV}\right)+\left(E_{WT}{\times C}_{unit,WT}\right)+\left(E_{Batt}{\times C}_{unit,Batt}\right)+\left(E_{DG}{\times C}_{unit,DG}\right)$$

(16)

The $C_{arep}$ represents the annualized value for all replacement costs incurred throughout the project’s lifetime, and it can be calculated as:

$$C_{arep}=C_{rep}SSF\left(i,Y_{rep}\right)$$

(17)

Here, $C_{rep}$ denotes the replacement cost of the component (USD), while the sinking fund factor (SSF) is determined by the system’s lifetime ($Y_{rep}$).

The following details provide comprehensive insights into the economic aspects and expected lifetime of the HES components. The components, including PV modules, wind turbines, battery banks, inverters, and diesel generators, feature initial costs, potential replacement expenses, and operational and maintenance costs, as well as estimated lifespans and interest rates. The photovoltaic modules are originally priced at USD 1000/kW, with a negligible O&M cost of USD 20 per kW, operating over a 20-year lifespan at a 2% interest rate. The wind turbines initially cost USD 1300 per kW, with a potential O&M cost of USD 30.33 per kW. Similarly, the battery banks start at USD 200 per kilowatt-hour (kWh), featuring O&M costs of USD 5 per kWh and operating across a 10-year span. The inverter shows an initial cost of USD 133 per kW, with a 10-year lifespan and a 2% interest rate. The diesel generator is priced at USD 300 per kW, involving an identical replacement cost, an O&M expense of USD 0.012 per kWh, and a 10-year lifespan.

3. The Framework and Implementation of RLNNA

The RLNNA framework encompasses five main phases, as shown in Figure 6, that have been discussed as follows:

3.1. Generating the Trial Population

In RLNNA, the trial population is mathematically formulated as follows:

$$x_{w,j}^t=\sum _{i=1}^N{w}_{i,j}^t{x}_i^t i=2,3,\dots N, j=2,3,\dots N$$

(18)

$$v_i^t=x_i^t+x_{w,i}^t i=2,3,\dots N$$

(19)

where N represents the population size, t denotes the current iteration, $x_i^t$ refers to the ith individual of the population $X^t=\{x_1^t,x_2^t,\dots ,x_N^t\}$ at time t, $v_i^t$ signifies the ith trial individual of trial population $v_i^t=\{v_1^t,v_2^t,\dots v_N^t\}$ at time t, and $w_{i,j}^t$ represents the jth weight value of the ith individual at time t. The weight matrix $W^t=\{v_1^t,v_2^t,\dots ,v_N^t\}$ complies with the following condition:

$$\sum _{i=1}^N{w}_{i,j}^t{x}_i^t=1 0<w_{i,j}^t<1$$

(20)

3.2. Updating the Weight Matrix

The weight matrix, instrumental in generating the trial population, has been updated by the equation:

$$w_i^{t+1}=\left|w_i^t+2{\lambda }_1(w_{Best}^t+w_i^t)\right| i=2,3,\dots N$$

(21)

where ${\lambda }_1$ is a random integer number uniformly distributed in the range of 0 to 1, and $w_{Best}^t$ represents the target weight vector.

3.3. Bias Operator

The role of the modification factor, denoted as ${\beta }_i^t$, within RLNNA is crucial in balancing exploration and exploitation. The principle behind this factor lies in the identification of outstanding individuals, those who secure better solutions within an iteration. These exceptional individuals provide essential insights into approaching the global optimal solution. The strategy is to encourage these outstanding individuals to focus on their vicinity by engaging in a search operator around themselves, which is known as local exploitation. This approach is governed by a designed rule within RLNNA that leverages the fitness values obtained from each individual within an iteration.

$${\beta }_i^{t+1}={\beta }_i^t+{\beta }_i^t\tau \left(g(h\left(x\right)+g(s\left(x\right)\right) i=2,3,\dots N$$

(22)

where τ represents the penalty factor, $h\left(x\right)$ and $s\left(x\right)$ signify two difference functions, and $g\left(x\right)$ denotes a control function.

$h\left(x\right)$, $s\left(x\right)$, and $g\left(x\right)$ can be expressed as:

$$h\left(x\right)=f_{i,1}^{t+1}-f_i^t$$

(23)

$$s\left(x\right)=f_{i,2}^{t+1}-\mathrm{m}\mathrm{i}\mathrm{n}\left(f_{i,1}^{t+1},f_i^t\right)$$

(24)

$$g\left(x\right)=\left\{\genfrac{}{}{0pt}{}{ -1 \mathrm{i}\mathrm{f} x<0 (\mathrm{A}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{v} \mathrm{a}\mathrm{t}\mathrm{e}\mathrm{a}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n})}{0 \mathrm{i}\mathrm{f} x\ge 0 (\mathrm{K}\mathrm{e}\mathrm{e}\mathrm{p} \mathrm{a}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n})}\right.$$

(25)

where $f_i^t$ represents the fitness value of the ith individual at a given time t. Meanwhile, $f_{i,1}^{t+1}$ and $f_{i,2}^{t+1}$ denote the fitness values of the ith individual at time t + 1, derived from the first and second function evaluations, respectively. Triggered by this information, two potential actions emerge: the “keep action” and the “activate action”. The “keep action” prompts an individual to maintain the same modification factor used in the previous iteration for their subsequent search. Conversely, the “activate action” instigates the adoption of a new modification factor for the next search. In RLNNA’s dynamic, outstanding individuals—those securing superior solutions within the iteration—will cause the “activate action” to begin. Consequently, their iteration-by-iteration modification factor will decrease, raising the probability that the transfer operator (exploitation) will be executed in the next iteration. A visual representation of the procedure for updating the modification factor is provided in Figure 7.

The bias operator in RLNNA comprises both a bias population and a bias weight matrix. The bias population encompasses a random integer $N_R$ and a set $\theta$ with $N_R$ elements. Variables $l=\{l_1,l_2\dots l_D\}$ and $u=\left\{u_1,u_2\dots u_D\right\}$ show the variables’ lower and upper bounds, respectively, with D being the variables number. Furthermore, $N_p$ equates to (βt × D), and $\theta$ encompasses $N_p$ integer random numbers chosen from 1 to D. One way to express the bias population is as follows:

$$v_{i,\theta \left(k\right)}^t=l_{\theta \left(k\right)}+{\lambda }_2(u_{\theta \left(k\right)}-l_{\theta \left(k\right)}) k=1,2,\dots N_R$$

(26)

Here, ${\lambda }_2$ is a uniformly distributed, randomly generated number between 0 and 1.

Similarly, the bias matrix involves two inputs: a set ϑ comprising $N_w$ elements and a random integer $N_w$. The definition of the bias weight matrix is:

$$w_{i,\mathsf{\vartheta }\left(k\right)}^t=\xi , k=1, 2,\dots N_R$$

(27)

ξ represents a randomly generated number in the range of 0 to 1.

3.4. Transfer Operator

The transfer operator serves as a mechanism for exploitation within RLNNA, helping to identify superior solutions within the current search space. Comprising a transfer period from the past as well as one from the present, it is represented as:

$$v_i^t=v_i^t+\underset{Current transfer term}{\underbrace{\left|k_1\right|(x_{Best}^t-v_i^t)}}+\underset{Historical transfer term}{\underbrace{\left|k_2\right|(x_{Best}^t-x_{old,i}^t)}}$$

(28)

Here, $k_1$ and $k_2$ are random numbers drawn from a standard normal distribution. The purpose of these random values is to extend the search space, aiding in avoiding local optimal solutions.

The vector $x_{old,i}^t$ represents the ith individual in the historical population $x_{old}^t={\{x}_{old,1}^t,x_{old,2}^t,\dots x_{old,N}^t\}$ at time t, and it is updated according to the following condition:

$$x_{old}^t=\left\{\begin{array}{c}{X}^t, \mathrm{i}\mathrm{f} {\lambda }_2<0.5\hfill \\ X_{old}^t, \mathrm{O}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\hfill \end{array}\right.$$

(29)

Here, ${\lambda }_2$ is uniformly distributed and represents a random integer number in the range of 0 to 1. The final version of $x_{old}^t$ is collected by a permuting function, $\phi$(·), which sorts the vectors randomly.

$$x_{old}^t=\phi \left(x_{old}^t\right)$$

(30)

The population initialization within RLNNA is performed by:

$$x_i^0=l+(u-l){\lambda }_3 i=2,3,\dots N$$

(31)

where ${\lambda }_3$ is a random integer number in the range of 0 to 1, uniformly distributed.

Following the completion of the transfer operator, a selection operator is executed:

$$v_i^t=\left\{\begin{array}{c}{v}_i^t, \mathrm{i}\mathrm{f} f\left(v_i^t\right)<f\left(x_i^t\right) \hfill \\ x_i^t, \mathrm{O}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\hfill \end{array}\right.$$

(32)

3.5. Feedback Operator

The feedback operator, intended to hasten convergence, further optimizes the trial population obtained via the transfer operator:

$$v_i^{t+1}=\left\{\genfrac{}{}{0pt}{}{v_i^t+\underset{Current transfer term}{\underbrace{\left|k_3\right|\left(v_m^t-v_i^t\right)}}+\underset{Historical transfer term}{\underbrace{\left|k_4\right|\left(x_{old,i}^t-v_i^t\right)}}, \mathrm{i}\mathrm{f} f\left(v_m^t\right)<f\left(v_i^t\right)}{v_i^t+\underset{Current transfer term}{\underbrace{\left|k_3\right|\left(v_i^t-v_m^t\right)}}+\underset{Historical transfer term}{\underbrace{\left|k_4\right|\left(x_{old,i}^t-v_i^t\right)}}, \mathrm{O}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}}\right.$$

(33)

Here, $k_3$ and $k_4$ are random numbers drawn from the normal distribution. m represents a random integer number in the range of 1 to N, while $f_m^t$ signifies the mth individual fitness. The feedback operator offers two primary advantages. It takes into consideration the local feedback in addition to current feedback. The strong randomness of the historical feedback term contributes to enhanced population diversity. Finally, following the completion of the feedback operator, a selection operator is executed:

$$x_i^{t+1}=\left\{\begin{array}{c}{v}_i^{t+1}, \mathrm{i}\mathrm{f} f\left(v_i^{t+1}\right)<f\left(v_i^t\right) \hfill \\ v_i^t, \mathrm{O}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\hfill \end{array}\right.$$

(34)

Figure 8 displays the pseudo-code for the RLNNA technique.

4. Simulation Results, Discussion, and Analysis

The RLNNA approach is introduced as a novel technique for optimizing the sizing of a microgrid aimed at supplying a remote region in Saudi Arabia. The proposed RLNNA algorithm leverages the hourly wind speed and solar radiation, offering a unique perspective on HES optimization. The HES under consideration encompasses PV arrays, wind turbine generators (WTs), batteries, and diesel generators, reflecting a comprehensive system design for off-grid energy supply. To assess the performance of the RLNNA algorithm, a thorough comparison was conducted with five contemporary artificial intelligence (AI)-based optimization techniques: GA, PSO, SDO, NNA, and MRFO. The effectiveness of these algorithms was evaluated using four performance indices: the mean, the standard deviation (STD), the optimal/ideal solution, and the worst solution. Key parameters for each algorithm, such as the maximum number of iterations and independent runs number, were standardized at 500 and 50, respectively, for consistency across all evaluations.

Table 3. The primary characteristics/parameters of the six AI-based algorithms.

Technique	Characteristic	Value
Genetic Algorithm (GA)	Population	100
	Selection	Roulette wheel
	Rate of mutation	0.2
	Rate of crossover	0.8
Particle Swarm Optimizer (PSO)	Swarm No.	100
	Inertia weight	1
	Cognitive constant	0.25
SDO	Supply weight	2
	Demand weight	2
RLNNA	Beta	1
	Bias reduction	0.99
NNA	Beta	1
	Bias reduction	0.99
MRFO	Constant parameter	α = 1
	Somersault factor	2
	Random numbers r1, r2, r3	[1 0]

lists the primary characteristics of every AI-based method that was employed in this study. The implementation of all soft-computing algorithms was carried out using MATLAB R2019b on a Windows 10/64-bit platform.

Table 4. Performance indicators.

Algorithm	Performance Indicators (USD/Year)				Elapsed Time (s)
	Optimal	Worst	Mean	STD.
GA	1,232,654.5	1,280,428.9	1,247,373.6	10,504.0	27.9
PSO	1,222,098.2	1,227,140.5	1,222,181.4	2081.2	41.9
SDO	1,222,098.2	1,222,457.7	1,222,166.2	68.20	47.6
RLNNA	1,219,744.0	1,222,465.4	1,221,659.2	112.8	62.9
NNA	1,222,319.5	1,230,897.2	1,224,626.9	2388.4	27.4
MRFO	1,222,098.4	1,229,389.3	1,222,770.8	1485.6	44.8

shows a comprehensive analysis of six intelligent soft-computing algorithms—GA, PSO, SDO, RLNNA, NNA, and MRFO. For validity purposes, RLNNA was compared to the other five soft-computing algorithms. The focus of this analysis is the optimal sizing of a microgrid/HES required for supplying a residential load in the Kingdom of Saudi Arabia. The concluding remarks collected from this table can be summarized as follows:

▪

RLNNA: Excels with the lowest optimal cost of USD 1,219,744.0, indicating superior performance. While the mean cost of USD 1,221,659.2 is competitive, a slightly higher STD of 112.8 suggests some variability. RLNNA, however, requires a longer elapsed time of 62.9 s.

▪

MRFO: Performs competitively with an optimal cost of USD 1,222,098.4, akin to PSO and SDO. The mean cost of USD 1,222,770.8 demonstrates consistent performance, and the low STD of 1,485.6 signifies reliability. MRFO operates efficiently, with an elapsed time of 44.8 s.

▪

PSO: Yields an optimal cost of USD 1,222,098.2 per year, showcasing effectiveness in HES sizing. The mean cost of USD 1,222,181.4 indicates stability, and the low STD of USD 2,081.2 suggests reliability. PSO demonstrates moderate computational efficiency, with an elapsed time of 41.9 s.

▪

SDO: Mirrors PSO in achieving an optimal cost of USD 1,222,098.2, emphasizing consistent performance. The mean cost of USD 1,222,166.2 and low STD of 68.20 signify reliability in sizing outcomes. SDO exhibits a slightly longer elapsed time of 47.6 s.

▪

NNA: Achieves an optimal cost of USD 1,222,319.5, demonstrating effectiveness in HES sizing. The mean cost of USD 1,224,626.9 suggests stable performance, while a higher STD of 2,388.4 indicates some variability. NNA operates efficiently, with an elapsed time of 27.4 s.

▪

GA: Demonstrates competitive performance with an optimal cost of USD 1,232,654.5 per year. The algorithm exhibits a mean cost of USD 1,247,373.6, indicating consistent performance. However, a relatively high standard deviation (STD) of USD 10,504.0 suggests variability. Notably, GA operates efficiently, with an elapsed time of 27.9 s.

In summary, RLNNA stands out with the lowest optimal cost, while NNA and MRFO demonstrate effectiveness with competitive results and computational efficiency. The choice of algorithm may depend on the specific priorities, such as minimizing costs or achieving a balance between cost optimization and computational speed, in the context of HES design for isolated areas in Saudi Arabia.

Figure 9 illustrates the convergence rates of the six AI-based techniques—GA, PSO, SDO, NNA, MRFO, and RLNNA. The findings depicted in the figure reveal distinct patterns among the algorithms in terms of their convergence behavior. It is evident from the figure that MRFO, PSO, and SDO exhibit the capability of tracking the optimal global solution. Remarkably, RLNNA demonstrates the fastest convergence among all the algorithms, followed by PSO and SDO. This observation highlights the efficiency of RLNNA in swiftly reaching the optimal solution. On the contrary, GA, and NNA are shown to face challenges in avoiding local solutions and exhibit longer convergence times. The figure indicates that these algorithms may become trapped in local optima, leading to delayed convergence. In conclusion, the analysis of Figure 9 indicates that the RLNNA method performs better than the other five AI techniques for convergence time and global solution capture. The faster convergence of RLNNA, coupled with its capability to follow the optimal global solution, positions it as a promising and effective algorithm for addressing the optimization challenges in the context of the presented computational intelligence-based algorithms.

Table 5. Optimized parameters generated by the six intelligence-based algorithms.

Algorithm	PSolarPV (kW)	PWind (kW)	PBattery (kW)	PDG (kW)	REF (%)	ASC (USD/Year)	Surplus (%)	Fuel Cost (USD/Year)
GA	3217.1	3051.47	8257.17	1697.9	87.11	1,232,654.5	32.2	257,866.9
PSO	2990.8	3025.03	8290.23	1858.9	86.37	1,222,098.2	28.0	280,896.4
SDO	2990.8	3025.03	8290.23	1858.8	86.33	1,222,098.2	28.0	280,745.4
RLNNA	2990.8	3025.04	8290.23	1858.8	86.37	1,219,744.0	28.1	280,745.5
NNA	2989.3	3023.27	8296.33	1858.1	86.36	1,222,319.5	28.0	281,080.6
MRFO	2990.8	3025.02	8290.26	1858.9	86.41	1,222,098.4	28.1	281,015.5

provides a detailed comparison of the optimized parameters generated by different techniques for the microgrid/HES design. The considered AI algorithms include GA, PSO, SDO, RLNNA, NNA, and MRFO. Several key observations highlight the efficacy of RLNNA in comparison to the other AI algorithms:

▪

Superior Economic Optimization:

RLNNA (USD 1,219,744.0) achieves the lowest annualized system cost, showcasing its superior ability to optimize economic parameters.

Versatility: RLNNA demonstrates adaptability in balancing economic feasibility while considering renewable energy integration.

▪

Competitive Renewable Energy Fraction:

RLNNA (86.37%) maintains a competitive REF, emphasizing the algorithm’s ability to balance renewable energy utilization while minimizing costs.

Integrated Approach: RLNNA excels not only in economic optimization but also in considering the ecological impact through effective renewable integration.

▪

Efficient Convergence and Solution Capture:

Convergence Rate: RLNNA exhibits efficient convergence, minimizing the time required to reach optimal solutions.

Global Solution Capture: RLNNA demonstrates effectiveness in capturing global solutions, ensuring well-balanced and reliable hybrid energy system designs.

▪

Algorithmic Robustness:

RLNNA consistently outperforms in achieving the dual objectives of economic optimization and renewable energy integration.

Table 6. Optimized parameters of four HESs generated by the RLNNA algorithm.

Configuration	PPV (kW)	PWT (kW)	PBatt (kWh)	PDG (kW)	REF (%)	ASC (USD/Year)	Surplus (%)	Fuel Cost (USD/Year)
PV/Battery/Diesel	5139	0	18661.7	1765	83.9	1,494,367.8	24.9	312,195.5
Wind/Battery/Diesel	0	4970.5	3676.2	2182	65.1	1,593,713.7	43.0	724,655.7
PV/Wind/Diesel	3100.6	3863.8	0	1941.8	78.2	1,454,777.1	77.4	547,018.8
PV/Wind/Battery/Diesel	2990.8	3025.04	8290.23	1858.8	86.37	1,219,744.0	28.1	280,745.5

presents a comprehensive comparison of optimized parameters derived from the RLNNA algorithm for various hybrid energy system configurations. In the PV/battery/diesel scenario, the power generation is distributed with 5139 kW from PV and 1765 kW from a diesel generator. The system achieves an 83.9% REF with an ASC of USD 1,494,367.8, a surplus energy of 24.9%, and a fuel cost of USD 312,195.5. Conversely, the wind/battery/diesel configuration relies on 4970.5 kW from wind, a battery capacity of 3676.2 kWh, and 2182 kW from a diesel generator. This setup achieves an REF of 65.1%, an ASC of USD 1,593,713.7, a notable surplus energy of 43.0%, and a higher fuel cost of USD 724,655.7. In the PV/wind/diesel scenario, the system incorporates 3100.6 kW from PV, 3863.8 kW from wind, and 1941.8 kW from a diesel generator, resulting in an REF of 78.2%, an ASC of USD 1,454,777.1, a surplus energy of 77.4%, and a fuel cost of USD 547,018.8. Finally, the PV/wind/battery/diesel configuration combines 2990.8 kW from PV, 3025.04 kW from wind, 8290.23 kWh in battery capacity, and 1858.8 kW from a diesel generator, yielding the highest REF at 86.37%, an ASC of USD 1,219,744.0, a surplus energy of 28.1%, and a fuel cost of USD 280,745.5. In comparing these configurations, the PV/wind/battery/diesel system stands out as a well-balanced option, offering high renewable energy utilization, reasonable surplus energy, and competitive costs. These results offer insights into the trade-offs between renewable energy integration, system costs, and surplus energy for each configuration, aiding decision making based on specific objectives and constraints.

Figure 10 offers crucial insights into the behavior and effectiveness of RLNNA when compared to GA, MRFO, NNA, PSO, and SDO in optimizing battery bank capacities within the examined hybrid energy system (HES).

▪

Competitive Performance: RLNNA, NNA, and PSO exhibit comparable and competitive performance, providing viable alternatives for optimization.

▪

Consistency: SDO, along with MRFO, emerges as a consistently robust performer, showcasing reliable convergence and optimal solutions across multiple runs.

▪

Efficiency: GA displays higher variability and may require further tuning for improved convergence and consistency.

Figure 11 shows the optimal capacities of photovoltaic, wind turbine generators, and diesel in RLNNA when compared to GA, MRFO, NNA, PSO, and SDO over 50 simulation runs. Across these runs, the PV and WT capacities exhibit slight variations, indicating the RLNNA’s ability to consistently suggest optimal solutions. The battery capacity remains relatively stable at approximately 8298.34 kWh, demonstrating the algorithm’s consistent sizing recommendation for the battery bank. Similarly, the diesel generator capacity shows minor fluctuations, ranging between 1857.36 kWh and 1869.03 kWh. These findings collectively highlight the stability and reliability of the RLNNA algorithm in determining optimal component capacities for a hybrid energy system across diverse scenarios.

5. Conclusions

This paper delves into the optimization of microgrid/HESs with a particular focus on sizing configurations that incorporate photovoltaic, wind turbine generators, battery banks, and diesel generators. This study employs a novel soft-computing/metaheuristic algorithm, the reinforcement learning neural network algorithm (RLNNA), to minimize the ASC and enhance system reliability, catering to the energy needs of off-grid areas in Saudi Arabia. The RLNNA algorithm, benchmarked against several other soft-computing algorithms, has demonstrated superior performance regarding faster convergence, capturing optimal global solutions, and minimizing ASC. The results, derived from 50 runs of RLNNA optimization, provide valuable insights into the optimal capacities of key components within HESs. Over the course of the experimentation, RLNNA consistently demonstrated its superior performance, achieving optimal capacities for photovoltaic, wind turbine generators, battery banks, and diesel generators. The comprehensive comparison of algorithms reveals that RLNNA outperforms others regarding both convergence time and global solution capture. Specifically, RLNNA achieves the lowest ASC at USD 1,219,744, showcasing its effectiveness in addressing economic considerations and optimizing system performance. As civilization transitions towards a future dependent on renewable energy, the optimization of HES configurations assumes a more critical role. The RLNNA algorithm, as evidenced by this study, stands as a promising tool for researchers and practitioners alike, offering enhanced efficiency and reliability in the pursuit of sustainable and economically viable off-grid energy solutions. The findings presented herein contribute to the growing body of knowledge in the field, providing a foundation for further exploration and advancement in the optimization of renewable energy conversion systems. For further endeavors, the potential benefits of hybridizing RLNNA with other optimization algorithms should be investigated. Combining the strengths of RLNNA with complementary techniques may yield improved results, especially in addressing multi-objective optimization. This approach could enhance the algorithm’s versatility and robustness across different HES configurations and operational scenarios. The exploration of hybridization strategies presents an exciting avenue for advancing the state of the art in renewable energy system optimization.