Physics-Informed Deep Reinforcement Learning for Compact VBT Farms: Integration, Power Quality, and Economics ^†

Abstract

This paper presents a physics-informed Deep Q-Network (DQN) framework for optimizing the deployment of 100 vortex bladeless turbines (VBTs) in a Saharan microgrid. The proposed approach integrates wake interaction modeling, land-use constraints, techno-economic factors, and power quality (PQ) indicators at the point of common coupling. The novelty lies in coupling aerodynamic modeling with reinforcement learning and grid constraints. Results show that dense layouts (≤400 m²) yield up to 41% gains but degrade PQ (Pst > 1.0, THD > 5%). An optimal range of 500–800 m² achieves stable performance with moderate gains (6–9%) and acceptable PQ. Larger surfaces (>1000 m²) show limited benefits (<4%). The framework supports efficient and sustainable wind deployment in constrained environments.

1. Introduction

The transition toward decentralized and sustainable energy systems requires compact and efficient solutions capable of operating under constrained land conditions. In this context, Vortex Bladeless Turbines (VBTs) have emerged as a promising alternative to conventional horizontal-axis wind turbines. Based on vortex-induced vibrations (VIV), these systems convert oscillatory motion into electrical energy, eliminating rotating blades and reducing mechanical complexity, maintenance requirements, and environmental impact. Recent studies have confirmed the potential of VBT systems to achieve simplified design with acceptable energy performance under realistic operating conditions [1,2,3].

VBTs are particularly suitable for deployment in dense or constrained environments such as urban areas and isolated microgrids. However, when deployed in compact configurations, strong wake interactions may significantly affect their performance. Previous studies have addressed this issue using analytical wake models and metaheuristic optimization techniques such as Ant Colony Optimization (ACO) and wake-loss minimization approaches [4,5], identifying an optimal surface range between 500 and 800 m² for balancing energy production and land use.

Despite these advances, conventional optimization approaches remain limited by their static nature and lack of adaptability to dynamic operating conditions. To overcome these limitations, this paper proposes a Deep Reinforcement Learning approach based on the Deep Q-Network (DQN) algorithm. The proposed framework enables an intelligent agent to iteratively learn optimal turbine layouts by considering aerodynamic interactions, spatial constraints, and techno-economic factors. Reinforcement learning approaches have recently demonstrated strong performance in complex energy systems and microgrid management problems characterized by nonlinear dynamics and uncertainty [6,7,8,9].

In addition, the integration of VBT systems into electrical networks raises power quality challenges due to the oscillatory nature of VIV-based generation. This work evaluates the impact of turbine layout on key power quality indicators, including harmonic distortion and flicker at the point of common coupling. These issues are consistent with recent findings on renewable-rich microgrids, where power quality degradation remains a critical challenge for stable grid operation [10,11,12]. A comparative analysis between random and optimized configurations is conducted within a realistic microgrid context in Boujdour, Morocco, representative of hybrid renewable systems deployed in remote and constrained environments [13,14,15].

The main contributions of this work are threefold: (i) the development of a physics-informed DQN framework for compact VBT layout optimization, (ii) the integration of aerodynamic, electrical, and economic constraints into a unified model, and (iii) the identification of an optimal deployment range ensuring both energy efficiency and grid compatibility.

The physics-informed aspect of the proposed framework lies in embedding aerodynamic wake interactions and power quality constraints directly into the state representation and reward function, ensuring that the learning process remains consistent with physical and grid-level operational constraints.

2. System Modeling and Environment

2.1. Vortex Bladeless Turbine Modeling and VIV Mechanism

Vortex Bladeless Turbines (VBTs) exploit vortex-induced vibrations (VIV) to convert wind energy into electricity through oscillatory motion rather than rotational dynamics. When wind flows around a cylindrical mast, alternating vortex shedding generates periodic transverse forces, producing structural oscillations. This phenomenon, known as a von Kármán vortex street, governs the energy conversion process.

The vortex shedding frequency $f_s$ is defined by the Strouhal relation:

$$f_s={\displaystyle \frac{s_t\cdot U}{D}}$$

(1)

where U is the wind speed, D the mast diameter, and $S_t$ ≈ 0.2 represents the Strouhal number for circular cylinders under subcritical flow conditions [16]. This relation governs vortex shedding dynamics and directly links flow velocity to excitation frequency.

Maximum energy transfer occurs under the lock-in condition when $f_s\approx f_n$, leading to resonance between fluid excitation and structural dynamics. This regime is typically observed within a reduced velocity range:

$$V_r={\displaystyle \frac{U}{D\cdot f_n}}$$

(2)

where $V_r$ denotes the reduced velocity, U is the incoming wind speed, D is the characteristic diameter of the mast, and $f_n$ is the natural frequency of the structure. This parameter controls the lock-in phenomenon, which is essential for efficient energy extraction in VIV-based systems.

Which corresponds to practical wind speeds between 4 and 8 m/s for VBT systems.

2.2. Structural Dynamics and Energy Conversion

The dynamic response of a VBT can be approximated by a single-degree-of-freedom system:

$$m\ddot{x}+c\dot{x}+kx=F\left(t\right)$$

(3)

where m, c, and k represent mass, damping, and stiffness, respectively.

F(t) represents the aerodynamic forcing induced by vortex shedding, intrinsically linked to periodic fluid–structure interactions governed by the Strouhal relation, thereby ensuring consistency between fluid excitation and structural response [16].

Advanced designs incorporate nonlinear stiffness mechanisms (e.g., magnetic tuning) to extend the lock-in range and improve energy capture. Geometric optimization, including tapered mast configurations, further enhances vortex synchronization and aerodynamic efficiency.

Geometric optimization, including tapered mast configurations, further enhances vortex synchronization and aerodynamic efficiency, as demonstrated in recent studies on VBT design and optimization [2,17].

The extracted power is expressed as:

$$P={\displaystyle \frac{1}{2}}\cdot \rho \cdot A\cdot C_g\cdot U^3$$

(4)

where $\rho$ is the air density, A is the effective swept area of the turbine, $C_g$ is the conversion efficiency coefficient, and U is the incoming wind speed. Although lower than conventional turbines, VBT efficiency (2–15%) is offset by simplified structure and dense deployment capability.

The resulting power corresponds to the electrical output after conversion losses, and although lower than conventional turbines, VBT efficiency is compensated by simplified structure and dense deployment capability [1].

2.3. Wake Modeling and Interactions

In multi-turbine configurations, wake interactions significantly impact performance. Although VBT wakes differ from classical HAWT wakes, simplified analytical models such as Jensen-type formulations remain widely used due to their computational efficiency and suitability for layout optimization [4,5].

A modified Jensen-type formulation is adopted:

$$R= R_0 + kx$$

(5)

$$U_x=U_0(1-a_x)$$

(6)

with the velocity deficit coefficient:

$$a_x={\displaystyle \frac{(1-\sqrt{1-C_T})}{{\left(\frac{R}{R_0}\right)}^2}}$$

(7)

where R is the wake radius at a downstream distance x, R₀ is the initial rotor radius, k is the wake expansion coefficient, $U_0$ is the free-stream wind speed, $U_x$ is the wind speed at distance x, $a_x$ is the velocity deficit coefficient, and $C_T$ is the thrust coefficient.

Multiple wake interactions are modeled through quadratic superposition:

$${\left(A_x\right)}^2={\displaystyle {\sum }_1^{NVBT}}{{a_x}_i}^2$$

(8)

This quadratic superposition approach provides a practical approximation of cumulative wake effects in multi-turbine configurations and is widely adopted in wind farm optimization studies [4].

This approach captures the cumulative impact of upstream turbines on downstream flow conditions.

Figure 1 illustrates the vortex-induced vibration mechanism and the simplified wake interaction model adopted for the VBT array.

Figure 1. Schematic representation of the vortex-induced vibration mechanism and wake interactions in VBT systems.

The dashed line indicates the wake centerline used to represent the idealized downstream trajectory of the vortex street.

2.4. Power Output Under Wake Influence

The effective wind speed at each turbine is reduced by wake interactions:

$$P={\displaystyle \frac{1}{2}}\cdot C_g\cdot \rho \cdot A\cdot {(U_0\cdot (1-A_x\left)\right)}^3$$

(9)

This cubic dependence highlights the importance of optimal spacing to limit energy losses. The model provides a computationally efficient framework for integration within optimization algorithms.

2.5. Site Description and Deployment Scenario

The study is conducted for the Boujdour coastal region (southern Morocco), characterized by favorable wind conditions and low terrain complexity. Average wind speeds typically range between 4 and 8 m/s in the operating regime of VBT systems.

The considered system includes 100 VBT units (≈25 W each), deployed over surfaces ranging from 100 m² to 2000 m². This configuration enables evaluation of the trade-off between turbine density, wake interactions, and land use. Previous optimization studies identified an optimal range between 500 and 800 m², which serves as a reference for the present analysis. Such configurations are consistent with hybrid microgrid deployments in remote regions, where renewable integration must balance resource availability, land constraints, and system reliability [13,14,15].

2.6. Grid Integration and Power Quality Constraints

Beyond aerodynamic considerations, VBT deployment introduces power quality (PQ) challenges due to the oscillatory nature of energy conversion. Key issues include voltage fluctuations, flicker, harmonic distortion, and ramp-rate variability, particularly in weak or islanded microgrids. These power quality challenges are well documented in renewable-rich microgrids, where intermittent generation and converter-based systems introduce harmonic distortion and voltage fluctuations [10,11,12].

To address these aspects, the proposed framework incorporates PQ indicators at the PCC, including:

• short-term flicker (Pst);
• total harmonic distortion (THD);
• power ramp-rate variations.

To quantitatively assess harmonic distortion at the point of common coupling (PCC), the total harmonic distortion (THD) is defined as the ratio of the root mean square of higher-order voltage harmonics to the fundamental component, in accordance with standard power quality formulations. The acceptable THD limits considered in this study follow IEC 61000 standards series of international standards governing electromagnetic compatibility and power quality [18,19].

$$THD={\displaystyle \frac{\sqrt{{\sum }_{n=2}^{\infty }{V}_n^2}}{V_1}}$$

(10)

where $V_n$ represents the RMS voltage of the nth harmonic component and $V_1$ denotes the RMS value of the fundamental voltage. The Total Harmonic Distortion (THD) quantifies the distortion level of the voltage waveform and is typically expressed as a percentage. This formulation follows standard power quality definitions and is consistent with international grid codes and IEC 61000 standards [18,19].

Mitigation strategies such as converter control, filtering, and battery energy storage systems (BESS) are considered to ensure compliance with grid standards.

This integrated approach aligns with recent advances in renewable-rich microgrid operation, where co-optimization of generation and grid constraints improves stability and system reliability.

3. DQN-Based Optimization Framework

To address the limitations of static optimization approaches, a Deep Q-Network (DQN) framework is employed to optimize the spatial deployment of 100 Vortex Bladeless Turbines (VBTs) under varying land constraints. The objective is to maximize energy production while minimizing wake interactions and ensuring efficient land utilization. As a Deep Reinforcement Learning (DRL) method, DQN has demonstrated strong performance in high-dimensional and nonlinear optimization problems, particularly in energy systems, microgrid control, and spatial planning [6,7,8,9].

3.1. Reinforcement Learning Formulation

The layout optimization problem is formulated as a Markov Decision Process (MDP) defined by the tuple $(S, A, T, R, \gamma ),$ where S represents the state space, including turbine configuration, wake interactions, and land occupancy, A denotes the action space corresponding to feasible layout modifications, T is the transition function governed by the simulation environment, R is the reward function, and $\gamma \in \left[\mathrm{0,1}\right]$ is the discount factor. At each timestep t, the agent observes a state $s_t$, selects an action $a_t$, receives a reward $r_t$, and transitions to a new state $s_{(t+1)}$. The objective is to maximize the expected cumulative reward:

$$R_t={\displaystyle {\sum }_{k=0}^{\infty }}{\gamma }^k{r}_t+k+1$$

(11)

The optimal action-value function is approximated using the Bellman update:

$$Q\left(s,a\right)\leftarrow Q\left(s,a\right)+\alpha [r+\gamma \underset{a^{\prime }}{\max }Q\left(s^{\prime },a^{\prime }\right)-Q\left(s,a\right)]$$

(12)

This formulation enables adaptive learning of optimal layouts under complex aerodynamic and grid-related constraints, as demonstrated in recent reinforcement learning applications to microgrid energy management and decision optimization [7,8,9].

3.2. State and Action Space Design

The state representation integrates both aerodynamic and electrical information, including turbine spatial coordinates, a wake interaction map based on Jensen-type modeling, land occupancy, wind conditions such as speed and direction, and power quality indicators including total harmonic distortion (THD), short-term flicker ($P_{st}$), and power ramp variations. This hybrid representation ensures that layout decisions simultaneously account for energy efficiency and grid compatibility.

The action space is defined through a set of discrete operations applied at each decision step, including displacement of a turbine in cardinal directions, swapping positions between turbines, or maintaining the current configuration. This formulation ensures controlled exploration while preserving system stability during the learning process.

3.3. Reward Function with PQ Integration

The reward function balances energy gain and spatial constraints while explicitly penalizing power quality (PQ) violations:

$$R_t={\Delta P}_t-\alpha \cdot C_t$$

(13)

where ${\Delta P}_t$ represents the incremental power gain, $C_t$ accounts for wake overlap and spatial penalties, and α is a weighting coefficient. Additional penalties are applied when PQ thresholds are exceeded, ensuring compliance with grid integration constraints. This formulation promotes solutions that are both energy-efficient and compliant with PQ standards, consistent with recent studies on grid-aware optimization of renewable energy systems [10,11,12].

3.4. Simulation Environment

The simulation environment models wake propagation using a modified Jensen approach, allowing estimation of local wind speed at each turbine. After each action, both aerodynamic conditions and grid-level indicators are updated. The aggregated power output is then injected into a simplified microgrid model, where PQ metrics are evaluated at the point of common coupling (PCC), in line with recent research on renewable integration in distributed energy systems [10,11,12].

3.5. Neural Network Architecture and Training

The Q-function is approximated using a feedforward neural network composed of two hidden layers with 64 and 32 neurons, respectively, using ReLU activation functions. The selected architecture represents a trade-off between computational efficiency and convergence stability. Training robustness is ensured through experience replay, target network updates, and an $\epsilon -greedy$ exploration strategy, where ε decreases progressively from 1.0 to 0.01.

The main hyperparameters include a learning rate of 0.001, a discount factor γ of 0.99, a replay buffer size of 10,000, a batch size of 64, and an episode length of 200 steps. Negative rewards are assigned in cases of turbine overlap, spacing violations, or excessive PQ deviations, enabling the learning process to incorporate both physical and grid-related constraints.

3.6. Multi-Surface Evaluation

The trained agent is evaluated across multiple land surfaces ranging from 100 to 2000 m². Performance metrics include total energy production, wake-induced losses, spatial efficiency, and power quality indicators such as THD, $P_{st}$, and ramp-rate variations. Results indicate that the DQN-based approach consistently outperforms baseline methods, particularly in constrained configurations between 500 and 800 m², where wake interactions and PQ issues are most significant. These findings are consistent with previous studies on layout optimization and renewable energy system integration [4,5,10].

3.7. Optimization Workflow

Figure 2 illustrates the overall DQN-based optimization workflow, including environment modeling, agent interaction, neural training, and performance evaluation. This framework enables adaptive spatial optimization by jointly considering aerodynamic interactions and grid-related constraints. The obtained results are benchmarked against Ant Colony Optimization (ACO), confirming the effectiveness of learning-based approaches for compact wind farm deployment. Similar studies have emphasized the importance of adaptive control strategies and wake-aware optimization in improving system performance [4,6].

Figure 2. Learning workflow of the DQN training process.

4. Results and Discussion

The proposed DQN-based optimization framework was evaluated across multiple land surface configurations ranging from 100 m² to 2000 m². The analysis focuses on energy production, power output, economic performance, and power quality (PQ) indicators at the point of common coupling (PCC). The results reveal a nonlinear relationship between surface area and optimization benefits, primarily driven by wake interactions, turbine density, and grid-side constraints, consistent with previous studies on wind farm layout optimization and renewable integration [4,5,10].

4.1. Small Surfaces (≤400 m²): Wake-Dominated Regime

At highly constrained surfaces (100–300 m²), turbine density induces strong wake interactions, significantly reducing effective wind velocity and oscillation amplitude. Even with DQN optimization, total power remains limited (<2 kW), with daily production ranging from 33 to 53 kWh. However, relative gains are substantial, reaching up to +41% at 100 m² and +20% at 200 m², as illustrated in Figure 3 and Figure 4, which show the variation in daily energy production and relative gain with surface area for both random and optimized layouts, highlighting the strong impact of wake interactions in highly constrained configurations. These results confirm the strong influence of wake losses in dense configurations, as widely reported in wind farm optimization studies [4,5].

Figure 3. Daily energy production versus surface area for random and optimized layouts.

Figure 4. Variation in energy gain (%) with surface area.

From a grid perspective, clustered power injection leads to pronounced variability, resulting in high flicker levels ($P_{st}$ > 1.0) and increased harmonic distortion. These observations are consistent with known power quality issues in renewable-rich microgrids, where high variability and converter-based generation lead to voltage fluctuations and harmonic distortion [10,11,12]. Although optimization improves energy yield, PQ degradation remains a critical limitation in dense layouts, requiring mitigation strategies such as energy buffering or filtering techniques.

4.2. Medium Surfaces (500–800 m²): Optimal Deployment Zone

The 500–800 m² range represents the optimal trade-off between energy efficiency and land utilization. Increased spacing reduces wake overlap while maintaining compact deployment. The optimized layouts achieve daily production above 58 kWh, with power output between 2.44 and 2.52 kW. Figure 5 and Figure 6 present the evolution of power output and optimization gain as a function of surface area, illustrating the improved performance achieved within the optimal deployment range. These findings align with previous studies highlighting the importance of spacing optimization in minimizing wake interactions [4].

Figure 5. Power output (kW) versus surface area for the optimized layout.

Figure 6. Combined analysis of optimized power output and percentage gain versus surface area.

Although relative gains are moderate (6–9%), they remain economically attractive due to limited land cost increase. More importantly, PQ performance significantly improves: flicker indices decrease below 0.8, THD remains within acceptable limits, and power fluctuations are reduced. This behavior is consistent with grid integration requirements for low-voltage microgrids and confirms that this range ensures a balance between aerodynamic performance, economic viability, and PQ compliance [10,11,12].

4.3. Large Surfaces (>1000 m²): Saturation Regime

For larger surfaces, wake interactions become negligible even in non-optimized layouts. Consequently, DQN gains fall below 4%, with marginal improvements (<2 kWh/day beyond 1600 m²). The power output stabilizes around 2.59 kW, approaching free-stream conditions. Figure 7 illustrates the evolution of energy and power gain with increasing surface area, highlighting the saturation behavior as wake interactions diminish under near free-stream conditions. This saturation behavior is typical of low-density wind farm configurations, where wake effects are minimized [4].

Figure 7. Energy and power gain versus surface area.

From an economic perspective, land cost increases significantly, resulting in very low gain-to-cost ratios. Figure 8 illustrates the relationship between energy gain and terrain cost, highlighting the decreasing economic efficiency as surface area increases. PQ performance is naturally improved, with negligible flicker and low harmonic distortion, consistent with reduced interaction between turbines. However, these benefits do not compensate for the increasing land cost, making such configurations less attractive for practical deployment.

Figure 8. Economic value of energy gain versus terrain cost.

4.4. Comparative Analysis

Table 1 summarizes the performance metrics across all surface configurations, including energy production, optimization gain, and PQ indicators. The results highlight a clear transition between wake-dominated, optimal, and saturation regimes. The observed trends confirm that energy performance alone is insufficient to assess optimal deployment, and that power quality and economic constraints must be jointly considered, as emphasized in recent renewable energy system studies [10,11,12].

Table 1. Summary of energy, optimization gain, and PQ indicators.

Surface (m2)	Power (kW)	Energy (kWh/day)	Gain (%)	Pst	THD (%)	ΔPmax	Interpretation
100	1.1–1.3	33–35	+41	1.5–1.7	7–8	High	Severe wake, critical PQ
200	1.6–1.8	45–53	+20	1.2–1.4	6–7	High	PQ still limiting
300–400	1.9–2.1	50–55	10–15	1.0–1.2	5–6	Moderate	Dense regime
500–800	2.44–2.52	58–62	6–9	0.6–0.8	3–4	Low	Optimal trade-off
1000–1600	2.55–2.58	60–62	<4	0.3–0.5	2–3	Low	Marginal gain
>2000	≈2.59	≈62	<2	<0.3	<2	Very Low	Free-stream regime

4.5. Key Insights

The results clearly identify three operating regimes. The dense regime (<400 m²) is characterized by strong wake losses and significant PQ degradation. The optimal regime (500–800 m²) provides the best compromise between energy, cost, and PQ. The sparse regime (>1000 m²) exhibits stable performance but limited optimization benefit. These findings confirm that layout optimization must simultaneously consider aerodynamic efficiency, economic constraints, and grid integration aspects, in agreement with recent work on hybrid renewable systems and microgrid operation [4,10].

4.6. Discussion and Perspectives

The proposed DQN framework demonstrates its ability to adaptively balance competing objectives in compact wind farm design. Compared to conventional approaches such as ACO, it enables dynamic and context-aware optimization, particularly relevant in constrained environments. Reinforcement learning approaches have shown similar advantages in energy systems optimization and adaptive control problems [6,7,8,9].

Future developments may include multi-agent reinforcement learning for distributed control, integration of real-time PQ monitoring, and co-optimization with storage systems and flexible loads, which are increasingly considered in advanced microgrid management strategies [7,8,9].

5. Conclusions

This work investigates a Deep Q-Network (DQN)-based framework for optimizing the spatial deployment of Vortex Bladeless Turbines (VBTs) in a Saharan microgrid. The results identified an optimal deployment range between 500 and 800 m², where moderate energy gains (6–9%) are achieved while ensuring acceptable power quality (PQ) performance.

Dense configurations (≤400 m²) provide high relative gains but suffer from significant wake interactions and PQ degradation, limiting their practical applicability. In contrast, larger surfaces (>1000 m²) offer stable operation but yield marginal optimization benefits due to reduced wake effects and increasing land cost.

Overall, the proposed approach demonstrates that combining deep reinforcement learning with physics-based modeling enables efficient and grid-compliant wind farm design in constrained environments.