Voltage/VAR Control in Active Distribution Networks via DRL Under False Data Injection Attacks on Distributed PV Systems

Abstract

Owing to its capability to handle uncertainties and provide rapid responses, Deep Reinforcement Learning (DRL) has been widely applied to Volt-Var Control (VVC) in Active Distribution Networks (ADNs). However, existing studies still present two main limitations. First, the characteristics of power equipment have not been adequately integrated with the action properties of DRL, which may compromise the control performance. Second, current DRL-based VVC methods for ADNs remain insufficiently resilient to False Data Injection Attacks (FDIAs) targeting Photovoltaic systems (PVs), significantly increasing the risks of voltage instability and operational insecurity in distribution networks. To address these challenges, a novel segmented power-constraint method is proposed to reconcile the mismatch between the control characteristics of traditional PV inverters and the action-generation mechanism of DRL agents. Furthermore, by incorporating a Distribution-Based Correction Observer into the twin delayed deep deterministic policy gradient algorithm, the proposed method enhances the resilience of DRL-based control against corrupted PV measurement data. This enables the agent to maintain reliable decision-making capabilities even when PV data are compromised. Simulation results demonstrate that the proposed method effectively enhances voltage stability, reduces power losses, and maintains robust control performance under false data injection attacks.

1. Introduction

Distributed energy resources (DERs), such as photovoltaic (PV) generation, have recently emerged as alternatives to fossil fuels, offering viable solutions for environmental protection by mitigating energy depletion [1]. However, the highly uncertain and stochastic nature of PV introduces significant power-flow fluctuations when integrated into large-scale ADNs, which eventually leads to voltage deviations and violations [2]. Meanwhile, the large-scale integration of DERs has significantly increased the communication demands on the ADN, correspondingly elevating its exposure to cyber attack risks [3]. Consequently, a robust voltage control mechanism should be integrated as an essential component of ADNs to effectively mitigate the uncertainties associated with PV generation [4] and the damage caused by cyber attacks [5].

Installing inverters on PV systems is a solution to improve the voltage regulation capability of ADN [6]. The inverter suppresses voltage deviations and violations by controlling reactive power and curtailing active power [7]. PV inverters are widely used in traditional VVC in ADN. For instance, the reactive power control capabilities of PV inverters are used to improve voltage quality and reduce power losses [8]. The DistFlow model has been used to achieve VVC under high PV penetration [9]. Second-order cone programming has been used to solve VVC problems involving PV inverters in ADNs [10]. Moreover, PV uncertainties can be addressed by using various methods including Stochastic Programming [11], Robust Optimization [12], Model Predictive Control approaches [13], etc. Nevertheless, these model-based techniques require precise knowledge of ADN, but in practice, obtaining complete and accurate information about ADN is often challenging [14]. Moreover, subsequent genetic algorithms and particle swarm optimization are insufficient to satisfy the rapid response requirements of ADN [15]. By contrast, DRL, as a data-driven and model-free control method, can accommodate the uncertainties intrinsic to ADNs while rapidly generating control outcomes [16]. As a result, DRL has become a promising research direction for voltage regulation in ADN [17]. For instance, the Deep Deterministic Policy Gradient (DDPG) [18] is used to provide a fast-response VVC method for PV inverters. A phased VVC method is developed by combining Deep Q-Network, DDPG, and Twin Delayed Deep Deterministic Policy Gradient (TD3) [19]. To address the limitations of the centralized methods reported in [18,19], studies [20,21,22] proposed distributed VVC methods based on multi-agent TD3, multi-agent DDPG, and multi-agent soft actor-critic, respectively.

However, despite significant progress in the application of DRL methods for PV inverter control in ADNs, existing studies remain bound by traditional inverter control constraints, which considerably undermines the effectiveness of DRL-based strategies. None of the existing methods have made adaptive improvements to the characteristics of DRL, because the normalized action ratios generated by DRL differ from the control variables used by conventional solvers. Equally critical, the inherent reliance of DRL on observed data makes it particularly vulnerable to cyber attacks [23]. This vulnerability is further amplified by a fundamental contradiction between the cybersecurity posture and the large-scale deployment of PV systems. On one hand, to reduce costs and enable rapid deployment, a vast number of DPV inverters employ standardized communication protocols, which were designed without adequate consideration of cybersecurity requirements and lack essential encryption and authentication mechanisms. On the other hand, DPV systems are widely distributed across rooftops, parking lots, and remote areas, where physical access control is weak, leaving communication lines susceptible to close-range wireless intrusion or physical tampering. In this context, an adversary is considered capable of exploiting protocol vulnerabilities in DPV systems to falsify the active power measurement channels at PV nodes. The attacker’s ultimate objective is to distort the agent’s perception of the system state by injecting false PV generation data, thereby misleading the controller into issuing incorrect voltage control commands that result in voltage fluctuations or even violations. Specifically, the decision-making of the DRL agent relies entirely on the high-dimensional state data (such as bus voltage, power, PV, etc.) observed during its training and control processes [24]. Once the PV measurements are compromised, the agent perceives a corrupted system state, and the resulting control actions become misaligned with actual operating conditions, breaking the essential link between voltage quality monitoring and effective inverter control. Such “data poisoning” or “adversarial observation” attacks can directly undermine the reliability of DRL strategies, leading to severe consequences such as voltage violations and instability in the ADN [25].

To date, methods for detecting cyberattacks can be broadly categorized into two classes. One category is the model correction approach, typically represented by state estimation [26,27], while the other is the feature learning method based on machine learning [28]. These studies collectively demonstrate the importance of considering cyber attack scenarios in ADNs. Regrettably, existing research is predominantly confined to attack detection and data restoration, with few studies integrating these aspects with control strategies. Consequently, recent studies have increasingly emphasized the “detection-reconstruction-control” closed-loop strategy against cyber attacks in power systems, as evidenced by a growing body of recent references [29,30,31,32,33]. Specifically, study [29] embeds an artificial neural network-based state estimator into the control loop to detect and mitigate false data injection attacks, followed by voltage regulation via droop control. Study [30] proposes a distributed secondary control method based on a sliding mode. Similarly, the control logic in study [31] follows the same principle. Study [32] employs a hybrid approach combining MPC and DRL to defend against cyber attacks, while study [33] incorporates the cyber attack model into the training process to develop a control strategy resilient to such attacks.

In summary, to date, there remains a lack of a method capable of precisely mapping the physical operational characteristics of PV inverters into the action-generation mechanism of DRL agents. In addition, developing a VVC method capable of directly resisting cyber attacks, rather than merely detecting them, has emerged as an active research topic. Moreover, the “plug-and-play” nature of distributed PV systems results in diverse equipment suppliers, heterogeneous communication protocols, and inconsistent security standards. This convergence creates a “weakest-link effect” in cybersecurity, making unified and effective defense mechanisms difficult to implement. Targeted research in this area is essential. Therefore, a novel segmented power-constraint method is proposed to resolve the power constraints of traditional PV inverters by combining the influence characteristics of the control ratio of DRL. Meanwhile, based on the operational mechanism of DRL-based VVC, a method capable of effectively mitigating the impact of FDIAs in PV is designed. The main contributions of this paper are summarized as follows:

• Unlike existing methods, the proposed model accounts for the action characteristics of DRL and decouples traditional PV inverter constraints. A two-stage PV inverter power constraint model is introduced to accurately map the physical limits of PV inverters into the actions of the DRL agent, thereby enabling effective VVC;
• In contrast to conventional detection-only methods (e.g., forecasting-residual-based detectors, anomaly detection) that merely raise alarms or output binary flags without interfacing with the controller, and unlike standard Kalman filtering or state estimation that require explicit system models and full-network computation, a Distribution-Based Correction Observer (DBCO) is proposed that is lightweight, model-free, and tightly embedded at the state-input interface of the DRL agent, which provides continuous real-time correction of corrupted PV measurements without modifying the agent’s internal structure;
• Unlike robust/adversarial DRL approaches that require costly retraining for each new attack pattern and often overfit to specific attack strategies, the proposed DBCO is plug-and-play for pre-trained agents, requires no retraining for the FDIAs scenarios considered in this study, and offers a practical advantage over adversarial retraining approaches that must be re-executed for each new attack pattern.

The remainder of this paper is structured as follows: Section 2 introduces the problem formulation and the system model. Section 3 provides a detailed description of the two-stage PV inverter power constraint and Distribution-based correction observer. Section 4 presents the simulation results and their verification. Section 4 concludes the study with a summary of findings.

2. Problem Formulation

2.1. Optimization Objectives

Based on [8], a control objective for VVC in ADNs can be formulated as a multi-objective function [34,35]. Extending this foundation, this paper introduces a multi-objective function, as shown in Equation (1). Following normalization, the utility coefficient is employed to transform the multi-objective function into a single objective formulation, as shown in Equation (4). The resulting optimization model can be formulated as follows:

$$\min [b_{1,t},b_{2,t}]$$

(1)

$$b_{1,t}={\displaystyle \sum _{i=1}^N\left|V_{i,t}-V_0\right|}$$

(2)

$$b_{2,t}={\displaystyle \sum _{i=1,j=1}^N{I}_{\mathrm{ij},\mathrm{t}}^2{r}_{ij,t}}$$

(3)

$$\min \omega \left({\displaystyle \frac{b_{1,t}^{\mathrm{pu}}-b_{1,\min }^{\mathrm{pu}}}{b_{1,\max }^{\mathrm{pu}}-b_{1,\min }^{\mathrm{pu}}}}\right)+(1-\omega )\left({\displaystyle \frac{b_{2,t}^{\mathrm{pu}}-b_{2,\min }^{\mathrm{pu}}}{b_{2,\max }^{\mathrm{pu}}-b_{2,\min }^{\mathrm{pu}}}}\right)$$

(4)

where Equation (2) represents the total voltage deviations across all nodes in the distribution network. Equation (3) quantifies the overall active power loss in the ADN. The power loss on a specific line $ij$ is calculated using $I_{\mathrm{ij},\mathrm{t}}^2{r}_{ij}$. After being converted to per-unit values, the voltage deviation and network power loss are respectively normalized using the min-max normalization method to eliminate discrepancies in their dimensions and numerical ranges. A weight coefficient $\omega$ is then introduced to adjust the trade-off between these two normalized objectives, as formulated in Equation (4), while $\omega \in [0,1]$ serves as a user-defined weighting factor that determines the trade-off between voltage deviation and power loss. It should be noted that in both DC transmission and distribution networks, Equations (2) and (3) generally exhibit similar effects. However, AC power flow constraints are adopted for the ADN. Due to the complexity of AC power flow and the dual role of voltage, these two objectives are generally in conflict and represent competing goals. A detailed analysis can be found in reference [8].

2.2. The ADN Model

The ADN model with agent interaction is formulated as follows:

$$\left\{\begin{array}{l}{P}_{i,t}=V_{i,t}{\displaystyle \sum _{j=1}^N{V}_{j,t}}\left(G_{ij,t}\cos {\theta }_{ij,t}+B_{ij,t}\sin {\theta }_{ij,t}\right),\hspace{1em}\forall i\in N\\ Q_{i,t}=V_{i,t}{\displaystyle \sum _{j=1}^N{V}_{j,t}}\left(G_{ij,t}\sin {\theta }_{ij,t}-B_{ij,t}\cos {\theta }_{ij,t}\right),\hspace{1em}\forall i\in N\end{array}\right.$$

(5)

$$P_{i,t}=P_{i,t}^{\mathrm{PV}}-P_{i,t}^{\mathrm{load} }$$

(6)

$$Q_{i,t}=Q_{i,t}^{\mathrm{PV}}-Q_{i,t}^{\mathrm{load} }$$

(7)

$$V_{\min }\le V_{i,t}\le V_{\max },\hspace{1em}\forall t\in T,i\in N$$

(8)

$$0\le P_{i,t}^{\mathrm{PV}}\le P_{ i,\max }^{\mathrm{PV}},\hspace{1em}\forall t\in T,i\in N$$

(9)

$$Q_{i,\min }^{\mathrm{PV}}\le Q_{i,t}^{\mathrm{PV}}\le Q_{i,\max }^{\mathrm{PV}},\hspace{1em}\forall t\in T,i\in N$$

(10)

$${(S_{i,t}^{\mathrm{PV}})}^2={(P_{i,t}^{\mathrm{PV}})}^2+{(Q_{i,t}^{\mathrm{PV}})}^2,\hspace{1em}\forall t\in T,i\in N$$

(11)

where the dynamic characteristics of the ADN are governed by constraints on active and reactive power flow, as specified in Equation (5). Node power balance is maintained through Equations (6) and (7), while voltage safety is ensured by the constraint defined in Equation (8). The constraints of traditional PV inverters are described in Equation (9) through Equation (11).

2.3. Markov Decision Process (MDP)

In general, objectives in Equations (2) and (3) under AC power flow constraints often cannot be directly solved due to their non-convex and nonlinear nature. Simplification of the model or relaxation of constraints, as reported in [9,10], is therefore required. In contrast, DRL-based approaches are not constrained by the complexities of power flow equations, as they observe the state of the ADN through data acquisition systems and directly generate control actions. After applying the control actions to the ADN, the resulting system state is evaluated to guide the iterative update of the control policy. This entire process can be formulated as an MDP [18,19], as shown in Figure 1. The state set $S$, action set $A$, and reward function $R$ constitute the fundamental components of MDP. The MDP framework for the agent is described below.

Figure 1. Markov Decision Process.

2.3.1. The State Set $S$

The agent observes the current state of the ADN and determines the appropriate control actions. Typically, the state set should include the target variable and variables that affect the target variable. The state space of the agent is defined as follows:

$$S_t=\left(P_{i,t}^{\mathrm{load}},Q_{\mathrm{i},\mathrm{t}}^{\mathrm{load}},V_{\mathrm{i},\mathrm{t}},P_{i,t}^{\mathrm{PV}}\right)$$

(12)

2.3.2. The Action Set $A$

The action set of the agent is given as follows.

$$A_t=\left(Q_{i,t}^{\mathrm{PV}}\right)$$

(13)

2.3.3. The Reward Function $R$

To ensure alignment with the control objectives, the reward function of the agent must be designed consistently with the control objectives, thereby guaranteeing that the optimization direction of the reinforcement learning process is consistent with the underlying physical control goals. Based on Equation (4), the reward function is formulated in Equation (14), where the voltage-violation penalty term is defined in Equation (15).

$$r_t=-\left(\omega \left({\displaystyle \frac{b_{1,t}^{\mathrm{pu}}-b_{1,\min }^{\mathrm{pu}}}{b_{1,\max }^{\mathrm{pu}}-b_{1,\min }^{\mathrm{pu}}}}\right)+(1-\omega )\left({\displaystyle \frac{b_{2,t}^{\mathrm{pu}}-b_{2,\min }^{\mathrm{pu}}}{b_{2,\max }^{\mathrm{pu}}-b_{2,\min }^{\mathrm{pu}}}}\right)+b_{3,t}\right)$$

(14)

$$b_{3,t}={\displaystyle \sum _{i=1}^N\left[\max \left(0,V_{\min }-V_{\mathrm{i},\mathrm{t}}\right)+\max \left(0,V_{\mathrm{i},\mathrm{t}}-V_{\max }\right)\right]}$$

(15)

$$R={\displaystyle \sum _{t=0}^{\infty }{\gamma }^t}{r}_t$$

(16)

The voltage-safety constraint in Equation (8) is incorporated into the reward function through the penalty term defined in Equation (15), as in [36]. This inclusion ensures that the system operates within permissible limits. Equation (16) indicates that the agent aims to maximize cumulative reward.

3. DBCO-TD3

3.1. TD3

In ADNs, PV inverters are critical components that operate within a continuous action space. The TD3 algorithm is particularly effective for addressing such continuous action space problems, compared to other algorithms such as DDPG [19]. This algorithm employs an Actor-Critic architecture, which includes a policy network (Actor) and two Critic networks to estimate state-action values $Q^{\pi }(S,A)$. The Actor network uses the Policy ${\pi }_{\phi }$ to generate an Action $A$ under state $S$, i.e., $A={\pi }_{\phi }(S)$. The Actor network optimizes the policy by maximizing the Q value of the Critic network. Specifically, TD3 uses two independent Critic networks to output two Q-values: $Q_{{\theta }_1}$ and $Q_{{\theta }_2}$, and then selects the smaller Q-value from the two to guide policy optimization. For a detailed description, refer to [19].

3.2. Impact of FDIAs on TD3

As described in the preceding section, acquiring the state space $S$ is crucial for DRL-based methods. All computations and parameter updates are based on $S$. If $S$ contains errors, the agent will be unable to obtain the correct control policy.

Suppose the actual state of the system is $S_t$ at a certain moment and the system state recorded by the data collector is $Z_t$. In practice, the data used by the agent consists of measured values.

$$S_t=F\left(S_{t-1},A_{t-1}\right)+{\omega }_t$$

(17)

$$Z_t=H(S_t)+v_t$$

(18)

$$A_t={\pi }_{\phi }\left(Z_t\right)+{ϵ}_t,\hspace{1em}{ϵ}_t~\mathcal{N}\left(0,{\sigma }^2\right)$$

(19)

$$A_t={\pi }^{\ast }\left(Z_t\right)$$

(20)

where $H$ denotes the measurement matrix and $v_t$ represents the measurement noise.

If a false data injection attack occurs, the actual state perceived by the agent becomes $Z_t^{\mathrm{attack}}$;

$$Z_t^{\mathrm{attack}}=Z_t+a_t^{\mathrm{attack}}=H(S_t)+v_t+a_t^{\mathrm{attack}}$$

(21)

where $a_t^{\mathrm{attack}}$ represents the attacker injecting attack signals into the measurement channel.

At this point, the action proposed by the agent will be:

$$A_t={\pi }_{\phi }\left(Z_t^{\mathrm{attack}}\right)+{ϵ}_t,\hspace{1em}{ϵ}_t~\mathcal{N}\left(0,{\sigma }^2\right)$$

(22)

$$A_t={\pi }^{\ast }\left(Z_t^{\mathrm{attack}}\right)\ne {\pi }^{\ast }\left(Z_t\right)$$

(23)

Because the agent’s immediate reward is directly fed back by the system, false system states, the corresponding actions, and rewards derived from the actual system state are packaged into an experience bundle and stored in the experience buffer. This will inevitably affect the calculation of the Q-value. In addition, it can be inferred from the preceding subsections that the effectiveness of DRL, particularly value-based algorithms such as TD3, depends on a fundamental assumption: the state observations during training and deployment follow the same underlying distribution. This assumption supports the convergence of Bellman iteration and the unbiased estimation of policy gradients. It should be noted that FDIAs do not alter the internal Volt/Var control logic of PV inverters. Instead, they corrupt the state observations used by the DRL agent, causing the agent to issue suboptimal reactive power commands such as that in Equation (23). The inverters simply execute these commands as received, making the attack effect indirect—mediated through the agent’s decision-making rather than through direct inverter manipulation.

It should be noted that the accuracy of observations is closely related to the intensity of attacks. When attack intensity is low, contaminated observations can maintain the state space distribution learned during training. This is primarily attributed to TD3’s inherent mechanisms, such as clipped double Q-learning, target network smoothing via soft updates, and exploratory noise. These mechanisms effectively resist such distributional perturbations, thereby ensuring performance degradation within acceptable and controllable levels. In contrast, strong or strategic FDIAs can systematically and persistently distort the observed state $S_a$, causing a significant divergence between the distribution of perceived states $P\left(S_a\right)$ and the expected state distribution $P\left(S\right)$. This distributional shift undermines the TD3 learning framework through the following two principal mechanisms:

• Experience Replay Poisoning: The corrupted observation $Z_t^{\mathrm{attack}}$, the corresponding suboptimal action $a_t^{\mathrm{attack}}$, and the reward generated by the actual environment are stored together as a corrupted experience tuple in the replay buffer. When sampled during training, these poisoned experiences directly influence the update of both Critic and Actor networks. This leads to biased Q-value estimates and propagation of erroneous policy gradients, thereby steering the policy toward an optimum defined in the adversarial space;
• Decoupling from the Bellman Equation: The TD3 critic update relies on the Bellman consistency relationship between consecutive state–action pairs. When the normal observations $Z_t$ and $Z_{t+1}$ are replaced by corrupted observations $Z_t^{\mathrm{attack}}$ and $Z_{t+1}^{\mathrm{attack}}$, this recursive relationship no longer accurately represents the dynamics of the actual ADN. Consequently, the algorithm effectively learns within a fictitious MDP constructed by the attacker, whose dynamics are decoupled from the true ADN dynamics. The resulting “optimal” policy is therefore optimal only with respect to this adversarial MDP and may be severely suboptimal or destabilizing in the real ADN.

This analysis reveals a fundamental limitation of conventional DRL: its learning process operates in an open-loop manner with respect to state observation integrity. Conventional approaches lack explicit mechanisms to perceive, verify, or adapt to the authenticity of sensory data. When an adversary persistently injects deceptive signals that fall outside the training distribution, the agent will be systematically guided to learn and execute a policy that is “optimal” within the attacker’s fabricated state representation but detrimental to the actual operational objectives of the physical system. This highlights the critical need for integrating observation trustworthiness awareness and enhanced robustness guarantees into DRL frameworks for secure power system applications. However, conventional agents cannot filter such attack signals. Therefore, a dedicated observer is designed.

3.3. Distribution-Based Correction Observer (DBCO)

As discussed in the previous section, the accuracy of observation data is crucial for agent decision-making. In ADN environments, PV power generation data are particularly vulnerable to FDIAs due to their distributed nature and high variability. Such attacks can lead to significant operational consequences, highlighting the need for enhanced protection in this critical area [37]. To address this issue, a Distribution-Based Correction Observer (DBCO) is proposed to mitigate the impact of FDIAs on the agent’s observation process, thereby improving the security and robustness of the system. Its core concept is to utilize the ‘dynamic baseline’ generated by a data-driven prediction model. Through the Kalman framework, the DBCO identifies and corrects deviations in observations caused by FDIAs in real time, thereby providing the DRL agent with a robust state-perception interface, as shown in Figure 2. This distribution can be easily obtained using current advanced forecasting techniques [38].

Figure 2. Schematic Diagram Based on Distribution Correction.

The procedure is described as follows:

DBCO uses historical distribution characteristics of photovoltaic output. For each point in time, the following statistics are calculated from historical data:

$$X_t^{\mathrm{PV}}~N\left({\mu }_t,{\sigma }_t^2\right),$$

(24)

$${\mu }_t={\displaystyle \frac{1}{N_h}}{\displaystyle \sum _{k=1}^{N_h}{P}_{i,t}^{\mathrm{PV},(k)}}$$

(25)

$${\sigma }_t=\sqrt{{\displaystyle \frac{1}{N_h-1}}{\displaystyle \sum _{k=1}^{N_h}{\left(P_{i,t}^{\mathrm{PV},(k)}-{\mu }_t\right)}^2}}$$

(26)

where ${\mu }_t$ denotes the historical mean PV output at time $t$, and ${\sigma }_t$ denotes the corresponding standard deviation; $N_h$ denotes the number of historical samples. These two statistics characterize the normal range of fluctuation in PV output and serve as the benchmark for determining whether the data has been tampered with.

Step 1: Initialize the covariance matrix:

$$P_t^K=\left\{\begin{array}{ll}{\sigma }_t^2,\hfill & \mathrm{if} {\sigma }_t>0\hfill \\ 1.0,\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(27)

Step 2: Calculate the Anomaly Score:

$$d_t={\displaystyle \frac{\left|z_t-{\mu }_t\right|}{{\sigma }_t}},\hspace{1em}{\sigma }_t>0$$

(28)

where $z_t$ is the measured PV active power.

Step 3: Adaptive Measurement Noise Settings:

$$R_{\mathrm{normal} }^K={\left(k_R\cdot {\sigma }_t\right)}^2,\hspace{1em}{\sigma }_t>0$$

(29)

$$R_t^K=\left\{\begin{array}{ll}{R}_{\mathrm{normal} }^K,\hfill & d_t\le d_{68}\hfill \\ R_{\mathrm{normal} }^K\cdot \left[1+{\beta }_1\cdot \left(d_t-d_{68}\right)\right],\hfill & d_{68}<d_t\le d_{95}\hfill \\ R_{\mathrm{normal} }^K\cdot \left[{\beta }_2+{\beta }_3\cdot \left(d_t-d_{95}\right)\right],\hfill & d_{95}<d_t\le d_{99}\hfill \\ R_{\mathrm{normal} }^K\cdot {\beta }_4,\hfill & d_t>d_{99}\hfill \end{array}\right.,\hspace{1em}{\sigma }_t>0$$

(30)

$$\varsigma =\left\{\begin{array}{ll}1.0,\hfill & d_t\le d_{68}\hfill \\ {\gamma }_1,\hfill & d_{68}<d_t\le d_{95}\hfill \\ {\gamma }_2,\hfill & d_{95}<d_t\le d_{99}\hfill \\ {\gamma }_3,\hfill & d_t>d_{99}\hfill \end{array}\right.,\hspace{1em}{\sigma }_t>0$$

(31)

$$R_t^K=\left\{\begin{array}{ll}{R}_{\min },\hfill & P_t^{PV,o}=0\hfill \\ R_{\max },\hfill & P_t^{PV,o}\ne 0\hfill \end{array}\right.,\hspace{1em}{\sigma }_t=0$$

(32)

$$\varsigma =\left\{\begin{array}{ll}1.0,\hfill & P_t^{PV,o}=0\hfill \\ {\zeta }_{\min },\hfill & P_t^{PV,o}\ne 0\hfill \end{array}\right.,\hspace{1em}{\sigma }_t=0$$

(33)

where $R_{\mathrm{normal} }^K$ is the baseline measurement noise; $k_R$ is a scaling coefficient used to determine the baseline measurement-noise covariance; $R_t^K$ is the adaptive measurement noise. $d_{68}$ is the threshold calculated based on a 68% confidence interval; $d_{95}$ is the threshold calculated based on a 95% confidence interval; $d_{99}$ is the threshold calculated based on a 99% confidence interval. The values of these confidence thresholds comply with the Pauta criterion (3$\sigma$ criterion); $\varsigma$ is the confidence score; ${\beta }_1$ is the attenuation coefficient for the first section; ${\beta }_2$ is the denominator for the second segment; ${\beta }_3$ is the attenuation coefficient for the second section; ${\beta }_4$ is the denominator at maximum attenuation; ${\gamma }_1$ is the first-level trust factor ($0<{\gamma }_1<1$); ${\gamma }_2$ is the second-level trust factor ($0<{\gamma }_2<{\gamma }_1$); ${\gamma }_3$ is the third-level trust factor ($0<{\gamma }_3<{\gamma }_2$); $R_{\min }$ is the minimum measurement noise covariance set for conditions without attacks; $R_{\max }$ maximum measurement-noise covariance under attack conditions; ${\zeta }_{\min }$ is the minimum trust factor when an attack is detected.

Step 4: Correct the measured values:

$$z_t^{\mathrm{eff}}=\left\{\begin{array}{ll}\varsigma \cdot z_t+(1-\varsigma )\cdot {\mu }_t,\hfill & \varsigma <{\varsigma }_{th}\hfill \\ z_t,\hfill & \varsigma \ge {\varsigma }_{th}\hfill \end{array}\right.$$

(34)

$${\nu }_t=\left\{\begin{array}{ll}{z}_t^{\mathrm{eff} }-z_t^{\mathrm{pre}},\hfill & \varsigma <{\varsigma }_{th}\hfill \\ 0,\hfill & \varsigma \ge {\varsigma }_{th}\hfill \end{array}\right.$$

(35)

$$K_t={\displaystyle \frac{{P_t^K}_t}{P_t^K+R_t^K}}$$

(36)

$${\widehat{z}}_t=\left\{\begin{array}{ll}\max \left(0,z_t^{\mathrm{pre}}+K_t\cdot {\nu }_t\right),\hfill & \varsigma <{\varsigma }_{th}\hfill \\ \max \left(0,z_t+K_t\cdot {\nu }_t\right),\hfill & \varsigma \ge {\varsigma }_{th}\hfill \end{array}\right.$$

(37)

where ${\widehat{z}}_t$ is the valid measurement; $z_t^{\mathrm{eff}}$ represents the corrected observed value; $z_t^{\mathrm{pre}}$ denotes the prior predicted PV output obtained from the historical baseline or forecasting model; $K_t$ denotes the Kalman-like correction gain; ${\nu }_t$ is the secondary adjustment value; ${\varsigma }_{th}$ is the confidence factor threshold, used to determine whether to apply the correction value. In this study, ${\varsigma }_{th}$ is set to 0.6.

Therefore, in the proposed formulation, $S_t$ will be replaced entirely with ${\widehat{S}}_t$:

$${\widehat{S}}_t=\left(P_{i,t}^{\mathrm{load}},Q_{\mathrm{i},\mathrm{t}}^{\mathrm{load}},V_{\mathrm{i},\mathrm{t}},{\widehat{z}}_t\right)$$

(38)

$$A_t={\pi }_{\phi }\left({\widehat{S}}_t\right)+{ϵ}_t,\hspace{1em}{ϵ}_t~\mathcal{N}\left(0,{\sigma }^2\right)$$

(39)

$$A_t={\pi }^{\ast }\left({\widehat{S}}_t\right)$$

(40)

The structure of DBCO-TD3 proposed in this paper is illustrated in Figure 3.

Figure 3. The Simplified structure of DBCO-TD3.

The proposed method introduces a DBCO module to reformulate the state input pathway on the basis of the traditional TD3 framework. In conventional TD3, the raw measured state $Z_t$ is directly fed into the Actor and Critic networks, as shown in Figure 1. In the proposed structure, however, the measured state $Z_t$ is first processed by the DBCO for state estimation and noise reduction, yielding a filtered state ${\widehat{S}}_t$, which is then forwarded to the Actor to generate an action $A_t$ and evaluated by the Critic. This design substantially enhances the robustness of the agent under attack conditions. By smoothing and refining the state through DBCO, the accuracy of state estimation is improved, thereby enabling the reinforcement learning agent to make more stable and reliable decisions even when observations are corrupted. In addition, as an independent preprocessing module, DBCO incurs significantly lower computational overhead than training large neural networks online. Moreover, it requires no modification to the internal structure of DRL algorithms, which facilitates its integration into existing architectures.

It should be noted that the DBCO module does not require strictly Gaussian PV data. The confidence intervals based on mean and standard deviation serve as practical heuristic thresholds for anomaly identification rather than strict normality tests. They offer a computationally efficient and interpretable measure of deviation from historical baselines. The adaptive noise covariance adjustment is smooth and continuous; even under non-Gaussian distributions, the correction pulls measurements toward the historical mean proportionally to the deviation, and remains reasonable for most practical scenarios.

3.4. Analysis of the DBCO-TD3

To better position the proposed DBCO-TD3 within the existing literature, we provide a systematic comparison with five categories of closely related methods as shown in Table 1: (a) Kalman filtering, (b) conventional state estimation, (c) forecasting-residual-based detection, (d) general anomaly detection, and (e) robust/adversarial DRL. While each of these methods has demonstrated effectiveness in specific contexts, none addresses the specific challenge of integrating a lightweight, model-free, retraining-free corrector into the DRL state-input pipeline for real-time ADN voltage control.

Table 1. Comparison of Methods.

Feature	Kalman Filter	State Estimation	Forecasting-Residual	Anomaly Detection	Robust/Adversarial DRL	Proposed DBCO-TD3
Model-free	×	×	√	√	√	√
Corrects (not just detects)	√	√	×	×	×	√
Embedded in DRL control loop	×	×	×	×	√	√
Plug-and-play for pre-trained agent	×	×	×	×	×	√
No retraining required	√	×	√	√	×	√
Attack-pattern-insensitive	√	√	×	√	×	√
Lightweight per-node computation	√	×	√	√	×	√

In summary, the proposed DBCO-TD3 uniquely combines the following desirable properties that are not simultaneously present in any existing method: (1) it is model-free and requires no network topology or line parameters; (2) it corrects corrupted measurements continuously rather than merely detecting anomalies; (3) it is tightly coupled with the DRL decision loop; (4) it is plug-and-play for already-deployed agents without retraining; (5) it is attack-pattern-insensitive; and (6) it maintains lightweight, real-time computational complexity suitable for edge deployment.

3.5. Two-Stage PV Inverter Power Constraint

In DRL, agents usually output normalized control ratios to regulate device output. Specifically, for continuous action spaces, the agent’s output is normalized within the interval $[-1,1]$. The sign indicates the output direction, whereas the magnitude corresponds to 0–100% of the controllable output of the device’s output respectively. Accordingly, the conventional inverter control law can be expressed as follows.

$$Q_{i,t}^{\mathrm{PV},\mathrm{use}}=A_{i,t}^{\mathrm{PVQ}}\sqrt{{\left(S_{i,t}^{\mathrm{PV}}\right)}^2-{\left(P_{i,t}^{\mathrm{PV}}\right)}^2}$$

(41)

where $Q_{i,t}^{\mathrm{PV},\mathrm{use}}$ represents the reactive power injected into the ADN; $\sqrt{{\left(S_{i,t}^{\mathrm{PV}}\right)}^2-{\left(P_{i,t}^{\mathrm{PV}}\right)}^2}$ represents the currently available reactive power capacity. However, the control variable $A_{i,t}^{\mathrm{PVQ}}$ in Equation (39) is normalized within the range of $[-1,1]$, and $\sqrt{{\left(S_{i,t}^{\mathrm{PV}}\right)}^2-{\left(P_{i,t}^{\mathrm{PV}}\right)}^2}$ is not a constant. This nonlinear interactive relationship may reduce the training efficiency and control performance of the agent. Based on these considerations, a segmented inverter-constraint strategy is proposed which decomposes reactive power control into two independent phases: positive and negative, as depicted as formulated in Equations (42) and (43).

$$Q_{i,t}^{\mathrm{PV},\mathrm{use}}=\min \left(A_{i,t}^{\mathrm{PVQ}}{Q}_{i,\max }^{\mathrm{PV}},\sqrt{{\left(S_{i,t}^{\mathrm{PV}}\right)}^2-{\left(P_{i,t}^{\mathrm{PV}}\right)}^2}\right), \forall i\in N,t\in T, A_{i,t}^{\mathrm{PVQ}}>0$$

(42)

$$Q_{i,t}^{\mathrm{PV},\mathrm{use}}=\max \left(A_{i,t}^{\mathrm{PVQ}}{Q}_{i,\max }^{\mathrm{PV}},\left(-\sqrt{{\left(S_{i,t}^{\mathrm{PV}}\right)}^2-{\left(P_{i,t}^{\mathrm{PV}}\right)}^2}\right)\right), \forall i\in N,t\in T, A_{i,t}^{\mathrm{PVQ}}<0$$

(43)

where $Q_{i,\max }^{\mathrm{PV}}$ is the maximum reactive power of the inverter, and its value is limited by the rated capacity of the equipment. This scheme significantly reduces the complexity of policy learning by the agent by decoupling the control logic for positive and negative directions. Moreover, it explicitly introduces min and max functions to ensure that the output remains within a physically feasible range, thereby improving system stability and robustness.

4. Experiments

4.1. Systems and Parameters

The simulation-based experiments are implemented on a hardware platform consisting of an Intel Core i9-13900HX CPU and an NVIDIA RTX4070 GPU. PV units are installed at nodes 10, 13, 15, 18, 20, 22, 23, 25, 27, 29, 30, and 32 of the IEEE 33-bus test system. The capacity of each PV inverter is set to 1 MW. The reference voltage level is set to 12.66 kV, with the root node voltage standard value set at 1.00 p.u. The safe voltage limits range from 0.95 p.u. to 1.05 p.u. Typical daily data for the PV system are shown in Figure 4, and the constructed data distribution is shown in Figure 5.

Figure 4. Typical Daily PV Data.

Figure 5. Data Distribution.

The reinforcement learning environment is configured to run 3000 training episodes, with each episode using one randomly selected day of data for training. To enhance the model’s generalization and improve control performance under stochastic ADN operating conditions, both the training samples and the ADN’s initial state are randomized. The agent’s action time interval is set to one hour, which corresponds to a VVC control resolution of one hour. All simulations in this paper are based on the TD3 algorithm. The parameters of TD3 are shown in Table 2. The initial state of the system is set to its actual operating condition, while the initial observation, action, and reward of the agent are all set to zero. The reward curves presented in this paper are all computed using a moving average over 100 episodes. The test results are obtained by averaging the data collected over a 30-day testing period. To ensure reproducibility and prevent data leakage, all data are strictly partitioned in chronological order into three non-overlapping subsets. The historical distribution data (the 190 consecutive days immediately preceding the training period) are used exclusively to pre-compute the per-time-step mean and standard deviation of PV outputs required by the DBCO module, serving as the static baseline for online correction. The training data (the subsequent period) are used for the 3000-episode DRL training process; within each episode, data are processed strictly in chronological hourly steps without any cross-day concatenation, ensuring that the agent never observes future information before making decisions. The test results are obtained by averaging the data collected over 30 independent test days (episodes) using a single trained agent with a fixed random seed (seed = 2).

Table 2. Algorithm Parameter Settings.

Parameters	Value
Random seed	2
Stochastic exploration noise	0.1
Batch size	128
Replay buffer size	1,000,000
Discount Factor	0.98
Sample time	1
Learning rate for actor-network	0.005
Learning rate for critic-network	0.005
Soft update coefficient	0.002
Target policy smoothing coefficient	0.002
Policy update frequency	2
Target update frequency	2
Number of agents	1

4.2. Verification of Two-Stage PV Inverter Power Constraint

In this validation section, six scenarios are designed based on common control objectives in VVC for verification, as detailed in Table 3. Furthermore, the proposed observer is not used in this section. The method for setting the coefficients in Equation (4) is detailed in [34].

Table 3. Configuration for Different Scenarios.

Cases	Constraint	Objectives
Scenario 1	Proposed	voltage deviation and voltage violation
Scenario 2	Proposed	power loss and voltage violation
Scenario 3	Proposed	voltage deviation, power loss, and voltage violation
Scenario 4	Traditional	Same as Scenario 1
Scenario 5	Traditional	Same as Scenario 2
Scenario 6	Traditional	Same as Scenario 3

As shown in Figure 6, under the same control objective, the adoption of the two-stage PV inverter constraint significantly enhances the training performance of TD3. Specifically, by comparing the reward trajectories of Scenario 1 vs. Scenario 4, Scenario 2 vs. Scenario 5, and Scenario 3 vs. Scenario 6, the results indicate that the training process of TD3 with the two-stage inverter constraint is more stable and exhibits faster convergence. This constraint compresses the originally unbounded action space into a physically feasible convex set, thereby preventing the agent from exploring invalid regions and enabling the critic network to evaluate action values more accurately.

Figure 6. Reward Trajectories in Different Scenarios.

The extreme difference in Table 4 is calculated from $AvgVolatility={\displaystyle \sum _{n=1}^{33}\left({\max }_t{V}_{t,n}-{\min }_t{V}_{t,n}\right)}/33$. This metric indicates the degree of voltage fluctuation. By comparing Table 4 and Figure 7, Figure 8, Figure 9, Figure 10, Figure 11 and Figure 12, the six scenarios exhibit different performance in voltage quality and power loss. Scenario 4 shows acceptable performance in maintaining voltage close to the nominal value, but exhibits relatively high power loss and moderate voltage fluctuation. Scenario 5 successfully reduces the power loss to the lowest among all scenarios; however, this comes at the expense of the largest total voltage deviation and pronounced voltage oscillations. Scenario 6, which attempts to balance voltage deviation, loss, and violation penalties, results in the most imbalanced outcome: it presents the highest voltage volatility across all scenarios while still retaining elevated loss levels.

Table 4. Comparison of Control Outcomes Across Different Scenarios.

Scenario	${\mathit{V}}_{\mathbf{max}}$	${\mathit{V}}_{\mathbf{min}}$	${\mathit{b}}_{1,\mathit{t}}$	${\mathit{b}}_{2,\mathit{t}}$	Extreme Difference	Solving Time	Training Time
No control	1.1572	0.9009	31.1018	0.6392	4.0102	/	/
Scenario 1	1.0276	0.9767	4.4319	1.1333	0.7216	0.013 s	1.86 h
Scenario 2	1.0470	0.9758	10.2456	1.1352	1.1263	0.018 s	1.85 h
Scenario 3	1.0360	0.9691	5.1693	1.0470	0.7505	0.016 s	1.89 h
Scenario 4	1.0460	0.9508	6.0073	1.1326	1.2130	0.014 s	1.84 h
Scenario 5	1.0393	0.9565	10.5488	1.0132	1.3192	0.009 s	1.86 h
Scenario 6	1.0403	0.9560	7.4004	1.1411	1.3982	0.015 s	1.89 h

Figure 7. Voltage in Scenario 1.

Figure 8. Voltage in Scenario 2.

Figure 9. Voltage in Scenario 3.

Figure 10. Voltage in Scenario 4.

Figure 11. Voltage in Scenario 5.

Figure 12. Voltage in Scenario 6.

In contrast, the scenarios employing the two-stage inverter constraint generally demonstrate superior voltage concentration and operational smoothness. Among them, Scenario 1 achieves the best voltage-control performance: its voltages remain closest to the nominal value, fluctuations are minimal, and the voltage operating range is the most desirable. Scenario 2 achieves a considerable reduction in power loss, but at the cost of substantially increased voltage deviation. Scenario 3 provides a favorable compromise, achieving low levels in all three aspects—voltage deviation, power loss, and fluctuation—and thus achieves the best overall performance in terms of comprehensive operational quality. Therefore, Scenario 3 is selected as the benchmark for the proposed DBCO-TD3.

4.3. Verification of DBCO-TD3

The FDIAs settings are shown in Table 5. The attack is modeled as a random scaling attack. During the training phase, an attack is launched with a specified probability in the current episode with a certain probability. When an attack occurs, at each time step, 5 to 12 DPV units are randomly selected as attack targets, and for each attacked DPV, a random attack magnitude between 0% and 100% is independently generated. Specifically, if the normal active power of the DPV at that time step is zero, the observed value is modified to the rated power multiplied by the random attack magnitude; if the normal value is non-zero, the observed value is tampered with as the actual power multiplied by the random attack magnitude. The detailed comparison schemes are presented in Table 6 and Table 7, where adversarial training refers to the strategy in which the agent learns the characteristics of attacks during training to mitigate their impact.

Table 5. Setting of FDIAs.

Parameter Categories		Parameter Values
Attack Rules	Zero-Power Time-Step Attacks	Observed value = Rated power × Random attack amplitude
	Non-Zero-Power Time-Step Attacks	Observed value = Actual power × Random attack amplitude
Target of the Attack	Attack PV Selection	Random selection
	Number of Attacks	5 to 12 random selections

Table 6. Comparative Experiment Setup.

Group	Comparison Number	Training Phase Settings	Control Phase Settings
Baseline (Scenario 3 in Section 4.2)	Comparison 1	No attack	An attack occurred
Robust Optimization	Comparison 2	/	Each test has a 50% chance of triggering an attack.
Adversarial Training TD3	Comparison 3	Probability of an attack occurring in each episode: 6.25%
	Comparison 4	Probability of an attack occurring in each episode: 12.5%
	Comparison 5	Probability of an attack occurring in each episode: 25%
	Comparison 6	Probability of an attack occurring in each episode: 50%
	Comparison 7	Probability of an attack occurring in each episode: 100%
DBCO-TD3 (Used Two-stage PV inverter power constraint)	Comparison 8	Same as Comparison 3
	Comparison 9	Same as Comparison 4
	Comparison 10	Same as Comparison 5
	Comparison 11	Same as Comparison 6
	Comparison 12	Same as Comparison 7
	Comparison 13	No attack

Table 7. DBCO-TD3 under Different PV Penetration Levels.

Penetration Levels	Comparison Number	Training Phase Settings	Control Phase Settings
50%	Comparison 14	Probability of an attack occurring in each episode: 100%	Each test has a 50% chance of triggering an attack.
100%	Comparison 15
150%	Comparison 16

As shown in Figure 13 and Table 8, when the agent trained under Scenario 3 in Section 4.2 is deployed in an environment with FDIAs, the control strategy of the agent fails to maintain voltage constraints due to the disruption of observations caused by the attacks. Figure 14 presents the voltage control results of robust optimization, which serves as the theoretical optimum for comparison.

Figure 13. Voltage in Comparison 1.

Table 8. Voltage Data for Comparison 1–2.

Comparison Experiment Number	Mean (p.u.)	Median (p.u.)	Standard Deviation (p.u.)	Minimum (p.u.)	Maximum (p.u.)	Cumulative Number of Nodes Crossing the Threshold	Solving Time	Training Time
Comparison 1	0.9940	0.9982	0.0448	0.7598	1.1417	133	0.012 s	1.89 h
Comparison 2	0.9997	0.9976	0.0088	0.9833	1.0356	0	36.45 s	/

Figure 14. Voltage in Comparison 2.

As shown in Figure 15, Figure 16, Figure 17, Figure 18 and Figure 19 and Table 9, adversarial training TD3 exhibits a clear trend of performance improvement as the attack probability increases from 6.25% to 100%. At low attack probabilities, such as 6.25% and 12.5%, the mean and standard deviation are relatively large, and voltage violations occur, indicating insufficient algorithmic control stability under low-level disturbances. As the attack probability increases to 25% and 50%, the mean and variance decrease notably, with violations dropping to 37 and 11 nodes, respectively. When the attack probability reaches 100%, the number of violations is reduced to zero, the mean is very close to 1.0 (0.9976), and the standard deviation is only 0.0124, indicating that a satisfactory control strategy is achieved only after the algorithm has sufficiently learned the attack characteristics.

Figure 15. Voltage in Comparison 3.

Figure 16. Voltage in Comparison 4.

Figure 17. Voltage in Comparison 5.

Figure 18. Voltage in Comparison 6.

Figure 19. Voltage in Comparison 7.

Table 9. Voltage Data for Comparison 3–13.

Comparison Experiment Number	Mean (p.u.)	Median (p.u.)	Standard Deviation (p.u.)	Minimum (p.u.)	Maximum (p.u.)	Cumulative Number of Nodes Crossing the Threshold	Solving Time	Training Time
Comparison 3	1.0001	0.9982	0.0294	0.9185	1.1387	6	0.019 s	/
Comparison 4	1.0000	0.9992	0.0286	0.8769	1.1137	61	0.023 s	1.91 h
Comparison 5	1.0051	1.0003	0.0217	0.9516	1.1182	37	0.016 s	1.90 h
Comparison 6	0.9964	0.9981	0.0160	0.9214	1.0379	11	0.018 s	1.95 h
Comparison 7	0.9976	0.9978	0.0124	0.9538	1.0343	0	0.016 s	1.96 h
Comparison 8	1.0008	0.9995	0.0128	0.9648	1.0374	0	0.019 s	1.92 h
Comparison 9	0.9997	0.9995	0.0123	0.9620	1.0401	0	0.016 s	1.91 h
Comparison 10	0.9972	0.9977	0.0132	0.9579	1.0376	0	0.021 s	1.93 h
Comparison 11	0.9962	0.9968	0.0125	0.9643	1.0405	0	0.015 s	1.94 h
Comparison 12	0.9969	0.9971	0.0131	0.9515	1.0358	0	0.022 s	1.93 h
Comparison 13	1.0023	0.9964	0.0153	0.9554	1.0405	0	0.021 s	1.89 h

As shown in Figure 20, Figure 21, Figure 22, Figure 23 and Figure 24 and Table 9, under the FDIAs identification and correction mechanism proposed in this paper, the DBCO-TD3 algorithm exhibits consistently excellent stability across all attack scenarios. No voltage violations occur in any experiment, with the mean fluctuating slightly between 0.9962 and 1.0008, and the standard deviation controlled within the extremely narrow range of 0.0123 to 0.0132. In contrast to the adversarial training control strategy, this mechanism already achieves zero violations at low attack probabilities, such as 6.25%, demonstrating its stronger fundamental capability against disturbances. Variations in attack probability have minimal impact on the median and extreme values of control accuracy, with the maximum never exceeding 1.041 and the minimum never falling below 0.9515, indicating a high degree of robustness. This demonstrates that the algorithm can achieve satisfactory control strategies even without sufficiently learning the characteristics of the attacks. Furthermore, as shown in Figure 25 and Table 9 (Comparison 13), the DBCO-equipped TD3 algorithm is still capable of delivering a control strategy that satisfies voltage constraints even when no training against attacks is performed. Although the performance is inferior to that of Robust Optimization, DBCO-TD3 is significantly faster and more suitable for real-time response requirements. It should be noted that for the adversarial trained TD3 comparison group, no attacks occur during the training phase while attacks are present during the control phase, which is essentially the same as Comparison 1.

Figure 20. Voltage in Comparison 8.

Figure 21. Voltage in Comparison 9.

Figure 22. Voltage in Comparison 10.

Figure 23. Voltage in Comparison 11.

Figure 24. Voltage in Comparison 12.

Figure 25. Voltage in Comparison 13.

As shown in Figure 26, Figure 27 and Figure 28, when the DBCO mechanism is evaluated under different PV penetration levels, such as 50%, 100%, and 150%, it still achieves satisfactory voltage control performance.

Figure 26. Voltage in Comparison 14.

Figure 27. Voltage in Comparison 15.

Figure 28. Voltage in Comparison 16.

By comparing the solution times in Table 4, Table 8 and Table 9, it can be observed that DBCO introduces only a slight computational overhead compared with the baseline TD3 setting, while remaining substantially faster than robust optimization. Therefore, the improvement in voltage stabilization outweighs the limited impact on computational efficiency.

To further investigate why the control performance in Comparisons 8–13 outperforms that of Scenario 3 in Section 4.2, the two-stage PV inverter power constraint is removed from the settings of Comparisons 8–13, and additional Comparisons 17–22 are conducted using TD3 with DBCO only. The corresponding configurations and results are presented in Table 10 and Table 11, respectively.

Table 10. The Comparative Configuration for TD3 Incorporating DBCO Exclusively.

Group	Comparison Number	Training Phase Settings	Control Phase Settings
TD3 solely with DBCO	Comparison 17	Same as Comparison 8	Each test has a 50% chance of triggering an attack.
	Comparison 18	Same as Comparison 9
	Comparison 19	Same as Comparison 10
	Comparison 20	Same as Comparison 11
	Comparison 21	Same as Comparison 12
	Comparison 22	Same as Comparison 13

Table 11. Voltage Data for Comparison 17–22.

Comparison Experiment Number	Mean (p.u.)	Median (p.u.)	Standard Deviation (p.u.)	Minimum (p.u.)	Maximum (p.u.)	Cumulative Number of Nodes Crossing the Threshold	Solving Time
Comparison 17	0.9996	0.9983	0.0150	0.9610	1.0408	0	0.018 s
Comparison 18	0.9985	0.9982	0.0145	0.9582	1.0435	0	0.015 s
Comparison 19	0.9958	0.9964	0.0156	0.9540	1.0410	0	0.023 s
Comparison 20	0.9947	0.9955	0.0150	0.9605	1.0438	0	0.016 s
Comparison 21	0.9953	0.9958	0.0156	0.9576	1.0392	0	0.020 s
Comparison 22	1.0011	0.9951	0.0180	0.9510	1.0441	0	0.022 s

The key results are presented in Table 11. As shown in Table 11, Comparisons 17–22 indicate that TD3 with DBCO can still achieve voltage control under FDIAs even without the two-stage PV inverter power constraint. Specifically, its control performance outperforms that of Adversarial Training TD3 and the TD3 with only the proposed two-stage inverter constraint (Baseline), while its solution speed is faster than that of Robust Optimization. However, its effectiveness is inferior to that of DBCO-TD3 with the two-stage PV inverter power constraint.

To verify the scalability of the complete DBCO-TD3 framework, the test system is extended to a 123-bus distribution system, with PV units installed at buses 3, 6, 8, 11, 14, 16, 19, 21, 24, 26, 28, 31, 33, 36, 38, 41, 43, 46, 48, 50, 53, 55, 58, 60, 63, 65, 68, 70, 73, 75, 78, 80, 83, 85, 88, 90, 93, 95, 98, 100, 103, 105, 108, 110, and 113. The PV models and typical outputs are the same as those in the 33-bus distribution system. The comparison settings for Comparisons 23–28 are shown in Table 12.

Table 12. Setup of comparisons in the 123-bus system.

Group	Comparison Number	Training Phase Settings	Control Phase Settings
DBCO-TD3 (Used Two-stage PV inverter power constraint)	Comparison 23	Same as Comparison 8	Each test has a 50% chance of triggering an attack.
	Comparison 24	Same as Comparison 9
	Comparison 25	Same as Comparison 10
	Comparison 26	Same as Comparison 11
	Comparison 27	Same as Comparison 12
	Comparison 28	Same as Comparison 13

The key results for Comparisons 23–28 are presented in Table 13. The complete DBCO-TD3 framework is still able to provide control schemes that satisfy the voltage requirements when extended to the 123-bus system.

Table 13. Voltage Data for Comparison 23–28.

Comparison Experiment Number	Mean (p.u.)	Median (p.u.)	Standard Deviation (p.u.)	Minimum (p.u.)	Maximum (p.u.)	Cumulative Number of Nodes Crossing the Threshold	Solving Time
Comparison 23	0.9992	0.9978	0.0165	0.9512	1.0420	0	0.264 s
Comparison 24	0.9980	0.9975	0.0160	0.9585	1.0452	0	0.222 s
Comparison 25	0.9952	0.9958	0.0172	0.9538	1.0425	0	0.292 s
Comparison 26	0.9940	0.9948	0.0168	0.9501	1.0458	0	0.208 s
Comparison 27	0.9945	0.9950	0.0175	0.9565	1.0405	0	0.306 s
Comparison 28	1.0005	0.9942	0.0198	0.9503	1.0465	0	0.292 s

5. Conclusions

A DRL-based VVC framework is proposed for ADNs that addresses two key challenges: the mismatch between DRL action characteristics and PV inverter constraints, and the vulnerability to FDIAs targeting PV measurements. To resolve the first issue, a two-stage inverter power constraint is introduced to decouple reactive power control into independent positive and negative phases, aligning the agent’s action space with physical equipment limits and significantly improving training efficiency and control stability. To counter FDIAs, a distribution-based correction observer (DBCO) is designed as a lightweight, model-free preprocessing module that continuously corrects corrupted PV observations in real time. Unlike detection-only or retraining-dependent methods, DBCO is plug-and-play for pre-trained agents and requires no structural modifications. Simulations on IEEE 33-bus and 123-bus systems demonstrate that DBCO-TD3 maintains zero voltage violations under various attack intensities and PV penetration levels, outperforming adversarial training approaches in both control accuracy and computational efficiency.

Future work will extend the current framework in several directions. First, while this study focuses on PV active-power measurement manipulation under random scaling attacks, other attack vectors—including voltage measurement manipulation, load measurement manipulation, and coordinated multi-node attacks—remain to be investigated. Second, low-amplitude stealthy attacks that fall within the normal statistical bounds pose a greater challenge to the DBCO’s detection mechanism and warrant further study. Third, adaptive distribution-tracking and update mechanisms will be explored to address non-stationary PV output variations over time. Fourth, the method will be extended to large-scale distribution systems, and distributed or multi-agent architectures will be investigated to alleviate the computational and communication bottlenecks of centralized algorithms. More realistic PV data distributions will also be adopted to further validate the proposed DBCO module.