Dynamic State Estimation for Sustainable Distribution Systems Considering Data Correlation and Noise Adaptiveness

Abstract

The integration of distributed renewable energy sources into distribution networks is a key approach to achieving sustainable and low-carbon power systems. However, high renewable penetration significantly increases the volatility and uncertainty of distribution systems, posing challenges to renewable energy accommodation and reliable operation. To address these challenges, active control of distribution networks is required, which in turn relies on accurate system states. In practice, the limited number and accuracy of measurement devices in distribution networks make dynamic state estimation a critical technology for sustainable distribution systems. In this paper, a novel dynamic state estimation method for sustainable distribution systems is proposed, incorporating spatiotemporal data correlation and adaptiveness to process and measurement noise. A CNN-BiGRU-Attention model is developed to reconstruct high-accuracy real-time pseudo-measurements, compensating for insufficient sensing infrastructure. Furthermore, a noise adaptive dynamic state estimation method is proposed based on an improved unscented Kalman filter. An amplitude modulation factor (AMF) is applied to track time-varying process noise, while an evaluation method based on robust Mahalanobis distance (RMD) is embedded to deal with non-Gaussian measurement noise. Finally, simulation studies on the IEEE 33-bus three-phase unbalanced distribution network demonstrate the effectiveness and robustness of the proposed method.

1. Introduction

Serving as a critical function for the intelligent management of distribution networks, state estimation helps identify weak points in the network and propose remedial measures to improve system reliability [1]. It utilizes readings of power, voltage, and current gathered from conventional systems and devices—including supervisory control and data acquisition (SCADA) systems, phasor measurement units (PMUs), and smart metering devices—for estimating the true conditions of the electric power system [2]. With the widespread integration of controllable resources like electric vehicles (EVs) and rooftop photovoltaics (PV) into distribution networks, enhancing their control capabilities has become a key direction for the development of power systems [3,4]. Consequently, distribution network state estimation is receiving increasing attention [5].

In contrast to transmission networks, which are equipped with a large number of secondary data acquisition devices like PMUs, distribution networks have a relatively weak measurement infrastructure. Despite the gradual integration of SCADA systems and Micro-PMUs, their widespread, high-density deployment faces restrictions due to high economic costs, the vast quantity of network nodes, and inadequate communication infrastructure [6]. This leads to a common problem of low measurement coverage in distribution networks, accompanied by challenges such as insufficient measurement accuracy and low refresh rates. These factors result in a data redundancy far below what is required for state estimation, making it difficult to support accurate and real-time state estimation for distribution networks [7]. To address the issue of measurement sparsity, pseudo-measurement data have been identified as an efficient means of enhancing the observability of distribution networks. Pseudo-measurement generation models fall into two major classes: statistical analysis models and machine learning models [8]. Statistical analysis models typically use historical measurement data to infer missing measurements through methods such as non-parametric least-squares density estimation [9] and kernel density estimation [10]. Conversely, machine learning models yield more precise pseudo-measurement data by exploiting the intricate, non-linear dependencies embedded in prior data. These approaches include attention-enhanced recurrent neural networks [11], support vector machines [12], and generative adversarial networks (GANs) [13]. However, most existing studies either focus solely on the temporal correlation of available measurement data or fail to balance the depth of spatiotemporal feature extraction with computational efficiency. The loads and distributed energy generation at different nodes inherently exhibit significant spatial correlation [14]. While GANs demonstrate strong capabilities in modeling complex data distributions, they often entail high computational complexity and training instability due to the adversarial min–max optimization process, which can be burdensome for real-time applications. Therefore, comprehensively considering and effectively exploiting the spatiotemporal correlation among measurement data while maintaining high computational efficiency is a potential key to further improving the accuracy of pseudo-measurement generation.

Due to the increasing randomness and volatility of distribution networks, traditional static state estimation, which relies solely on information from a single time snapshot, struggles to cope with rapid fluctuations in system states, leading to delayed or inaccurate estimation results. Built upon the framework of the Kalman filter (KF), dynamic state estimation continuously tracks the state transition trajectory across multiple time slots, allowing for a more accurate depiction of system dynamics. In tackling the pronounced non-linearity inherent in the distribution network’s load flow characteristics, researchers have put forth techniques including an extended Kalman filter (EKF) [15], UKF [16], and a cubature Kalman filter (CKF) [17]. The EKF achieves linearization of the electrical system by relying on a first-order Taylor expansion, a process that inevitably results in truncation errors [18]. These errors increase as the system’s non-linearity deepens, potentially leading to decreased estimation accuracy or even numerical instability. To avoid EKF’s linearization errors, the UKF and CKF employ a deterministic sampling strategy to handle non-linear transformations, significantly improving estimation performance for non-linear systems. However, these methods still face two major challenges: (1) when system dynamics change drastically, the actual process noise distribution may not own a fixed covariance matrix, which reduces the tracking performance and robustness; (2) the effects of these methods are based on the supposition that measurement noise adheres to a standard normal distribution, which is inconsistent with real-world physical systems [19].

To handle time-varying process noise, Ref. [20] proposed the Sage–Husa noise model, which calculates and adjusts the process noise covariance dynamically. Ref. [21] embedded a sub-optimal fading factor to enhance the response to dynamic changes in process noise, while Ref. [22] achieved adaptive estimation by adjusting a modulation factor. However, the values or decay strategies of the modulation factors in these methods are often heuristically fixed, lacking the flexibility to autonomously adjust to the time-varying intensity of process noise in real-time. This limitation restricts their estimation accuracy during severe dynamic state mutations. Therefore, a crucial current research direction is how to dynamically calibrate the forgetting factor by analyzing real-time changes in the innovation sequence to achieve more precise state variable estimation.

To handle the impact of non-Gaussian measurement noise, existing methods mainly fall into a few categories: filters based on robust statistics [23,24]; filters based on information-theoretic criteria [25]; and filters based on probabilistic models [26]. Although these methods have, to some extent, solved the non-Gaussian noise problem, they generally suffer from high computational complexity and sensitivity to parameter selection, thus failing to satisfy the demanding requirements for real-time operation and adaptation in distribution networks. Therefore, the ability to quickly and accurately identify non-Gaussian noise is key to improving the performance of state estimation in distribution networks.

A dynamic state estimation approach for distribution networks is presented in this paper, leveraging the spatiotemporal correlation of data and adaptiveness to process and measurement noise. The principal achievements of this work are listed below:

(1) Pseudo-measurement generation: A CNN-BiGRU-Attention model is presented to generate highly accurate pseudo-measurement data by effectively extracting both the spatial correlation within the network topology and the temporal correlation in the data.

(2) Dynamic state estimation: We propose an innovative unscented Kalman filter with adaptiveness to process and measurement noise (NA-UKF). This method includes a process noise adaptive estimation component based on an AMF and a measurement noise adaptive estimation component based on RMD. Compared to existing methods in the literature [27,28,29], the proposed algorithm more accurately tracks the network state and exhibits high resilience to both time-varying process and non-Gaussian measurement uncertainties.

The subsequent sections of this manuscript are structured as follows. The overall architecture is presented in Section 2. Section 3 is then dedicated to the pseudo-measurement generation model, which accounts for spatiotemporal correlation. Section 4 provides a detailed description of the NA-UKF algorithm. The algorithm’s effectiveness is assessed and confirmed in Section 5, and the final section, Section 6, summarizes the paper.

2. Overall Framework

In distribution networks, the main measurement information is provided by two different types of systems: SCADA systems and Micro-PMUs. SCADA systems are the most widely deployed in distribution networks due to their lower single-point cost. These terminals can be installed at nodes or on branches and primarily provide measurement data such as active and reactive power, but their accuracy and reporting rate are relatively low. In contrast, Micro-PMUs have a higher single-point cost and are deployed in smaller numbers, yet they can provide high-accuracy, very high-reporting-rate synchronous phasor measurement data. They are typically installed at critical nodes to measure nodal voltage phasors. In hybrid-measurement-based distribution network state estimation, the SCADA reporting rate is used as the base frequency for state estimation, matching the mainstream sampling capabilities of existing terminals. Then, the system error is corrected using the high-accuracy phasor data from Micro-PMUs, achieving a collaborative optimization of accuracy and economic efficiency.

Figure 1 presents the overall architecture of the study. Within the pseudo-measurement generation process, an offline-trained CNN-BiGRU-Attention model is used to fully leverage the spatiotemporal correlation of distribution network measurement data. This model generates pseudo-measurement data for nodes or branches without measurement devices. The input to the model includes historical and current measurement data; the output consists of the current pseudo-measurement data for the nodes and branches with missing measurements. In the dynamic state estimation stage, we employ the proposed NA-UKF algorithm. This model utilizes the temporal information of the measurement data and adaptively adjusts to its noise characteristics, allowing for continuous tracking and accurate estimation of the system state. Finally, the state variables, namely the magnitude and phase angle associated with the nodal voltages, are calculated.

3. A Pseudo-Measurement Generation Model Considering Spatiotemporal Correlation

Distribution network loads and distributed energy generation not only exhibit significant time-series characteristics but are also influenced by spatial coupling due to geographical location, neighboring node loads, similar load types and the complex radial topology. Existing pseudo-measurement generation methods based on neural networks often neglect the intrinsic spatiotemporal correlation within historical distribution network data, resulting in pseudo-measurement data that lacks sufficient precision. To address these deficiencies, this paper introduces a pseudo-measurement generation framework that leverages the CNN-BiGRU-Attention neural network.

3.1. Model Input and Output

The pseudo-measurement generation model undergoes training under a supervised learning paradigm. The construction of the dataset is a critical foundation for ensuring model performance. The input vector T_i is defined as

$$T_i={[P_{inj,Micro-PMUs} Q_{inj,Micro-PMUs} P_{inj,SCADA} Q_{inj,SCADA} P_{l,SCADA} Q_{l,SCADA} P_{inj} Q_{inj} P_l Q_l]}^T$$

(1)

where P, Q denote active and reactive power, respectively; subscripts inj and l refer to nodal injection and branch power flow. Subscripts Micro-PMUs and SCADA indicate historical measurements from Micro-PMUs and SCADA devices. Variables without device subscripts (i.e., P_inj, Q_inj, P_l, Q_l) represent historical data derived from power flow calculations based on historical load records. The output vector T_o is expressed as

$$T_o={\left[P_{inj,pseudo} Q_{inj,pseudo} P_{l,pseudo} Q_{l,pseudo}\right]}^T$$

(2)

where the subscript pseudo denotes the generated pseudo-measurements for unmeasured locations.

After completion of the network training, the model operates in an online spatial imputation mode. To ensure high accuracy in real-time deployment, a rolling prediction strategy is adopted. By inputting the measurement data sequence which includes both the historical data (from T − L to T − 1) and the actual real-time readings captured by SCADA/Micro-PMUs at the current time slot T, we can infer the pseudo-measurement data for all nodes and branches without measurement configuration at the same time slot T. This mechanism continuously slides the input window forward as new data arrives, minimizing error accumulation. This provides more accurate and comprehensive real-time pseudo-measurement information for subsequent dynamic state estimation.

3.2. CNN-BiGRU-Attention Model

Leveraging the inherent spatiotemporal correlations within the data to enhance the precision of pseudo-measurement generation, this paper constructs a CNN-BiGRU-Attention model, with its network architecture presented in Figure 2.

To effectively encode the distribution network’s topology, the input measurement matrix is constructed based on a depth-first search traversal sequence. This serialization maps physically connected or electrically adjacent nodes to proximal positions in the input tensor, ensuring that the topological structure is preserved in the 1-D data format.

The CNN is utilized to effectively capture the spatial correlation and topological dependencies between different nodes and branches [7]. It leverages local connectivity to learn data features across different ranges of dimensions. Specifically, the convolutional layers enhance feature extraction by applying convolution operators to the input measurements, utilizing learnable weights and biases to map raw data into high-level spatial feature maps. This process allows the model to automatically abstract intricate spatial dependencies between nodes. Subsequently, pooling layers are utilized to down-sample the features extracted by the convolutional layers, thereby reducing information redundancy while preserving the most critical spatial features.

To capture temporal features, the model incorporates a BiGRU layer. By combining the forward and backward hidden state information, the BiGRU layer can comprehensively mine the periodic patterns and bidirectional temporal correlation within historical data. This substantially improves the model’s capacity to detect dynamic variations within the pseudo-measurement data. The feature vector extracted by the CNN is used as the input for the BiGRU layer.

Finally, an Attention network is introduced to dynamically assign weights to data from different types of measurement sources (e.g., SCADA, Micro-PMUs) and key nodes (e.g., PV power stations, EV charging stations). This empowers the model to selectively concentrate upon essential features for generating more accurate pseudo-measurements.

During the model training process, this paper adopts mean squared error (MSE) as the objective loss function. The mathematical formula is provided as follows:

$$\mathrm{MSE}={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n{(y_i-{\widehat{y}}_i)}^2}$$

(3)

Finally, the high-dimensional feature vector, processed and refined by the CNN, BiGRU and Attention layers, is non-linearly mapped through a fully connected layer to output the ultimate pseudo-measurement data.

4. A Dynamic State Estimation Method for Distribution Networks Considering Adaptiveness to Process and Measurement Noise

4.1. Dynamic State Estimation Model for Distribution Networks

The mathematical model for dynamic state estimation in distribution networks comprises the state transition functions f(·) and the measurement function h(·). The state variable vector x is defined as ${[V_1^{\varphi },\cdot \cdot \cdot ,V_k^{\varphi },\cdot \cdot \cdot ,V_N^{\varphi },{\theta }_1^{\varphi },\cdot \cdot \cdot ,{\theta }_k^{\varphi },\cdot \cdot \cdot ,{\theta }_N^{\varphi }]}^T$. The measurement variable vector z is defined as ${[V_i^{\varphi },{\theta }_i^{\varphi },P_i^{\varphi },Q_i^{\varphi },P_{ij}^{\varphi },Q_{ij}^{\varphi }]}^T$.

Given the stochastic nature of load and renewable generation in distribution networks, the system state transition is modeled as a random walk process [30]. The state transition equation is expressed as x_k+1 = x_k + w_k, where w_k denotes the process noise vector, generally assumed to follow a Gaussian distribution with zero mean and covariance matrix Q_k. The measurement equation is expressed as z_k = h(x_k)+ v_k, where v_k denotes the measurement noise vector, also assumed to follow a Gaussian distribution with zero mean and covariance matrix R_k. The measurement function h(·) corresponding to phase $\varphi \in \{a,b,c\}$ for a three-phase distribution network is given by

$$\left\{\begin{array}{l}{h}_{V_i^{\varphi }}(x)=V_i^{\varphi }\hfill \\ h_{{\theta }_i^{\varphi }}(x)={\theta }_i^{\varphi }\hfill \\ h_{P_i^{\varphi }}(x)=V_i^{\varphi }{\displaystyle \sum _{j\in N_i}{\displaystyle \sum _{\psi \in \mathbb{P}}{V}_j^{\psi }}}\left(G_{ij}^{\varphi \psi }\cos {\theta }_{ij}^{\varphi \psi }+B_{ij}^{\varphi \psi }\sin {\theta }_{ij}^{\varphi \psi }\right)\hfill \\ h_{Q_i^{\varphi }}(x)=V_i^{\varphi }{\displaystyle \sum _{j\in N_i}{\displaystyle \sum _{\psi \in \mathbb{P}}{V}_j^{\psi }}}\left(G_{ij}^{\varphi \psi }\sin {\theta }_{ij}^{\varphi \psi }-B_{ij}^{\varphi \psi }\cos {\theta }_{ij}^{\varphi \psi }\right)\hfill \\ h_{P_{ij}^{\varphi }}(x)=V_i^{\varphi }{\displaystyle \sum _{\psi \in \mathbb{P}}{V}_i^{\psi }}\left(g_{si}^{\varphi \psi }\cos {\theta }_{ii}^{\varphi \psi }+b_{si}^{\varphi \psi }\sin {\theta }_{ii}^{\varphi \psi }\right)-V_i^{\varphi }{\displaystyle \sum _{\psi \in \mathbb{P}}{V}_j^{\psi }}\left(g_{ij}^{\varphi \psi }\cos {\theta }_{ij}^{\varphi \psi }+b_{ij}^{\varphi \psi }\sin {\theta }_{ij}^{\varphi \psi }\right)\hfill \\ h_{Q_{ij}^{\varphi }}(x)=V_i^{\varphi }{\displaystyle \sum _{\psi \in \mathbb{P}}{V}_i^{\psi }}\left(g_{si}^{\varphi \psi }\sin {\theta }_{ii}^{\varphi \psi }-b_{si}^{\varphi \psi }\cos {\theta }_{ii}^{\varphi \psi }\right)-V_i^{\varphi }{\displaystyle \sum _{\psi \in \mathbb{P}}{V}_j^{\psi }}\left(g_{ij}^{\varphi \psi }\sin {\theta }_{ij}^{\varphi \psi }-b_{ij}^{\varphi \psi }\cos {\theta }_{ij}^{\varphi \psi }\right)\hfill \end{array}\right.$$

(4)

where $V_i^{\varphi }$, ${\theta }_i^{\varphi }$ are the voltage magnitude and phase angle of phase Φ at node i. ${\theta }_{ij}^{\varphi \psi }={\theta }_i^{\varphi }-{\theta }_j^{\psi }$ is the phase angle difference. $\mathbb{P}=\{a,b,c\}$ denotes the set of phases. $N_i$ is the set of nodes connected to node i. $G_{ij}^{\varphi \psi }$, $B_{ij}^{\varphi \psi }$ are the real and imaginary parts of the element in the bus admittance matrix corresponding to node i phase Φ and node j phase ψ. $g_{ij}^{\varphi \psi }$, $b_{ij}^{\varphi \psi }$ denote the mutual conductance and susceptance of the transmission line between node i and j. $g_{si}^{\varphi \psi }$, $b_{si}^{\varphi \psi }$ denote the self-admittance and shunt parameters of the transmission line at the sending end.

4.2. Adaptive UKF Algorithm

For dynamic state estimation of non-linear distribution networks, the adaptive UKF is employed. Built upon the framework of the standard UKF, this algorithm accurately captures non-linear characteristics through its sigma-point sampling strategy. It preserves the first- and second-order moment information of the state distribution by deterministically sampling 2n + 1 sigma points, thereby avoiding the truncation errors inherent in the EKF and significantly reducing computational complexity. However, unlike the standard UKF, the adaptive UKF incorporates an adaptive factor to control the measurement prediction covariance matrix and state-measurement cross-covariance matrix. The specific steps of the adaptive UKF algorithm [18] are as follows:

First, sigma points are established through sampling the estimated state distribution to approximate the state variable’s distribution, which are generated by

$$\left\{\begin{array}{ll}{\mathcal{X}}_{k-1}^{(0)}={\widehat{x}}_{k-1}\hfill & \hfill \\ {\mathcal{X}}_{k-1}^{(i)}={\widehat{x}}_{k-1}+\sqrt{(n+\lambda )}{L}_i,\hfill & i=1,\dots ,n\hfill \\ {\mathcal{X}}_{k-1}^{(i)}={\widehat{x}}_{k-1}-\sqrt{(n+\lambda )}{L}_i,\hfill & i=n+1,\dots ,2n\hfill \end{array}\right.$$

(5)

where n denotes the dimension of the state vector, and L_i is the i-th column of the Cholesky decomposition of ${\widehat{P}}_{k-1}$. The scaling parameter is defined as $\lambda ={\alpha }^2(n+\kappa )-n$, which determines the spread of the sigma points.

The weights of the sigma points for the mean and covariance are calculated by

$$\left\{\begin{array}{l}{W}_m^{(0)}={\displaystyle \frac{\lambda }{n+\lambda }}\hfill \\ W_c^{(0)}={\displaystyle \frac{\lambda }{n+\lambda }}+(1-{\alpha }^2+\beta )\hfill \\ W_m^{(i)}=W_c^{(i)}={\displaystyle \frac{1}{2(n+\lambda )}},\hspace{1em}i=1,\cdots ,2n\hfill \end{array}\right.$$

(6)

where α is a small positive constant controlling the spread of sigma points, which is set to 10⁻³. κ is a secondary scaling parameter, which is set to 0. β is used to minimize higher-order approximation errors. For Gaussian distributions, β = 2 is optimal.

Then, the predicted state vector ${\tilde{x}}_k$ and error covariance matrix ${\tilde{P}}_k$ are calculated by

$$\left\{\begin{array}{l}{\tilde{x}}_k={\displaystyle \sum _{i=0}^{2n}}{W}_m^{(i)}{\mathcal{X}}_{k|k-1}^{(i)}\\ {\tilde{P}}_k={\displaystyle \sum _{i=0}^{2n}}{W}_c^{(i)}\left({\mathcal{X}}_{k|k-1}^{(i)}-{\tilde{x}}_k\right){\left({\mathcal{X}}_{k|k-1}^{(i)}-{\tilde{x}}_k\right)}^T+{\widehat{Q}}_{k-1}\end{array}\right.$$

(7)

where ${\widehat{Q}}_{k-1}$ is the estimated process noise covariance.

Next, the predicted measurement vector ${\tilde{z}}_k$ and the theoretical innovation covariance matrix $P_{zz,k}$ are calculated by

$$\left\{\begin{array}{l}{\tilde{z}}_k={\displaystyle \sum _{i=0}^{2n}}{W}_m^{(i)}{\mathcal{Z}}_{k|k-1}^{(i)}\\ P_{zz,k}={\displaystyle \sum _{i=0}^{2n}}{W}_c^{(i)}\left({\mathcal{Z}}_{k|k-1}^{(i)}-{\tilde{z}}_k\right){\left({\mathcal{Z}}_{k|k-1}^{(i)}-{\tilde{z}}_k\right)}^T+{\widehat{R}}_k\end{array}\right.$$

(8)

where ${\mathcal{Z}}_{k|k-1}^{(i)}$ denotes the sigma points propagated through the measurement function. ${\widehat{R}}_k$ is the estimated measurement noise covariance.

Subsequently, the adaptive factor ${\mu }_k$ is expressed as

$$\left\{\begin{array}{l} 1 tr({\epsilon }_k{\epsilon }_k^T)\le tr(P_{zz,k})\hfill \\ {\displaystyle \frac{tr(P_{zz,k})}{tr({\epsilon }_k{\epsilon }_k^T)}} tr({\epsilon }_k{\epsilon }_k^T)>tr(P_{zz,k})\hfill \end{array}\right.$$

(9)

where ${\epsilon }_k$ denotes the innovation vector, and ${\epsilon }_k=z_k-{\tilde{z}}_k$, tr(·) denotes the trace of a matrix.

Next, the corrected innovation covariance matrix $P_{zz,k}$, and the cross-covariance matrix between state and measurement $P_{xz,k}$ are calculated incorporating the adaptive factor:

$$\left\{\begin{array}{l}{P}_{zz,k}={\displaystyle \sum _{i=0}^{2n}}{W}_c^{(i)}\left({\mathcal{Z}}_{k|k-1}^{(i)}-{\tilde{z}}_k\right){\left({\mathcal{Z}}_{k|k-1}^{(i)}-{\tilde{z}}_k\right)}^T+{\mu }_k{\widehat{R}}_k\hfill \\ P_{xz,k}={\mu }_k{\displaystyle \sum _{i=0}^{2n}}{W}_c^{(i)}\left({\mathcal{X}}_{k|k-1}^{(i)}-{\tilde{x}}_k\right){\left({\mathcal{Z}}_{k|k-1}^{(i)}-{\tilde{z}}_k\right)}^T\hfill \end{array}\right.$$

(10)

Finally, the Kalman gain matrix $K_k$, the estimated state vector ${\widehat{x}}_k$ and error covariance matrix ${\widehat{P}}_k$ are computed by

$$\left\{\begin{array}{l}{K}_k=P_{xz,k}{P}_{zz,k}^{-1}\\ {\widehat{x}}_k={\tilde{x}}_k+K_k\left(z_k-{\tilde{z}}_k\right)\\ {\widehat{P}}_k={\tilde{P}}_k-K_k{P}_{zz,k}{K}_k^T\end{array}\right.$$

(11)

4.3. NA-UKF Algorithm

In real-world distribution network operations, dynamic fluctuations and complex noisy environments cause process and measurement noise to deviate from the fixed assumption of Gaussian distributions. This severely compromises estimation precision. To address this, we extend the adaptive UKF by proposing a process noise adaptive estimation method based on an AMF and a measurement noise adaptive estimation method based on RMD. These methods effectively suppress the influence of non-ideal process and measurement noise, leading to a considerable improvement in the robustness and precision of dynamic state estimation.

4.3.1. Process Noise Adaptive Estimation Based on the AMF

Traditional Sage–Husa noise estimators typically employ an exponential decay form to weight historical correction information. The central principle involves allocating greater significance to the most recent data to adapt to noise changes, i.e., replacing the traditional equal weighting coefficient in Equation (12) [20] with the time-decaying exponential weighting one in Equation (13).

$${\widehat{Q}}_{k+1}={\displaystyle \frac{1}{k+1}}\sum _{j=1}^{k+1}[K_j{\epsilon }_j{\epsilon }_j^{\mathrm{T}}{K}_j^{\mathrm{T}}+{\widehat{P}}_j-\sum _{i=0}^{2n}{W}_c^{(i)}({\mathcal{X}}_{j|j-1}^i-{\tilde{x}}_j){({\mathcal{X}}_{j|j-1}^i-{\tilde{x}}_j)}^T]$$

(12)

$$\left\{\begin{array}{c}{\widehat{Q}}_{k+1}={\displaystyle \sum _{j=1}^{k+1}}{d}_j\left[K_j{\epsilon }_j{\epsilon }_j^T{K}_j^T+{\widehat{P}}_j-{\displaystyle \sum _{i=0}^{2n}}{W}_c^{(i)}({\mathcal{X}}_{j|j-1}^{(i)}-{\tilde{x}}_j){({\mathcal{X}}_{j|j-1}^{(i)}-{\tilde{x}}_j)}^T\right]\\ d_j={\displaystyle \frac{1-b}{1-b^{k+1}}}{b}^{k+1-j}\end{array}\right.$$

(13)

where d_i represents the weighting coefficient for the i-th time step, satisfying Σd_i = 1; b is the modulation factor.

However, when system states undergo severe fluctuations or abrupt changes, e.g., sudden changes in distributed power output, the statistical profile of the process noise can instantaneously shift. In such scenarios, traditional exponential decay estimators, due to their fixed modulation factor, cannot respond quickly to these abrupt changes, which leads to a noticeable estimation lag. This, in turn, prevents the model from effectively absorbing new uncertain information and increases the bias of the state estimation results.

Therefore, to solve this issue, we propose a process noise adaptive estimation method based on an AMF. The central principle of this technique involves the dynamic tuning of the modulation factor’s magnitude by continually tracking the amplitude of the process noise covariance change, thereby achieving an adaptive allocation of weights to new and old data. Specifically, when the amplitude of the noise covariance change is large, it indicates that the system state is undergoing severe dynamic changes. In this case, a smaller modulation factor is used to give higher weights to new data. Conversely, when the amplitude of the noise covariance change is small, the system state is relatively stable. A larger modulation factor is used to increase the importance of historical data, ensuring the smoothness and robustness of the estimation.

The change in the statistical properties of the process uncertainty is expressed as

$$B_k={\widehat{Q}}_k-{\widehat{Q}}_{k-1}$$

(14)

The maximum values among $B_k$ are defined as follows, respectively:

$$B_{k \max }=\max (\mathrm{diag}(B_k))$$

(15)

where diag(·) denotes the function that extracts the diagonal elements.

The AMF is determined as follows:

$$b_{k+1}^{\prime }=\left\{\begin{array}{c}{\displaystyle \frac{1}{k+1}} B_{k\max }>\gamma \\ {\displaystyle \frac{k}{k+1}} B_{k\max }\le \gamma \end{array}\right.$$

(16)

where γ is the threshold for the magnitude of process noise change.

The threshold γ is designed to distinguish between normal estimation deviations and structural state mutations. Theoretically, this threshold is determined based on the statistical significance level of the estimation error covariance, which is usually empirically set to be slightly larger than Q observed under steady-state conditions, serving as a safety margin. In this case study, given the steady-state fluctuation level, γ is set to 5 × 10⁻⁴.

According to Equation (13), a new iterative estimation formula for Q based on the AMF is given as follows:

$$\left\{\begin{array}{c}{\widehat{Q}}_{k+1}=(1-d_{k+1}){\widehat{Q}}_k+d_{k+1}[K_{k+1}{\epsilon }_{k+1}{{\epsilon }_{k+1}}^T{K}_{k+1}^T+{\widehat{P}}_{k+1}-{\displaystyle \sum _{i=0}^{2n}}{W}_c^{(i)}({\chi }_{k+1|k}^i-{\tilde{x}}_{k+1}){({\chi }_{k+1|k}^i-{\tilde{x}}_{k+1})}^T]\\ d_{k+1}={\displaystyle \frac{1-b_{k+1}^{\prime }}{1-{(b_{k+1}^{\prime })}^{k+2}}}\end{array}\right.$$

(17)

Remark 1.

It is theoretically recognized that the recursive update of ${\widehat{Q}}_{k+1}$ in Equation (17) implicitly involves a subtraction operation, specifically the term ${\widehat{P}}_{k+1}-{\displaystyle \sum _{i=0}^{2n}}{W}_c^{(i)}({\chi }_{k+1|k}^i-{\tilde{x}}_{k+1}){({\chi }_{k+1|k}^i-{\tilde{x}}_{k+1})}^T$. In scenarios where the estimated error covariance decreases significantly, this subtraction may dominate, rendering the updated ${\widehat{Q}}_{k+1}$ non-positive semi-definite (PSD). It is recommended to adopt a fault-tolerant estimator [31]. Specifically, if the result of Equation (17) fails the PSD check, the algorithm might switch to the following formulation:

$${\widehat{Q}}_{k+1}=(1-d_{k+1}){\widehat{Q}}_k+d_{k+1}[\mathrm{diag}(K_{k+1}{\epsilon }_{k+1}{{\epsilon }_{k+1}}^T{K}_{k+1}^T)+K_{k+1}{P}_{zz,k+1}{K}_{k+1}^T]$$

(18)

The modified estimator guarantees the positive semi-definiteness of the result by constructing the update using only additive semi-definite terms.

4.3.2. Measurement Noise Adaptive Estimation Based on RMD

The proposed measurement noise adaptive estimation algorithm, derived from RMD, has the following steps:

(1) At each time slot k, compute the innovation vector ${\epsilon }_k$;

(2) A sliding window is constructed consisting of the m most recent innovation vectors: $\left\{{\epsilon }_{k-m+1},{\epsilon }_{k-m+2},\dots ,{\epsilon }_k\right\}$, where m is the length of the sliding window, m = 10;

(3) For each innovation vector ${\epsilon }_i$ within the sliding window, the Mahalanobis distance (MD) is calculated by

$$D_M^i=\sqrt{{({\epsilon }_i-\overline{\epsilon })}^T{C}^{-1}({\epsilon }_i-\overline{\epsilon })}$$

(19)

where $\overline{\epsilon }={\displaystyle \frac{1}{m}}{\displaystyle \sum _{i=1}^m{\epsilon }_i}$, $C={\displaystyle \frac{1}{m-1}}{\displaystyle \sum _{i=1}^m({\epsilon }_i-\overline{\epsilon }){({\epsilon }_i-\overline{\epsilon })}^T}$.

The traditional MD represents the surface of an ellipsoid centered at the sample mean, with its size and orientation determined by the covariance matrix. For a Gaussian-distributed random variable, the squared MD value conforms to a ${\chi }^2$ distribution. However, the traditional MD calculation relies on the sample mean and covariance matrix, and these statistical measures are extremely sensitive to non-Gaussian outliers, which can lead to a biased MD estimate. Therefore, this paper adopts the RMD to replace the traditional sample mean and covariance matrix. Specifically, the median of the innovation vectors, denoted as M, is used to replace the sample mean $\overline{\epsilon }$, and the diagonal matrix constructed from the squared median absolute deviation (MAD) substitutes for the sample covariance C.

The RMD $D_{M,robust}^i$ is calculated using the following formulas:

$$M=\mathrm{median}({\epsilon }_{k-m+1},{\epsilon }_{k-m+2},\dots ,{\epsilon }_k)$$

(20)

$$MAD=\mathrm{median}(\left|{\epsilon }_{k-m+1}-M\right|,\left|{\epsilon }_{k-m+2}-M\right|,\dots ,\left|{\epsilon }_k-M\right|)$$

(21)

$$\mathsf{\Omega }=\mathrm{diag}({(2.0469\times MAD)}^2)$$

(22)

$$D_{M,robust}^i=\sqrt{{({\epsilon }_i-M)}^T{\mathsf{\Omega }}^{-1}({\epsilon }_i-M)}$$

(23)

where median(·) denotes the component-wise median function, which calculates the median value for each dimension of the input vectors independently. Ω denotes the robust innovation covariance matrix. The coefficient 2.0469 in Equation (22) is used to ensure unbiasedness under a Laplace distribution.

(4) Assign weights ω_i based on RMD:

$${\omega }_i=\min \left[1,{\displaystyle \frac{{\xi }_2}{{(D_{M,robust}^i)}^2}}\right]$$

(24)

where ${\xi }_2$ is the detection threshold, which is set to the 95% quantile of the ${\chi }_{N_z}^2$ distribution; N_z represents the dimension of the measurement vector.

(5) Calculate the robust sample covariance C_robust:

$$C_{robust}={\displaystyle \frac{1}{{\displaystyle {\sum }_{i=1}^m{\omega }_i}}}{\displaystyle \sum _{i=1}^m{\omega }_i({\epsilon }_i-M){({\epsilon }_i-M)}^T}$$

(25)

(6) Finally, the corrected ${\widehat{R}}_k$ is obtained by

$${\widehat{R}}_k=C_{robust}-H{\widehat{P}}_k{H}^T$$

(26)

where H denotes the measurement matrix.

Remark 2.

Considering that ${\widehat{R}}_k$ calculated in Equation (26) may yield non-positive variances due to sample bias or model mismatch, and a simple diagonalization might neglect the statistical correlation among measurement data, it is recommended to apply a projection-based regularization step:

$${\widehat{R}}_k=U·\mathrm{diag}(\max ({\delta }_i,ϵ))·U^T$$

(27)

where δ_i and U denote the i-th eigenvalue and the corresponding eigenvector matrix of ${\widehat{R}}_k$, respectively. $ϵ$ is a small positive regularization term.

5. Case Study

To establish the efficacy and superior performance of our proposed approach, simulation tests were performed in the MATLAB R2023a setting on the IEEE 33-bus three-phase unbalanced grid, the configuration of which is presented in Figure 3. In this case study, the system is powered by thermal power from the main grid and distributed PVs, with a renewable energy penetration rate of approximately 10.94%.

To replicate the actual operational conditions of the power system, three types of typical load are modeled: residential, industrial, and commercial. Each type of load has distinct daily fluctuation curves and characteristics. Based on these realistic load profiles, a dataset comprising 2000 temporal snapshots was constructed via time-series power flow analysis. To strictly evaluate the model’s generalization capability, the dataset was partitioned into a training set (80%) and a testing set (20%). The true power flow values are obtained by running the power flow solver. Measurement values are generated by adding zero-mean Gaussian measurement errors to these true values. Specifically, the standard deviation of measurement errors for Micro-PMUs is set to 0.3% for voltage magnitudes, with 0.05° designated for voltage phase angles. For current magnitudes and phase angles, the standard deviation is set to 0.4% and 0.05°, respectively. The power measurement error standard deviation for the SCADA system is fixed uniformly at 1%. For pseudo-measurements generated by the proposed model, the error standard deviation is set to 10%. The time interval for dynamic state estimation is uniformly set at 5 s, which is consistent with the SCADA system’s reporting rate. Furthermore, all initial and empirical parameters used in the algorithm were determined through repeated simulations and tuning. The initial process and measurement noise covariance matrices, Q₀ and R₀, are initialized as a diagonal matrix with diagonal elements set to 10⁻⁴.

The architectural specifications for the proposed CNN-BiGRU-Attention model are detailed below. The CNN component comprises two one-dimensional convolutional layers and two corresponding max pooling layers. Specifically, the initial convolutional layer utilizes 32 filters of size 5, and the subsequent layer doubles the filter count to 64 with the same kernel size. The BiGRU layer is configured with 256 hidden units. Furthermore, the Attention component leverages eight parallel attention heads, and the projection dimension is established as 512. Concerning the training regimen, we fixed the initial learning rate at 0.03%, used a mini-batch size of 32, and performed the training over 300 epochs.

5.1. Evaluation of Pseudo-Measurement Generation Method

To assess the efficacy and advantages of the proposed CNN-BiGRU-Attention pseudo-measurement generation method, we generated pseudo-measurement data for the nodes and branches lacking measurement devices. We compared our method with five other models: Model-1 (CNN), Model-2 (LSTM), Model-3 (GRU), Model-4 (BiGRU), and Model-5 (BiGRU-Attention). Our method is designated as Model-6. Pseudo-measurement data between 11:30 and 11:40 are generated and compared with the true values. The results of the injected power at node 9, phase B and the branch power on branch 16–17, phase B are shown in Figure 4. The performance of each model on the same test set was evaluated using the following metrics: mean absolute percentage error (MAPE), root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination (R2). The results are summarized in

Table 1. Performance comparison of different prediction models.

Model	MAPE/(%)	RMSE/(MW)	MAE/(MW)	R2
Model-1	23.4008	3.2294	0.01762	0.9959
Model-2	33.1361	5.4913	0.03276	0.9774
Model-3	18.3087	2.9429	0.01512	0.9968
Model-4	16.8753	2.8603	0.01479	0.9954
Model-5	16.2015	2.6814	0.01433	0.9973
Model-6	15.8809	2.5612	0.01303	0.9977

.

From Figure 4, it is evident that Model-2 fails to accurately predict data with significant fluctuations, resulting in a low goodness-of-fit. Compared to Model-3, Model-4’s MAPE is only 16.8753% of the former, demonstrating that a bidirectional GRU is a reasonable choice for extracting temporal dependencies. Both Figure 4 and Table 1 clearly show that Model-6, the proposed pseudo-measurement generation model, delivers superior predictive performance compared to all competing models. The results closely align with the true values, demonstrating its superior performance.

5.2. Basic Test of the NA-UKF Algorithm

To assess the efficacy of the proposed spatiotemporal pseudo-measurement generator and the NA-UKF algorithm, five comparative methods are designed. It is important to note that the proposed NA-UKF is built upon the architecture of the robust adaptive unscented Kalman filter (RAUKF) [27], with specific improvements made to the estimation mechanisms for process and measurement noise covariance to enhance robustness. To ensure a fair comparison, all common parameters among the UKF, adaptive extended Kalman filter (AEKF), RAUKF, and NA-UKF algorithms are set to identical values.

M1: Employ the proposed pseudo-measurement model with the standard UKF algorithm [27].

M2: Employ the proposed pseudo-measurement model with the AEKF algorithm [28].

M3: Employ the proposed pseudo-measurement model with the RAUKF algorithm [29].

M4: Employ a proportional method based on typical load curves for pseudo-measurement generation with the proposed NA-UKF algorithm.

M5: Employ the proposed pseudo-measurement model with the proposed NA-UKF algorithm.

Figure 5 and Figure 6 illustrate the state estimation results and MAE metrics for these five methods. From Figure 5 and Figure 6, it is evident that the adaptive algorithms (AEKF, RAUKF and NA-UKF) generally perform well. The unscented transformation-based adaptive methods (M3 and M5) demonstrate superior overall tracking accuracy compared to both M1 and M2. To quantitatively analyze the estimation precision,

Table 2. Comparison of evaluation metrics for the five methods.

Method	MAE		RMSE		MVE/(10−3)
	Magnitude (10−3/p.u.)	Phase Angle (10−3/°)	Magnitude (10−3/p.u.)	Phase Angle (10−3/°)
M1	2.1358	0.6822	2.7060	0.8681	2.2840
M2	1.4854	0.7285	1.8847	1.0264	1.7344
M3	1.3196	0.4706	1.6367	0.6180	1.4325
M4	1.4979	0.4221	1.8714	0.5265	1.5710
M5	0.9187	0.3809	1.1518	0.4810	1.0273

presents their evaluation metrics. The mean vector error (MVE) is introduced to comprehensively evaluate the estimation deviation of both voltage magnitude and phase angle for phase A. Most importantly, the proposed M5 yields the lowest errors across all metrics, confirming the effectiveness of the proposed noise adaptive mechanism.

Furthermore, M5 exhibits better tracking performance than M4. This is because the CNN-BiGRU-Attention model effectively leverages spatiotemporal correlation to generate pseudo-measurement data of superior precision.

5.3. Test of Process Noise Adaptive Capability

This subsection assesses the adaptive estimation capabilities of our developed algorithm when facing time-varying statistical profiles in process uncertainty. To simulate the sudden mutation of process noise, the system is initially set to operate in a quasi-steady state. However, during the time interval 11:48–12:12, additional Gaussian process noise is injected into the true state values to mimic severe system fluctuations. Specifically, the standard deviations of the injected noise are set to 0.005 p.u. for voltage magnitude and 0.0015 rad for voltage phase angle. Four different algorithms, i.e., UKF, AEKF, RAUKF, and NA-UKF, are used for state estimation. Figure 7 displays the state estimation results.

Table 3. Comparison of evaluation metrics for the four algorithms under process noise mutation.

Algorithm	MAE		RMSE		MVE/(10−3)
	Magnitude (10−3/p.u.)	Phase Angle (10−3/°)	Magnitude (10−3/p.u.)	Phase Angle (10−3/°)
UKF	3.2182	0.9631	4.5582	1.3373	3.4084
AEKF	2.5325	0.8349	4.1187	1.3137	2.7201
RAUKF	2.3170	0.7591	3.9867	1.2240	2.4860
NA-UKF	1.4387	0.3603	1.8120	0.4605	1.4884

presents the overall evaluation metrics for the four state estimation algorithms.

From Figure 7 and Table 3, it is evident that under severe changes in process noise, the standard UKF fails to track the dynamic changes effectively due to its fixed process noise covariance, resulting in the highest estimation errors. While the AEKF and RAUKF algorithms provide limited improvements, the proposed NA-UKF algorithm shows the most accurate estimation results. Compared to the RAUKF algorithm, the NA-UKF significantly reduces the RMSE error. Specifically, the RMSE for magnitude is reduced by 54.55%, and for the phase angle, it is reduced by 62.38%.

5.4. Test of Measurement Noise Adaptive Capability

To evaluate the algorithm’s robustness against non-Gaussian measurement outliers, a Laplace distribution is selected as a representative of non-Gaussian noise. The noise under this distribution exhibits “peaky” and “heavy-tailed” characteristics, which are significantly different from the Gaussian distribution. To ensure a rigorous comparison, the standard deviations for each measurement type are set identically to those used in the Gaussian scenarios. Figure 8 displays the state estimation results.

Table 4. Comparison of evaluation metrics for the four algorithms under Laplace measurement noise.

Algorithm	MAE		RMSE		MVE/(10−3)
	Magnitude (10−3/p.u.)	Phase Angle (10−3/°)	Magnitude (10−3/p.u.)	Phase Angle (10−3/°)
UKF	2.0300	0.6590	2.7718	0.8514	2.1892
AEKF	1.2512	0.4467	1.6115	0.5884	1.3690
RAUKF	0.9029	0.3722	1.1457	0.4706	1.0097
NA-UKF	0.3687	0.3144	0.4733	0.4034	0.5231

presents the evaluation metrics for the four state estimation algorithms.

From Figure 8 and Table 4, it is clear that under Laplace measurement noise, the proposed NA-UKF algorithm achieves the highest estimation precision. Relative to the RAUKF method, the NA-UKF significantly reduces the RMSE error. For voltage magnitude, the RMSE is reduced by 58.69%. For the phase angle, it is reduced by 14.28%.

5.5. Test of Load and Renewable Source Mutation

To evaluate the algorithm’s estimation performance under load and renewable source mutation scenarios, a specific simulation case is designed; specifically, the system operates normally until 11:55. At this timestamp, the active and reactive loads at nodes 9 and 25, as well as the charging loads at EV charging stations at nodes 19 and 23, are instantly increased by 50%. Simultaneously, the output of PVs at nodes 18 and 33 is cut to zero to simulate a sudden generation loss. These mutation conditions persist until 12:00, after which all loads and generation outputs are restored to their normal operational levels. Figure 9 and

Table 5. Comparison of evaluation metrics for the four algorithms under load and renewable source mutation.

Algorithm	MAE		RMSE		MVE/(10−3)
	Magnitude (10−3/p.u.)	Phase Angle (10−3/°)	Magnitude (10−3/p.u.)	Phase Angle (10−3/°)
UKF	2.0431	0.6752	2.5672	0.8565	2.1949
AEKF	1.3890	0.4111	1.7644	0.5204	1.4739
RAUKF	1.2497	0.4635	1.5701	0.5993	1.3695
NA-UKF	0.9117	0.3822	1.1489	0.4825	1.0223

display the state estimation results and the comparative evaluation metrics for the four algorithms.

From Figure 9 and Table 5, it is evident that under load and renewable source mutation, the proposed NA-UKF algorithm demonstrates the most robust tracking capability, yielding the lowest errors across all metrics. Relative to the RAUKF method, the NA-UKF significantly improves the estimation accuracy. Specifically, the RMSE for voltage magnitude is reduced by approximately 26.83%, and the MVE is reduced by 25.35%.

5.6. Test of Robustness Against Bad Data

To validate the proposed algorithm’s robustness against bad data, a specific test involving a random bad data injection is conducted. Specifically, bad data are randomly added to the voltage measurements, with deviation magnitudes set to exceed 3% for voltage magnitude and 3° for voltage phase angle, simulating severe measurement outliers. Figure 10 and

Table 6. Comparison of evaluation metrics for the four algorithms under bad data conditions.

Algorithm	MAE		RMSE		MVE/(10−3)
	Magnitude (10−3/p.u.)	Phase Angle (10−3/°)	Magnitude (10−3/p.u.)	Phase Angle (10−3/°)
UKF	2.1867	1.0365	3.0627	3.4360	2.6097
AEKF	1.5085	0.7043	2.2552	2.6666	1.8255
RAUKF	1.3119	0.5715	1.7940	2.3373	1.4843
NA-UKF	0.9646	0.4901	1.2964	1.1043	1.1933

display the state estimation results and the comparative evaluation metrics for the four algorithms under bad data conditions.

From Figure 10 and Table 6, it is clearly observable that the proposed NA-UKF demonstrates strong robustness. Relative to the RAUKF method, the NA-UKF significantly improves estimation precision. Specifically, the RMSE for voltage magnitude is reduced by approximately 27.73%, the RMSE for phase angle is reduced by 52.75% and the MVE is reduced by 19.60%.

6. Conclusions

To address the challenges of low measurement redundancy and complex noise environments in renewable-dominated distribution networks, this paper proposes a dynamic state estimation method incorporating spatiotemporal data correlation and noise adaptiveness. The results demonstrate that the inherent spatiotemporal correlations in distribution grids can be effectively utilized to reconstruct missing measurement data. The proposed CNN-BiGRU-Attention model achieves accurate mapping from sparse real-time measurements to unmonitored nodes, significantly enhancing network observability. Furthermore, this study validates that adaptive tuning of noise statistics is essential for maintaining estimation accuracy under complex operating conditions. By integrating the NA-UKF algorithm, the proposed method mitigates the performance degradation often observed in traditional filters constrained by static noise covariance assumptions. Specifically, the AMF mechanism enables the estimator to rapidly track severe system fluctuations by dynamically adjusting the process noise covariance, while the RMD-based strategy effectively suppresses the impact of non-Gaussian measurement outliers. Numerical simulations on the IEEE 33-bus system confirm that this approach exhibits superior robustness and tracking precision compared to existing methods.

Future work will focus on two key directions to further advance the scalability and precision of dynamic state estimation. First, to address the challenges inherent in large-scale distribution networks, we aim to explore the deployment of the NA-UKF algorithm onto edge computing nodes. As grid complexity grows, transmitting vast quantities of raw measurement data to the cloud for centralized processing imposes significant bandwidth pressure and introduces latency. By enabling distributed, localized data processing, we aim to alleviate the computational and communication burden on the core network, realizing a more responsive monitoring architecture. Second, while the proposed CNN-based method effectively balances efficiency and accuracy, we acknowledge the theoretical advantages of graph neural networks (GNNs) in explicit topology modeling. Future research will investigate advanced GNN architectures and develop lightweight variants to overcome their computational bottlenecks and sensitivity to data quality, thereby further enhancing the extraction of spatial correlations in complex mesh networks.