1. Introduction
Cyber-physical systems (CPS) aim at improving the system safety, security, sustainability and efficiency in large-scale in-network applications, such as smart grids, medical and health, industrial manufacturing, traffic control, smart buildings, etc. [
1]. Such applications require sensing and information analysis in a wide area network and claim higher safety and quality of the measurement network. Beyond the traditional sensing network solution, cyber-physical systems combine the communication, computation and control process and offer a better performance [
2]. On the demand side of the smart grid, the cyber-physical energy system (CPES) is dedicated as a special case of CPS dealing with the electrical safety and quality problems among large-scale industrial and commercial power utilization [
3]. To provide the consumers with a secure and trustworthy power supply, the distributed metering architecture is exploited for large-scale, hybrid network measurement [
4].
On the demand side, the automated meter reading (AMR) system, which behaves only as one-way manual centralized detection, is at the last gasp of its use corrections , while the advanced metering infrastructure (AMI) with a distributed architecture and integrated electrical information analysis service is gradually taking its place [
5]. For the prospect of the electrical metering system, the smart grid is a promising solution for a less centralized and more consumer-interactive network. Within this measuring framework, electrical characteristics are collected through scattered smart meters, then meter data management systems dispose integrated computing and make deploying strategies accordingly to ensure high quality power. Harmonic measurement is an essential part of power quality assessment. Harmonics on the demand side mainly come from nonlinear appliances. Distorted harmonics introduce fluctuations, sags and disequilibrium into the grid, leading to potential damages and power failures. Fast and online harmonic analysis is significant in power grids and facing new challenges under distributed metering architectures.
The out-of-sequence measurement (OOSM) problem is one of the challenges in large-scale distributed sensor networks [
6]. The AMI architecture contains thousands of electrical measuring devices. Electrical data from different sensing nodes reaches the data management center with latencies that result in temporal disorders of the data sequence, as shown in
Figure 1. Data fusion, such as harmonic analysis and identification, will be influenced by disordered measurement. In target tracking sensor networks, OOSM methods have been developed and verified. OOSM has been implemented by the Kalman filter and the particle filter. Bar-Shalom [
7] introduced the OOSM in multisensor systems and studied data retrodiction based on statistics theory. Bar-Shalom proposed an optimal method, A1, and suboptimal methods, B1 and C1 [
8], based on the Kalman filter for one-lag disorder, then developed multi-lag OOSM Methods Al1 [
9], Bl1 [
10] and NBl [
11]. Lanzkron proposed a two-step multi-lag OOSM method [
12]. Subhash proposed an augmented state Kalman filter with a Bayesian solution [
13], and Keshu improved the performance of this algorithm [
14]. OOSM algorithms based on the particle filter were developed following the Kalman filter-based methods. Mallick proposed suboptimal B-PF2 based on the heuristics that the target states at different time instants are independent given the past measurements in [
15]. B-PF1 was depicted in [
16] by Matthew, and also, A-PF is proposed in [
17,
18]. Gustafsson presented a storage-efficient particle filter algorithm (SEPF) in [
19,
20]. Anders promoted an RBPF (Rao–Blackwellized particle filter) method in [
21]. Decentralized methods for OOSM are presented in [
22,
23]. Handel [
24] exploited the Bayesian framework to minimize the error estimation of the disordered data. Zhang proposed a complete in-sequence information (CISI) method in [
25]. In the applications, Besada-Portas studied the out-of-sequence problem in mobile robot localization [
26]. Klaus [
6] applied OOSM for multi-sensor fusion in vehicles and proposed joint integrated probabilistic data association (JIPDA) fusion. Schueller [
27] provided a temporal solution to calibrate the data sequence in an advanced driver assistance system. Liu [
28] combined the OOSM algorithm with compressive sensing for harmonic measurement and identification.
Harmonic analysis requires harmonic data in the same temporal section. Out-of-sequence data will reduce the accuracy of harmonic monitoring and power quality assessment. Existing OOSM methods based on the Kalman filter or the particle filter are computationally complicated and rely on dynamic models of the tracking sources. For electrical harmonics, the dynamic model is irregular, and it is difficult to depict the precise relations of the harmonics. In addition, network-based monitoring indicates relative correlations between the signals in multiple channels. The Kalman filter- and the particle filter-based OOSM methods do not consider the network data as a whole, which is incomplete utilization of the harmonic information. The artificial neural network gains the capability to imitate a complex nonlinear system without a priori modeling, which is suitable for implementing measuring data reordering in a large-scale distributed network [
29,
30]. The nonlinear autoregressive model with exogenous inputs (NARX) model [
31,
32,
33,
34,
35] has long been adopted in the prediction and filtering of temporal data sequences. In this paper, NARX-based OOSM methods are adopted to perform data-aware retrodiction for distributed harmonic analysis.
The main contribution of this paper lies in introducing the artificial neural network into out-of-sequence measurement in harmonic analysis. The NARX neural network is employed to predict and retrodict the disordered data without the kinematic model of the harmonic behavior. The data-aware NARX network maintains the model in the hidden layers and provides a precise approximation of the real electrical harmonic changes. The theoretical analysis on the NARX-based OOSM method’s computational complexity and memory consumption is presented. Experiments on a practical distributed electrical sensing network evaluate the performance of the proposed method.
The rest of this paper is organized as follows.
Section 2 contains the OOSM problem statement and basic notations.
Section 3 depicts the OOSM method and proposes the NARX-based OOSM algorithm. This section also analyzes the NARX-based algorithm’s computational complexity and compares it to the former OOSM methods.
Section 4 shows the experiment results on the harmonic identification accuracy of disordered measuring data, demonstrating the validation of the measurement method proposed. Finally,
Section 5 is the conclusion and the overview of future work.
4. Experiments and Analysis
4.1. Experiment Setup
To evaluate the performance of the OOSM method on the harmonic analysis, a testbed to approximate the cyber-physical energy system in the residential power consumption network is built in this section. A two-way distributed metering framework for electrical information monitoring is deployed. The distributed network sensing architecture is depicted in
Figure 2. This network measures and transmits the electrical parameters by multichannels asynchronously. The reconstruction of the signal and harmonic analysis algorithm is realized on the data management center. The power supply is 220 V/50 Hz. The appliances contain linear loads, such as lamp humidities that do not produce harmonics, and nonlinear loads, like rotor rigs, microwaves, air-conditioners, etc., which are demand-side harmonic sources. Nonlinear loads are controlled during the experiment. Switch actions will be detected in harmonic sensing. The Intelligent sensing nodes use ADE7754 to measure electrical parameters with a sampling frequency up to 14 kHz. Signal processing methods are realized on ARM. The electrical characteristics, including voltages, currents, frequency, power, harmonic amplitudes, etc., are reported to the data center. The chip CC2430 realizes data transmission through IEEE 802.15.4 protocols. The data packages follow the protocol standard DL/T 645-2007. The sensing network contains 100 sensing nodes, including 84 end devices, 13 routers and 3 coordinators. An end device calculates and uploads the electrical parameters every second. Then, the router fuses and relays the electrical data measured by end devices to the coordinators. All end devices and routers have the same hardware structure. The coordinators receive the data from the router and send them to the meter data management system. The data management center executes the retrodiction algorithm to correct the electrical data and analyzes the electric information with Intel i5 CPU and 4 G memory. The main structure of the testbed is shown in
Figure 4. The harmonic measurement results are compared to the standard Fluke 435. Within the distributed sensing network, the out-of-sequence problem can be observed.
Considering the measurement requirement of different electrical characteristics, harmonic measurement is selected to verify the proposed retrodiction method. In the following, the computation complexity, memory consumption and influence on harmonic measurement applications of the NARX-based OOSM algorithm is compared to other OOSM methods. Time-varying harmonic and transient harmonic measurements are tested. During the measurement process, the Kalman filter and the particle filter need kinematic equations to calculate the estimation of the harmonic. The training method of the NARX-based OOSM method is the same as in
Section 3.2, and the training dataset is explained in each case. In these experiments, the Kalman filter and the particle filter use the dynamics described in Equations (
8) and (
9) [
49,
50]. Apparently, the kinematic model is linear; the Monte Carlo method in the particle filter can approximate the nonlinear feature of the harmonics.
4.2. Measurement Precision Comparison
The out-of-sequence measurement is implemented on the centralized data management center. To examine the OOSM algorithm’s effectiveness, a simulation experiment is designed with data under various lag levels. Training datasets for the NARX neural network contain 70% of 4-h electrical data of 100 channels, and testing datasets contain another 30% of the electrical data. Based on electrical parameters measured through CPES, multi-lag data sequences can be inserted into the time sequence under a certain frequency through sorting data time sequences. The inserted lags follow a Poisson distribution
. Inserted lags grow from one lag to a maximum of nine lags. The NARX network is trained and tested under various multi-lag levels, which are supposed to turn the disordered data into correct time series. Training and testing datasets from different channels and lag-levels are coupled to examine if the NARX filter trained in a specific dataset is suitable for other lag level data. Average mean square errors (MSE) of the modified data from the original data are recorded to depict the performance of the NARX filter. The results are depicted in
Figure 5.
MSEs for the disordered data in the training datasets are between 0.49 and 2.31 under various lag levels. From
Figure 5, it can be seen that the MSE reduces between 0.184 and 0.345 after the NARX filter, and the least MSE is reached with 1-lag train datasets and 1-lag disordered data. With the disorder growing, disorder errors display an ascending trend. When the network is trained with multi-lag data, MSE rises, indicating that multi-lag disordered data vary from lag numbers, which is not compatible with the lower lag number situation. The optimal circumstance is the low lag number electrical data series, and the NARX neural network is also trained with the same lag data type. However, in the practical situation, the distortion occurs following an exponential behavior as the scale of the network grows [
27]. The main lag number remains 2–3 lags. Thus, the training dataset for the NARX-based OOSM method should be the same practical electrical environment. The flexibility of this method among various electrical situations should be examined in future work.
Following the harmonic measurement requirements in IEC 61000-4-7, the precision of the base voltage and current is ≤5% of the true values, and the harmonic measurement error limits are:
for the 3rd harmonic,
for the 5th harmonic,
for the 7th harmonic and
for the 9th harmonic and even harmonics. The NARX-based OOSM method is implemented for harmonic measurements and comparing the MSE with other OOSM methods, including Bl, ALG-I and SEPF. The measurement network scale is 100 nodes, and the average MSE of the harmonic measurement for each OOSM algorithm is shown in
Figure 6. The errors of the base current and odd harmonics from the 3rd–9th are tested. The dotted line indicates the requirements for each harmonic order in IEC 61000-4-7-2002. Apparently, without retrodiction, harmonic measurement does not fulfill the standard in a disorder manner. Bl and ALG-I, as suboptimal methods, do not offer a sufficient improvement in precision. SEPF and NARX-based OOSM reach the standard, and the data-aware method maintains a better performance in all orders. The computational complexity of these algorithms is tested in the following.
4.3. Computational Complexity Comparison
The computational complexity and memory cost of NARX-based OOSM methods are compared to other algorithms. To evaluate the memory and computational performance of the NARX-based method, Kalman filter-based methods, Bl and ALG-I, and the particle filter-based method, SEPF, are selected to compare. The 4 algorithms are tested under the datasets of electrical harmonics. The out-of-sequence harmonic data are measured on the electrical network. The measuring length
N is 256; the frequency resolution
Hz
Hz, which meets the requirements in IEC 61000-4-7-2002. The largest disorder of the harmonic sequence is 7. The calculation is implemented on MATLAB 2013b. The harmonic is sorted with the time stamps to be the correct target. The OOSM method is regarded as valid with respect to the MSE of the reordered data. The results of the computational complexity according to the delay length are shown in
Figure 7, and those according to the number of states are shown in
Figure 8.
From the results, it can be figured out that ALG-I performs the largest number of float calculations. As a suboptimal method, although Bl fails to offer a sufficient harmonic measuring precision, it provides the least computational consumption among the algorithms. NARX-based OOSM behaves more stably than the particle filter-based method SEPF. The NARX-OOSM does not increase the computational capacity on different delays, but increases as the input data grow. However, among the 4 algorithms, NARX managed to maintain a relatively low computation consumption, and the practical results comply with the theoretical analysis from above. Considering the harmonic measuring precision, the proposed data-aware methods are promising among the 4 algorithms in the multichannel harmonic measurement. Harmonic identification analysis depending on time series will be tested in the following section.
4.4. Case I: Data-Aware Retrodiction for Non-Stationary Harmonic Measurement
Harmonic measurement is examined to test the effects of various OOSM algorithms on harmonic analysis. Harmonic measurement is important in harmonic analysis and widely applied in safety monitoring and assessment of the smart grid. Harmonic identification relies on the analysis of a time sequence of multi-channel electrical data and is affected by out-of-sequence measured data. In this section, the non-stationary harmonics are injected in the grid to test the OOSM algorithms. The harmonic sources are inverters in speed-varying rotor machines. The rotation speed ranges of these rotor machines are 0–3000 rpm. The rotor standard power is kW at 3000 rpm, and the output power changes over time as the speed changes. These machines are controlled independently and inject unstable harmonics into the grid.
The harmonic series are measured under the distributed power utilization sensing network. The harmonic measurement algorithm deployed at the end nodes is adaptive linear neuron or, later, adaptive linear element (ADALINE). Data disorders do not exceed 4 lags. Electrical parameters in 100 channels are collected, and 3 channels of the original electrical harmonic current measurement and the harmonic sources are shown in
Figure 9. The 3 channels link to the same point of common coupling (PCC), and the harmonic currents of each nonlinear load are tangled with each other in the measurement data. Harmonic measurement results are compared to the sensing results of Fluke 435. Different orders of harmonic currents examine the effects of the OOSM algorithm on harmonic identification. The OOSM improvement in decreasing measurement error is shown in
Figure 10, and the MSEs of 100 channels are listed in
Table 3.
The 3rd–9th odd harmonics are used for harmonic measurement verification, which are capable of representing the harmonic measurement precision of the OOSM process. The results indicate that the NARX neural network can reduce the error introduced by the out-of-sequence measurement. It can also be figured that the OOSM algorithm alone does not convey a productive decrease on the out-of-sequence measurement error. The MSE of the measurements are reduced by more than .
The retrodiction accuracy of 4 algorithms shown in
Table 3 indicates that the NARX-based OOSM method gains a better harmonic measurement result than the other 3 methods, although the deviations are minor. For the 7th and 9th harmonic, the OOSM methods do not make a great difference in accuracy among each other. This indicates that as the magnitudes of the harmonics become small, the effects of the modification are not manifest. The data-aware method’s improvement on non-stationary harmonic measurement is verified.
4.5. Case II: Data-Aware Retrodiction for Transient Harmonic Measurement
Transient harmonic measurement is examined in this section to test the effects of various OOSM algorithms on harmonic analysis. The transient harmonics are mainly introduced by switching on or off large nonlinear loads, injecting a large current shock. The transient harmonic source is realized by switching on and off a light wave oven, which injects a 4 A current pulse and lasts for 136 ms. The current injection can be detected from the 3rd harmonic to the 9th harmonic in the channel. Out-of-sequence performance occurs at the time point of the current pulse, and the retrodiction performance is examined by the measurement accuracy of the transient harmonic.
The harmonics are measured under the same residential sensing network as in Case I, while the transient harmonic source is deployed. There are 100 sampling channels and 3 channels of the original electrical harmonic current measurement, and the harmonic sources are shown in
Figure 11. The lines of the 3 channels are linked to the same point of common coupling (PCC). The out-of-sequence phenomena occur during the measurement process, and at the transient harmonic injection, the measurement data are late for 2 periods, namely a 2-lag disorder.
Figure 12 shows the harmonic measurement error with and without OOSM methods. SEPF and NARX-based OOSM are compared. The 3rd–9th odd harmonics are used to compare the measurement precision.
When the transient harmonic pulse does not occur in Channel 1,
Figure 11 shows that the NARX-based OOSM method leads to a less standard relative error of harmonic measurement than SEPF, as has been discussed in Case I. When the transient harmonic pulse occurs with the 2-lag disorder, the estimation error is remarkable without retrodiction. SEPF and NARX-based OOSM methods both reduce the estimation error, and the NARX-based method offers a better performance. Yet, the estimation error at the transient harmonic event is still larger than other time points. It also shows that the OOSM method performs a larger reduction with frequently changing harmonic waves in Channels 2 and 3. The harmonics in Channel 1 change thoroughly, but smoothly; the OOSM makes a minor fluctuation. This indicates that OOSM modifies greatly instantaneous and fierce harmonic fluctuations, such as voltage sags and instant impacts.
Table 4 lists the MSEs of 100 channels. The results show that among the 4 algorithms, the NARX-based OOSM method gains a better harmonic measurement result than the other 3 methods with frequent transient harmonic injections. The MSEs of the measurements are reduced by more than
with frequent transient harmonic events. Yet, OOSM methods do not make a great difference in measurement accuracy for high order harmonics, since their magnitudes are small. Thus, the data-aware method’s improvement on transient harmonic measurement is verified.
4.6. Case III: Data-Aware Retrodiction-Based Harmonic Identification
To examine the harmonic measurement in a practical environment, the situation of a power outage is simulated in the experiment. An uninterruptible power supply (UPS) maintains the power supply to loads when public power is off. The UPS offers power with a base frequency of
Hz and voltage in
V. When switching the power from public power to UPS, a frequency variance occurs. The frequency fluctuation is shown in
Figure 13, and the base frequency is calculated by Fourier transfer. From 0–30 s, the base frequency fluctuates around 49.99–50.02 Hz. At time point
s, the power supply swiftly changes to UPS, and base frequency is steady at 49.99 Hz, which fulfills the requirement of power supply. The power supply switch leads to frequency fluctuations of
Hz. Harmonic identification is examined in this section to test the effects of various OOSM algorithms on harmonic analysis. Harmonic source identification is one of the key issues in harmonic analysis and a necessity of smart grid safety. Harmonic identification analyzes multi-channel electrical data in a period and is affected by out-of-sequence measured data. A widely-applied method for harmonic identification is independent component analysis (ICA); its principle and realization have been explained in [
41]. In this paper, this method is utilized to testify to the effects of disordered sequences on harmonic source identification accuracy.
The harmonic series are measured under the distributed power sensing network the same as in Cases I and II. The limits of data disorders do not exceed 4 lags. Electrical parameters in 100 channels are collected. The harmonic sources are detected by Fluke 435 and compared to the measurement results of the sensing network. Three channels of the original electrical harmonic current measurement and the harmonic sources are shown in
Figure 14. The 3 channels are linked to the same point of common coupling (PCC), and their harmonics affect each other. Harmonic source identification will separate the sources according to the independent component analysis method introduced in [
41]. The 3rd–9th odd harmonics are measured to examine the effects of the OOSM algorithm on harmonic identification. The sampled electrical data and the correspondent identified harmonic sources’ profile are shown in
Figure 14. The identification errors with 4 different OOSM methods are listed in
Table 5.
The results indicate that the out-of-sequence phenomenon does have an influence on the precision of harmonic identification, and OOSM methods can help to decrease the error. The harmonic identification error may vary according to the identification method. However, with ICA, the NARX-based method gives the largest reduction in identification error among the 4 retrodiction methods. For the 7th and 9th harmonic, since the harmonic magnitudes are small, the harmonic identification error is unstable, the OOSM methods do not make much contribution to improving the precision. The data-aware method’s improvement on harmonic identification is verified.
4.7. Discussion
The experiment results prove the effectiveness of the retrodiction method on improving harmonic measurement accuracy in the cyber-physical energy system testbed. Considering that the out-of-sequence phenomenon is more general in a bandwidth-limited measurement network environment, the retrodiction method is more important. In the case of harmonic sensing, following the requirements in IEC 61000-4-7-2002, only SEPF and NARX-based methods fulfill the standard. This is mainly because the Kalman filter-based methods depend on a linear transmission matrix. The changes of the time-varying harmonics are not linear dynamics, but rather a non-Gaussian statistical model. The particle filter and neural network approximate the model with a Monte Carlo method, offering a better estimation accuracy. With a more complex structure, a neural network can approximate a more sophisticated model. Nevertheless, the improvement of the harmonic measurement precision over SEPF is not very manifest. This indicates that harmonic events can be depicted with a probabilistic model.
The computation cost of the NARX-based method stays stable within the maximum delay of harmonic measurements. This is because of the static structure of neural networks. Although it maintains a better real-time performance over other methods, it limits the application areas. In the harmonic measurement problem, the communication intervals are usually among several seconds, and transmission delay can be limited in a reasonable interval. When designing the NARX-based algorithm, the interval of the practical measurement network should be considered to define the maximum limit of the out-of-sequence delay. In other applications of the cyber-physical energy system, where a high real-time interaction and response with frequent communication among the network are required, the delay limit of out-of-sequence phenomenon can be large. If the size of the NARX-based method increases, the computation and memory cost might grow to a considerable scale. There should be further consideration when dealing with a larger network scale or complicated power quality analysis tasks.
In the harmonic measurement experiment, the results have demonstrated that the out-of-sequence measurement problem does exist, and the OOSM algorithm is capable of improving the harmonic analysis accuracy. In contrast, the measurement of the transient harmonic injection is more vulnerable to the OOSM problem than non-stationary harmonics; the retrodiction method also offers less reduction on the MSE of the transient harmonic measurement. This is because the transient harmonic injection is an independent event and cannot be well estimated by the kinematical function. For the non-stationary harmonic, its amplitude can be predicted to a certain extent; thus, the retrodiction accuracy is guaranteed. Thus, the modeling of the kinematical function is vital to ensure the harmonic measurement accuracy.
The OOSM algorithms use the Kalman filter, the particle filter and the neural network, which correspond to the linear kinematical function, Monte Carlo modeling and multi-layer modeling, respectively. With the complexity of the system arising, the modeling method also needs to be improved to approximate the physical behavior. In cyber-physical energy systems with a large-scale monitoring network, the events of harmonic changes are triggered by customer behaviors of a macro temporal probability distribution. However, in a particular unit, the dynamics of the harmonic cannot be approximated with the Monte Carol method or the Markov chain. From the experiment results, it can be figured that the neural network approximates the harmonic changes better than the Monte Carlo model, rather than the linear model. Thus, the neural network-based retrodiction method offers a better harmonic measurement and analysis accuracy. With a time-varying character, the NARX-based model can update the approximation model from the historical data. Thus, the model is updated by the data through time. Apparently, the harmonic measurement precision can be improved with better modeling of cyber-physical energy systems.
In a real dynamic power network, the system topology and the load types keep changing. This means that the dynamic function described in Equation (
3) is not stable. If the NARX-based OOSM method still fulfills the requirements of online harmonic analysis, the time consumption of the neural network convergence to a new featured model should be limited. However, as the neural network will not be stable after decades or hundreds of training iterations, the time consumption is too long for the dynamic task. This problem can be solved by the idea of transfer learning. The NARX neural network is pre-trained under several load types: linear, stationary, transient, etc. The topology of the grid and the load type are identified continuously. If the input load type or the topology changes for the NARX neural network, the pre-trained weights are deployed. This shall reduce the time consumption to adjust to the new type of load. This is beyond the discussion of the OOSM method and is part of the future work.
Above all, the results suggest that the out-of-sequence measurement problem exists in large-scale distributed sensing networks, and the OOSM algorithm proposed can decrease the influence of the data disorders, which is helpful in online harmonic identification on cyber-physical energy systems.