# Lessons for Data-Driven Modelling from Harmonics in the Norwegian Grid

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

#### 1.1. Motivation and Background

#### 1.2. Relevant Literature

#### 1.2.1. Data-Driven Methods in Power Grids

#### 1.2.2. Harmonic Distortions

#### 1.2.3. The Literature Gap

#### 1.3. Contributions and Organization

## 2. Methodology

#### 2.1. Data Origin

#### 2.2. Data Flow, Extraction, and Processing

#### 2.3. Data Processing: THD and Harmonic Contributions

#### 2.4. Data Processing: Cumulative Distribution Functions, Histograms, and Percentiles

#### 2.5. Data Processing: Time-Distribution of THD Excursions

#### 2.6. Challenges

- Compression Thresholds—The ELSPEC PQA instruments have a compression algorithm that introduces a lower cut-off level for the harmonic components in their compression algorithm. Contributions to the overall signal below this cut-off value for each harmonic component will not be recorded in the stored data from the instrument. This threshold may vary between measuring devices, depending on the harmonic noise and the needs of the measurements at the given site. The threshold is usually set to be in a range from $0.1$ to $0.2$% of the base harmonic component. Values below this level will be stored as 0 values, and is referred to as such in the discussion below.
- Computational Tractability—We had initially set out to load 96 harmonics and six voltages for all three nodes over the four year time period. However, the database proved uncooperative and required frequent restarts during the extraction. We therefore limited the analysis of 96 harmonics to a month.
- THD Calculation—The Elspec instruments also record THD directly, although neither the aggregation interval nor function is clearly documented. We observe a median difference of 21% (ranging from 0 to 56% at the 1 and 99 percentile, respectively) between the THD calculated by our own procedure and the THD directly reported by the Elspec instrument. We base the analysis in this paper on the above THD calculation for transparency reasons.

## 3. Results

#### 3.1. Presence of Harmonics

- Across all voltage levels, non-negligible amounts of non-zero measurements occur only on the third, fifth, and seventh harmonics;
- At the $22\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$ level and across all phases, 95% of measurements of the seventh harmonic are non-zero. For the fifth harmonic, there are non-zero measurements in 70% of cases. The third harmonic differs across phases. On ${V}_{2}$, non-zero measurements are more common (40%) than on ${V}_{1}$ and ${V}_{3}$ (10% each). Non-zero observations on the third and fifth harmonic are clustered in time rather than being spread out evenly. The clusters are not evenly distributed and do not appear to correlate with seasons;
- At the $66\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$ level, we find the same patterns as at the $22\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$ level, with most non-zero measurements found in the seventh, fifth, and third harmonics. Across all phases, we find non-zero values for the seventh and fifth harmonics in 75 and 55% of cases, respectively. For the third harmonic, non-zero values are unbalanced across phases. On ${V}_{1}$, ${V}_{2}$, and ${V}_{3}$, we count 55, 65, and 35% of non-zero values, respectively. Observing no differences in the temporal distribution of counts, ${V}_{3}$ appears to have a generally lower level of non-zero counts;
- At the $300\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$ level, there is a marked difference between the periods of March 2017 to July 2018 and the remainder of the observation period. Inside this period, 45% of measurements across phases (and for the third, fifth, and seventh harmonic channel) are non-zero. Outside this period (and overall), only 3 (19)% of measurements are non-zero (again, for the third, fifth, and seventh harmonic channel). The temporal patterns (in the period of March 2017 to July 2018) are identical across phases, except for the third harmonic channel on ${V}_{2}$, where 87% of all samples are non-zero (compared to 45 and 55% for the third and fifth harmonic, respectively).

**Figure 3.**Fraction of non-zero observations for the first eight harmonics in each voltage channel (denoted as Instrument Channel), grouped by harmonic. Data for all three sites are shown, see panel titles. A period of four years is covered for each site. The fraction (see colormap on the right) is calculated over $\sim 1.3\times {10}^{8}$ samples in each month. It is clear that there are some channels (harmonics for each phase) that are clearly more present than others, and that the pattern is to a large degree transferable from phase to phase and from site to site. It is also clear that there is considerably less harmonic content in the higher voltage levels. This is confirmed in Figure 5 and Figure 6.

**Figure 4.**Variation in the occurrence of non-zero values for each node averaged on a daily basis. The 3, 5 and 7 harmonics have been selected.

**Figure 5.**Cumulative distribution function (CDF) of total harmonic distortion (THD) for three sites (columns). We use 256 bins of uniform logarithmic spacing from ${10}^{-2}$ to 10. Distributions are calculated over the time range from 2015 to 2018 for a total of $\sim 5.9\times {10}^{9}$ samples. Top: CDFs for each phase (see legend). Bottom: Difference between the CDFs for ${V}_{2}$ and ${V}_{1}$ as well as ${V}_{3}$ and ${V}_{1}$, respectively (see legend). Statistically, the three phases always remain within six percent of each other. See Table 2 for summary statistics.

**Figure 6.**Cumulative distribution function (CDF) of total harmonic distortion (THD) for three sites (see legend) for the phase-to-ground voltage ${V}_{1}$ plotted together for comparison. Binning and data basis is the same as for Figure 5. (

**Top**): CDFs for each site (see legend). (

**Bottom**): Difference between the CDFs for 22 and $300\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$ as well as the 66 and $300\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$ sites, respectively (see legend). See Table 2 for summary statistics. For higher voltage levels, the distribution shifts to smaller THD values—there is less noise in the system.

#### 3.2. Total Harmonic Distortion, Phases & Voltage Levels

- Overall, most ($99\%$) of the THD values are small and $\lesssim 1\%$ of their respective fundamental phase voltage. Distributions are narrow with most values concentrated in the range $0.1$ to 1%. Difference between different phases at the same voltage level are always smaller than differences between voltage levels;
- Across phases and voltage levels, the difference between phases is always $\le 6\%$. Note that this only means that the phases are STATISTICALLY within $6\%$ of one another. At any given point in time, their difference may be larger than that;
- Difference between phases cover a wider range of THD for higher voltage levels. The largest integral difference (The area between ${\mathrm{CDF}}_{i}$ and ${\mathrm{CDF}}_{j}$, i.e., $\int \sqrt{{({\mathrm{CDF}}_{i}-{\mathrm{CDF}}_{j})}^{2}}\phantom{\rule{0.277778em}{0ex}}\mathrm{d}\mathrm{THD}$.) between two phases is $0.024$, $0.49$, and $0.056$) for 22, 66, and $300\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$, respectively. For $22\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$, the median values of ${V}_{1}$, ${V}_{2}$, and ${V}_{3}$ remain within 4% of one another. This difference grows to 10% and 15% at 66 and $300\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$, respectively;
- At higher voltage levels, the distributions of THD consistently shift towards smaller values. For $22\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$ (66, 300), $99\%$ of THD measurements (on ${V}_{1}$) are ≤1.48 (≤1.01, ≤0.73). Median THD values shift similarly so that the median THD (on ${V}_{1}$) at $300\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$ ($66\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$) is half (a fifth) of that measured at $22\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$. The $22\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$ site is consistently about half a decade above the $300\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$ site, and the $66\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$ site is located between these a little towards the $300\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$ site.

#### 3.3. Harmonic Contributions to Total Harmonic Distortion

- Across all phases and voltage levels, ≳98% of the contribution towards the THD are concentrated in at most the 13th harmonic. The next largest contributions ($1.9$ percent) is the 29th harmonic on ${V}_{3}$ in the $66\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$ site. Beyond this, all other contributions are ≲1%;
- The highest individual harmonic with a total contribution of ≳2 % are 11th ($22\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$), 13th ($66\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$), and 13th ($300\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$). At 22 and $300\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$, these harmonics also have a significant (>10%) contribution on at least one phase. However, at $66\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$, the largest harmonic with a significant contribution is the 7th;
- At $22\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$, the 7th, 11th, and 5th harmonic contribute the most to THD. In order (and averaged over phases), they contribute ∼52, 37, and $11\%$. Across phases, the contributions to THD are balanced and remain within a few % of one another;
- At $66\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$, the 3rd, 7th, and 5th harmonics contribute the most to the THD. When averaged over all phases, they contribute ∼51, 40, and 7%, respectively. There is an imbalance in the contribution of ${V}_{3}$ which contributes 40% more than ${V}_{1}$ and ${V}_{2}$ to the THD on the 7th harmonic. For the 3rd harmonic, the reverse holds;
- At $300\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$, there are large differences (20 to 40%) between the contribution of each phase to the THD across different harmonics. For example, on ${V}_{1}$, the 5th harmonic dominates THD with a contribution 60%. On ${V}_{3}$, however, the 13th harmonic dominates with a 60% contribution. On ${V}_{3}$, the 3 harmonic drives THD (with a contribution of ∼50%). The authors are not able to attribute this imbalance to any specific phenomena, and this may be the subject of future investigations.

#### 3.4. Harmonic Distortions over Time

- For Sites 1 and 2, non-zero THD values are present during the entire measurement period from 2015 to 2018. For site 3, only 16 out 24 months in 2015 and 2016 and 18 out of 24 months between 2017 and 2018 record THD values above the compression threshold;
- For all sites, THD appears to follow a seasonal pattern. For Site 1 and Site 2, median THD is about 50% higher in summer and autumn than during the winter and spring. For Site 3, the difference is more pronounced due to many periods without observed THD. For 2015 and 2016, non-zero THD values are recorded only in the summer months;
- The spread (difference between the 1 and 99 percentile) of observed THD values (binned monthly) decreases with voltage level. Aggregating across months, the maximum spreads are $1.71$, $1.45$, and $1.00$% for Sites 1, 2, and 3, respectively. In the same order, the average spreads are $0.73$, $0.67$, and $0.27$%. Independent of voltage level, larger spreads always occur in the summer and autumns months;
- For Site 1 (and phase 1), the contribution of the third, fifth, and seventh harmonics to THD over a period of 48 months is in-line with the results for a single month (cf. Figure 7). Over time, the majority of THD is accounted for by the 5th and 7th harmonics. For most months, both harmonics track similar medians (and spreads), except in the spring and autumn of 2017. Over these periods, the 5th harmonic follows a seasonal pattern (lower during winter/spring, larger during summer/autumn) while the 7th harmonic keeps an almost constant median. Their combined contribution leads to the deviation from seasonality earlier observed in THD;
- For Sites 2 and 3, the contribution of individual harmonics to THD is more complicated. For Site 2, considering only January 2017 suggests that the 3rd and 7th harmonic should contribute most to THD. However, over time, we observe a different pattern. Here, the 7th harmonic appears to set a baseline of distortion (with slight seasonality), the 5th harmonic modulates additional (stronger) seasonality in the median as well as additional noise (larger spread), and the 3rd harmonic adds even more noise (larger spread). This shows that the analysis of a single month is insufficient and unlikely to be representative of THD and harmonic contributions over longer time frames.

#### 3.5. Temporal Distribution of THD Excursions

- At timescales $300\phantom{\rule{0.277778em}{0ex}}\mathrm{s}<s\le {10}^{5}\phantom{\rule{0.277778em}{0ex}}\mathrm{s}$ (a few days), we find $\mathcal{D}\sim 0.34<1$ (with slight variations across voltage levels, but a goodness of fit $R\sim 0.99$ for each level). This suggest multi-scale substructure of THD excursions in time. Visually, this is manifested as clumping of THD excursions (see Figure 9, lower panel). Clusters also vary in size (duration) and can be decomposed into further (sub-)clusters;
- At time-scale $s\ge {10}^{5}\phantom{\rule{0.277778em}{0ex}}\mathrm{s}$, we find $\mathcal{D}\sim 1$ ($R\sim 0.99$). This suggests no (or at least very little) temporal substructure in the distribution of THD excursions. Visually speaking, there are long sequences of THD excursions with similar timing (Figure 9, upper panel). There are only occasional large gaps in time.

## 4. Discussion

#### 4.1. Regulation on Harmonic Distortion

#### 4.2. Trends in THD and Harmonic Contributions

#### 4.3. Towards Event Prediction

#### 4.4. Statistical Robustness and Time-Correlations

#### 4.5. Actionable Event Predictions

## 5. Conclusions & Future Work

- The distribution of harmonics differs with phases and voltage level (site);
- There is little power (below the Elspec instrument cut-off) beyond the 13 harmonic;
- There is temporal clumping of events;
- There is seasonality on different time-scales.

- Variations in harmonic power with phase and voltage level suggests that two-step training procedures akin to transfer learning may be useful. In such a scheme, one would (i) train a baseline model on data from all nodes and all harmonics, and then (ii) fine-tune the model to with data from specific sites. This will result in a model specific to each site;
- The lack of power beyond the 13th harmonic suggests that including higher-order harmonics will not increase the predictive power of models;
- Clumping suggests that models should include features such as the time-since-last-event to distinguish between grid states (frequent alarms vs. nominal operations);
- Seasonality suggests that models should include features such as the hour of the day or the month of the year.

## Author Contributions

## Funding

## Data Availability Statement

## Conflicts of Interest

## Abbreviations

PQSCADA | Name of the Power Quality Management Software |

TSO | Transmission System Operator |

DSO | Distribution System Operator |

PQA | Power Quality Analyzer |

THD | Total Harmonic Distortion |

CDF | Cummulative Distribution Function |

ML | Machine Learning |

PQ | Power Quality |

## References

- Kumar, G.V.B.; Sarojini, R.K.; Palanisamy, K.; Padmanaban, S.; Holm-Nielsen, J.B. Large Scale Renewable Energy Integration: Issues and Solutions. Energies
**2019**, 12, 1996. [Google Scholar] [CrossRef][Green Version] - Muljadi, E.; McKenna, H. Power quality issues in a hybrid power system. IEEE Trans. Ind. Appl.
**2002**, 38, 803–809. [Google Scholar] [CrossRef] - Rönnberg, S.; Bollen, M. Power quality issues in the electric power system of the future. Electr. J.
**2016**, 29, 49–61. [Google Scholar] [CrossRef] - Balasubramaniam, P.M.; Prabha, S.U. Power Quality Issues, Solutions and Standards: A Technology Review. J. Appl. Sci. Eng.
**2015**, 18, 371–380. [Google Scholar] [CrossRef] - Sallam, A.A.; Malik, O.P. Electric Distribution Systems; Wiley-Blackwell: Hoboken, NJ, USA, 2018; pp. 1–604. [Google Scholar]
- Bashir, A.K.; Khan, S.; Prabadevi, B.; Deepa, N.; Alnumay, W.S.; Gadekallu, T.R.; Maddikunta, P.K.R. Comparative analysis of machine learning algorithms for prediction of smart grid stability. Int. Trans. Electr. Energy Syst.
**2021**, 31, e12706. [Google Scholar] [CrossRef] - Azad, S.; Sabrina, F.; Wasimi, S. Transformation of smart grid using machine learning. In Proceedings of the 29th Australasian Universities Power Engineering Conference (AUPEC), Nadi, Fiji, 26–29 November 2019; pp. 1–6. [Google Scholar]
- Rangel-Martinez, D.; Nigam, K.; Ricardez-Sandoval, L.A. Machine learning on sustainable energy: A review and outlook on renewable energy systems, catalysis, smart grid and energy storage. Chem. Eng. Res. Des.
**2021**, 174, 414–441. [Google Scholar] [CrossRef] - Hossain, E.; Khan, I.; Un-Noor, F.; Sikander, S.S.; Sunny, M.S.H. Application of Big Data and Machine Learning in Smart Grid, and Associated Security Concerns: A Review. IEEE Access
**2019**, 7, 13960–13988. [Google Scholar] [CrossRef] - Ibrahim, M.S.; Dong, W.; Yang, Q. Machine learning driven smart electric power systems: Current trends and new perspectives. Appl. Energy
**2020**, 272, 115237. [Google Scholar] [CrossRef] - LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature
**2015**, 521, 436–444. [Google Scholar] [CrossRef] - Shrestha, A.; Mahmood, A. Review of deep learning algorithms and architectures. IEEE Access
**2019**, 7, 53040–53065. [Google Scholar] [CrossRef] - Hastie, T.; Tibshirani, R.; Friedman, J.H.; Friedman, J.H. The Elements of Statistical Learning: Data Mining, Inference, and Prediction; Springer: Berlin/Heidelberg, Germany, 2009; Volume 2. [Google Scholar]
- James, G.; Witten, D.; Hastie, T.; Tibshirani, R. An Introduction to Statistical Learning; Springer: Berlin/Heidelberg, Germany, 2013; Volume 112. [Google Scholar]
- Raschka, S. Model evaluation, model selection, and algorithm selection in machine learning. arXiv
**2018**, arXiv:1811.12808. [Google Scholar] - Yu, T.; Zhu, H. Hyper-parameter optimization: A review of algorithms and applications. arXiv
**2020**, arXiv:2003.05689. [Google Scholar] - Probst, P.; Wright, M.N.; Boulesteix, A.L. Hyperparameters and tuning strategies for random forest. Wiley Interdiscip. Rev. Data Min. Knowl. Discov.
**2019**, 9, e1301. [Google Scholar] [CrossRef][Green Version] - CEER. 6th CEER Benchmarking Report on the Quality of Electricity and Gas Supply; CEER: Brussels, Belgium, 2016. [Google Scholar]
- García, S.; Ramírez-Gallego, S.; Luengo, J.; Benítez, J.M.; Herrera, F. Big data preprocessing: Methods and prospects. Big Data Anal.
**2016**, 1, 1–22. [Google Scholar] [CrossRef][Green Version] - Heaton, J. An empirical analysis of feature engineering for predictive modeling. In Proceedings of the SoutheastCon, Norfolk, VA, USA, 30 March–3 April 2016; pp. 1–6. [Google Scholar]
- Al-Sheikh, H.; Moubayed, N. Fault detection and diagnosis of renewable energy systems: An overview. In Proceedings of the 2012 International Conference on Renewable Energies for Developing Countries (REDEC), Beirut, Lebanon, 28–29 November 2012; pp. 1–7. [Google Scholar] [CrossRef]
- Pérez-Ortiz, M.; Jiménez-Fernández, S.; Gutiérrez, P.A.; Alexandre, E.; Hervás-Martínez, C.; Salcedo-Sanz, S. A Review of Classification Problems and Algorithms in Renewable Energy Applications. Energies
**2016**, 9, 607. [Google Scholar] [CrossRef] - Kusiak, A.; Li, W. The prediction and diagnosis of wind turbine faults. Renew. Energy
**2011**, 36, 16–23. [Google Scholar] [CrossRef] - Kusiak, A.; Verma, A. Analyzing bearing faults in wind turbines: A data-mining approach. Renew. Energy
**2012**, 48, 110–116. [Google Scholar] [CrossRef] - Betti, A.; Crisostomi, E.; Paolinelli, G.; Piazzi, A.; Ruffini, F.; Tucci, M. Condition monitoring and predictive maintenance methodologies for hydropower plants equipment. Renew. Energy
**2021**, 171, 246–253. [Google Scholar] [CrossRef] - Fu, C.; Ye, L.; Liu, Y.; Yu, R.; Iung, B.; Cheng, Y.; Zeng, Y. Predictive maintenance in intelligent-control-maintenance-management system for hydroelectric generating unit. IEEE Trans. Energy Convers.
**2004**, 19, 179–186. [Google Scholar] [CrossRef] - Garoudja, E.; Chouder, A.; Kara, K.; Silvestre, S. An enhanced machine learning based approach for failures detection and diagnosis of PV systems. Energy Convers. Manag.
**2017**, 151, 496–513. [Google Scholar] [CrossRef][Green Version] - Li, X.; Li, W.; Yang, Q.; Yan, W.; Zomaya, A.Y. An unmanned inspection system for multiple defects detection in photovoltaic plants. IEEE J. Photovolt.
**2019**, 10, 568–576. [Google Scholar] [CrossRef] - Berghout, T.; Benbouzid, M.; Ma, X.; Djurović, S.; Mouss, L.H. Machine Learning for Photovoltaic Systems Condition Monitoring: A Review. In Proccedings of the IECON 2021—47th Annual Conference of the IEEE Industrial Electronics Society, Toronto, ON, Canada, 13–16 October 2021; pp. 1–5. [Google Scholar] [CrossRef]
- Bosman, L.B.; Leon-Salas, W.D.; Hutzel, W.; Soto, E.A. PV System Predictive Maintenance: Challenges, Current Approaches, and Opportunities. Energies
**2020**, 13, 1398. [Google Scholar] [CrossRef][Green Version] - Sica, F.C.; Guimarães, F.G.; de Oliveira Duarte, R.; Reis, A.J. A cognitive system for fault prognosis in power transformers. Electr. Power Syst. Res.
**2015**, 127, 109–117. [Google Scholar] [CrossRef] - Kabir, F.; Foggo, B.; Yu, N. Data Driven Predictive Maintenance of Distribution Transformers. In Proccedings of the 2018 China International Conference on Electricity Distribution (CICED), Tianjin, China, 17–19 September 2018; pp. 312–316. [Google Scholar] [CrossRef]
- Mirowski, P.; LeCun, Y. Statistical Machine Learning and Dissolved Gas Analysis: A Review. IEEE Trans. Power Deliv.
**2012**, 27, 1791–1799. [Google Scholar] [CrossRef] - Donadio, L.; Fang, J.; Porté-Agel, F. Numerical Weather Prediction and Artificial Neural Network Coupling for Wind Energy Forecast. Energies
**2021**, 14, 338. [Google Scholar] [CrossRef] - Heinermann, J.; Kramer, O. Machine learning ensembles for wind power prediction. Renew. Energy
**2016**, 89, 671–679. [Google Scholar] [CrossRef] - Yagli, G.M.; Yang, D.; Srinivasan, D. Automatic hourly solar forecasting using machine learning models. Renew. Sustain. Energy Rev.
**2019**, 105, 487–498. [Google Scholar] [CrossRef] - Voyant, C.; Notton, G.; Kalogirou, S.; Nivet, M.L.; Paoli, C.; Motte, F.; Fouilloy, A. Machine learning methods for solar radiation forecasting: A review. Renew. Energy
**2017**, 105, 569–582. [Google Scholar] [CrossRef] - Foley, A.M.; Leahy, P.G.; Marvuglia, A.; McKeogh, E.J. Current methods and advances in forecasting of wind power generation. Renew. Energy
**2012**, 37, 1–8. [Google Scholar] [CrossRef][Green Version] - Riemer-Sørensen, S.; Rosenlund, G.H. Deep Reinforcement Learning for Long Term Hydropower Production Scheduling. In Proceedings of the 2020 International Conference on Smart Energy Systems and Technologies (SEST), Istanbul, Turkey, 7–9 September 2020; pp. 1–6. [Google Scholar] [CrossRef]
- Bordin, C.; Skjelbred, H.I.; Kong, J.; Yang, Z. Machine Learning for Hydropower Scheduling: State of the Art and Future Research Directions. Procedia Comput. Sci.
**2020**, 176, 1659–1668. [Google Scholar] [CrossRef] - Fotopoulou, M.C.; Drosatos, P.; Petridis, S.; Rakopoulos, D.; Stergiopoulos, F.; Nikolopoulos, N. Model Predictive Control for the Energy Management in a District of Buildings Equipped with Building Integrated Photovoltaic Systems and Batteries. Energies
**2021**, 14, 3369. [Google Scholar] [CrossRef] - Wu, X.; Hu, X.; Moura, S.; Yin, X.; Pickert, V. Stochastic control of smart home energy management with plug-in electric vehicle battery energy storage and photovoltaic array. J. Power Sources
**2016**, 333, 203–212. [Google Scholar] [CrossRef][Green Version] - Mouli, G.R.C.; Kefayati, M.; Baldick, R.; Bauer, P. Integrated PV charging of EV fleet based on energy prices, V2G, and offer of reserves. IEEE Trans. Smart Grid
**2017**, 10, 1313–1325. [Google Scholar] [CrossRef] - Wang, X.; Nie, Y.; Cheng, K.W.E. Distribution system planning considering stochastic EV penetration and V2G behavior. IEEE Trans. Intell. Transp. Syst.
**2019**, 21, 149–158. [Google Scholar] [CrossRef] - McLoughlin, F.; Duffy, A.; Conlon, M. A clustering approach to domestic electricity load profile characterisation using smart metering data. Appl. Energy
**2015**, 141, 190–199. [Google Scholar] [CrossRef][Green Version] - Haben, S.; Singleton, C.; Grindrod, P. Analysis and clustering of residential customers energy behavioral demand using smart meter data. IEEE Trans. Smart Grid
**2015**, 7, 136–144. [Google Scholar] [CrossRef] - Seyedzadeh, S.; Rahimian, F.P.; Glesk, I.; Roper, M. Machine learning for estimation of building energy consumption and performance: A review. Vis. Eng.
**2018**, 6, 1–20. [Google Scholar] [CrossRef] - Chou, J.S.; Tran, D.S. Forecasting energy consumption time series using machine learning techniques based on usage patterns of residential householders. Energy
**2018**, 165, 709–726. [Google Scholar] [CrossRef] - Gonzalez-Briones, A.; Hernandez, G.; Corchado, J.M.; Omatu, S.; Mohamad, M.S. Machine learning models for electricity consumption forecasting: A review. In Proceedings of the 2019 2nd International Conference on Computer Applications & Information Security (ICCAIS), Riyadh, Saudi Arabia, 1–3 May 2019; pp. 1–6. [Google Scholar]
- Manivinnan, K.; Benner, C.L.; Don Russell, B.; Wischkaemper, J.A. Automatic identification, clustering and reporting of recurrent faults in electric distribution feeders. In Proceedings of the 19th International Conference on Intelligent System Application to Power Systems, San Antonio, TX, USA, 17–20 September 2017. [Google Scholar] [CrossRef]
- Viegas, J.L.; Vieira, S.M.; Melicio, R.; Matos, H.A.; Sousa, J.M. Prediction of events in the smart grid: Interruptions in distribution transformers. In Proceedings of the 2016 IEEE International Power Electronics and Motion Control Conference, Varna, Bulgaria, 25–28 September 2016. [Google Scholar] [CrossRef]
- Eskandarpour, R.; Khodaei, A. Machine Learning Based Power Grid Outage Prediction in Response to Extreme Events. IEEE Trans. Power Syst.
**2017**, 32. [Google Scholar] [CrossRef] - Kumar, R.; Singh, B.; Shahani, D.T.; Chandra, A.; Al-Haddad, K. Recognition of Power-Quality Disturbances Using S-Transform-Based ANN Classifier and Rule-Based Decision Tree. IEEE Trans. Ind. Appl.
**2015**, 51. [Google Scholar] [CrossRef] - Zyabkina, O.; Domagk, M.; Meyer, J.; Schegner, P. A feature-based method for automatic anomaly identification in power quality measurements. In Proceedings of the 2018 International Conference on Probabilistic Methods Applied to Power Systems, Boise, ID, USA, 24–28 June 2018. [Google Scholar] [CrossRef]
- Vantuch, T.; Misak, S.; Jezowicz, T.; Burianek, T.; Snasel, V. The Power Quality Forecasting Model for Off-Grid System Supported by Multiobjective Optimization. IEEE Trans. Ind. Electron.
**2017**, 64, 9507–9516. [Google Scholar] [CrossRef] - Hoffmann, V.; Michałowska, K.; Andresen, C.; Torsæter, B.N. Incipient Fault Prediction in Power Quality Monitoring. In Proceedings of the 25th International Conference on Electricity Distribution (CIRED), Madrid, Spain, 3–6 June 2019. [Google Scholar]
- Andresen, C.A.; Torsæter, B.N.; Haugdal, H.; Uhlen, K. Fault Detection and Prediction in Smart Grids. In Proceedings of the 9th International Workshop on Applied Measurements for Power Systems, Bologna, Italy, 26–28 September 2018. [Google Scholar] [CrossRef][Green Version]
- Hoiem, K.W.; Santi, V.; Torsater, B.N.; Langseth, H.; Andresen, C.A.; Rosenlund, G.H. Comparative Study of Event Prediction in Power Grids using Supervised Machine Learning Methods. In Proceedings of the 2020 International Conference on Smart Energy Systems and Technologies (SEST), Istanbul, Turkey, 7–9 September 2020. [Google Scholar] [CrossRef]
- Rosenlund, G.H.; Hoiem, K.W.; Torsater, B.N.; Andresen, C.A. Clustering and Dimensionality-reduction Techniques Applied on Power Quality Measurement Data. In Proceedings of the 2020 International Conference on Smart Energy Systems and Technologies (SEST), Istanbul, Turkey, 7–9 September 2020. [Google Scholar] [CrossRef]
- Tyvold, T.S.; Nybakk Torsater, B.; Andresen, C.A.; Hoffmann, V. Impact of the Temporal Distribution of Faults on Prediction of Voltage Anomalies in the Power Grid. In Proceedings of the 2020 International Conference on Smart Energy Systems and Technologies (SEST), Istanbul, Turkey, 7–9 September 2020. [Google Scholar] [CrossRef]
- Michalowska, K.; Hoffmann, V.; Andresen, C. Impact of seasonal weather on forecasting of power quality disturbances in distribution grids. In Proceedings of the 2020 International Conference on Smart Energy Systems and Technologies (SEST), Istanbul, Turkey, 7–9 September 2020. [Google Scholar] [CrossRef]
- Li, Y.; Wang, T.; Zhou, S.; Liu, Y. Power quality data analysis based on a state-wide monitoring system in China. In Proceedings of the International Conference on Power System Technology, Guangzhou, China, 6–9 November 2018; pp. 3734–3739. [Google Scholar]
- Santoso, S.; Sabin, D.D.; McGranaghan, M.F. Evaluation of harmonic trends using statistical process control methods. In Proceedings of the Transmission and Distribution Conference and Exposition, Chicago, IL, USA, 21–24 April 2008; p. 1169. [Google Scholar]
- Guan, J.L.; Yang, M.T.; Gu, J.C.; Chang, H.H. Effect of harmonic power fluctuation on voltage flicker. In Proceedings of the 11th WSEAS International Conference on Systems; 2007; pp. 429–435. Available online: https://zenodo.org/record/1333048#.YpQj-1RBxPY (accessed on 26 April 2022).
- IRENA. Energy Profile; IRENA: Oslo, Norway, 2018. [Google Scholar]
- Van Rossum, G.; Drake, F.L. Python 3 Reference Manual; CreateSpace: Scotts Valley, CA, USA, 2009. [Google Scholar]
- Harris, C.R.; Millman, K.J.; van der Walt, S.J.; Gommers, R.; Virtanen, P.; Cournapeau, D.; Wieser, E.; Taylor, J.; Berg, S.; Smith, N.J.; et al. Array programming with NumPy. Nature
**2020**, 585, 357–362. [Google Scholar] [CrossRef] [PubMed] - McKinney, W. Data Structures for Statistical Computing in Python. In Proceedings of the 9th Python in Science Conference, Austin, Texas, 28 June–3 July 2010; pp. 56–61. Available online: https://conference.scipy.org/proceedings/scipy2010/pdfs/mckinney.pdf (accessed on 26 April 2022).
- Dubuc, B.; Quiniou, J.F.; Roques-Carmes, C.; Tricot, C.; Zucker, S.W. Evaluating the fractal dimension of profiles. Phys. Rev. A
**1989**, 39, 1500. [Google Scholar] [CrossRef] [PubMed] - Das, J. Power System Harmonics and Passive Filter Designs; John Wiley and Sons, Ltd.: Hoboken, NJ, USA, 2015; Chapter 1; pp. 1–29. [Google Scholar] [CrossRef]
- Zare, F.; Soltani, H.; Kumar, D.; Davari, P.; Delpino, H.A.M.; Blaabjerg, F. Harmonic Emissions of Three-Phase Diode Rectifiers in Distribution Networks. IEEE Access
**2017**, 5, 2819–2833. [Google Scholar] [CrossRef] - Kanao, N.; Hayashi, Y.; Matsuki, J. Analysis of Even Harmonics Generation in an Isolated Electric Power System. Electr. Eng. Jpn.
**2009**, 167, 56–63. [Google Scholar] [CrossRef] - Norges Vassdrags Og Energidirektorat (NVE). Forskrift om Leveringskvalitet i Kraftsystemet (FOL), 2021. Data Retrieved from Lovdata on 2021-12-21. Available online: https://lovdata.no/dokument/SF/forskrift/2004-11-30-1557#KAPITTEL_4 (accessed on 26 April 2022).
- Das, S.; Santoso, S.; Maitra, A. Effects of distributed generators on impedance-based fault location algorithms. In Proceedings of the IEEE Power and Energy Society General Meeting, National Harbor, MD, USA, 27–31 July 2014; Volume 2014. [Google Scholar]
- Xiao, F.; Ai, Q. Data-Driven Multi-Hidden Markov Model-Based Power Quality Disturbance Prediction That Incorporates Weather Conditions. IEEE Trans. Power Syst.
**2019**, 34, 402–412. [Google Scholar] [CrossRef] - Nandi, A.; Debnath, S. Recognition of harmonic sources in distribution network using fractal analysis. In Proceedings of the 1st International Conference on Control, Measurement and Instrumentation, Kolkata, India, 8–10 January 2016. [Google Scholar] [CrossRef]
- Zhou, J.; LI, X.; Ren, Z. Power-Load Fault Diagnosis via Fractal Similarity Analysis. In Proceedings of the 12th IEEE PES Asia-Pacific Power and Energy Engineering Conference (APPEEC), Nanjing, China, 20–23 September 2020. [Google Scholar] [CrossRef]
- Zhou, T.; Lu, J.; Li, B.; Tan, Y. Fractal analysis of power grid faults and cross correlation for the faults and meteorological factors. IEEE Access
**2020**, 8. [Google Scholar] [CrossRef] - Gneiting, T.; Schlather, M. Stochastic Models That Separate Fractal Dimension and the Hurst Effect. SIAM Rev.
**2004**, 46, 269–282. [Google Scholar] [CrossRef][Green Version] - Santi, V.M. Predicting Faults in Power Grids Using Machine Learning Methods; Technical Report; Norwegian University of Science and Technology (NTNU): Oslo, Norway, 2019. [Google Scholar]
- Meen, H.K.; Jahr, C. Power Wave Analysis and Prediction of Faults in the Norwegian Power Grid; Technical Report; Norwegian University of Science and Technology (NTNU): Oslo, Norway, 2020. [Google Scholar]
- Hoffmann, V.; Klemets, J.R.A.; Torsæter, B.N.; Rosenlund, G.H.; Andresen, C.A. The value of multiple data sources in machine learning models for power system event prediction. In Proceedings of the 2021 International Conference on Smart Energy Systems and Technologies (SEST), Vaasa, Finland, 8 September 2021; pp. 1–6. [Google Scholar]
- Weiss, K.; Khoshgoftaar, T.M.; Wang, D. A survey of transfer learning. J. Big Data
**2016**, 3, 1–40. [Google Scholar] [CrossRef][Green Version]

**Figure 1.**Illustration of the scope of the paper. The figure shows the entire value chain of electricity (from left to right—generation, distribution, and consumption). Machine learning techniques are relevant in all links of the chain (see also our literature overview). Our focus (highlighted in blue and black) is on the background state of the grid at transmission and distribution voltage levels.

**Figure 2.**Dataflow (left to right) from source to analysis. Boxes indicate processing steps and text on arrows indicates the file format used. Grey shading indicates proprietary technology. Unshaded steps are our own scripts (based on the Python). The ETL (Extract-Transform-Load) steps interact with Elspec’s proprietary PQSCADA system (using our own Dynamic Data Grabber package) to extract voltage and harmonics data as dataframes into Parquet files. Dataframes are aggregated by month and then consumed by various analysis scripts. These output data in HDF5 format for use by plotting scripts.

**Figure 7.**Total average contribution to THD over the period of one month per phase (columns) and site (rows). For each cycle, we calculate THD as well as each harmonics’ contribution to THD. We then average over all samples of January 2017.

**Figure 8.**Statistical descriptors of the THD and selected harmonics (rows) aggregated per month for the three sites (columns). For each month, we indicate the median as well as 1, 10, 25, 75, 90, and 99 percentile (see legend). Grey shaded bands indicate the passage of one year.

**Figure 9.**Left panel: Illustration of the temporal distribution of 99 percentile harmonic occurrence for all three sites spanning one year (top-left-panel) and one week (lower-left-panel). Lower time resolution in the figure is one minute. Right panel: Result from the application of the box-counting algorithm for determining the fractal nature of the distribution of the events in time.

**Table 1.**We extract two data packages. The first covers four years, six voltage channels, eight harmonics. The second covers a month, three voltage channels, 96 harmonics. They contain $3.6\times {10}^{11}$ and $3.9\times {10}^{10}$ samples, respectively.

Site | Voltage | Period | ${\mathit{V}}_{\phantom{\rule{0.277778em}{0ex}}\mathbf{Phases}}^{\mathbf{Harmonics}}$ | Aggregation |
---|---|---|---|---|

1 | $22\phantom{\rule{0.166667em}{0ex}}\mathrm{kV}$ | 2015 to 2018 | ${V}_{\phantom{\rule{0.277778em}{0ex}}1,2,3,12,23,31}^{0\cdots 8}$ | $1/50\phantom{\rule{0.166667em}{0ex}}\mathrm{Hz}$, Mean |

2 | $66\phantom{\rule{0.166667em}{0ex}}\mathrm{kV}$ | 2015 to 2018 | ${V}_{\phantom{\rule{0.277778em}{0ex}}1,2,3,12,23,31}^{0\cdots 8}$ | $1/50\phantom{\rule{0.166667em}{0ex}}\mathrm{Hz}$, Mean |

3 | $300\phantom{\rule{0.166667em}{0ex}}\mathrm{kV}$ | 2015 to 2018 | ${V}_{\phantom{\rule{0.277778em}{0ex}}1,2,3,12,23,31}^{0\cdots 8}$ | $1/50\phantom{\rule{0.166667em}{0ex}}\mathrm{Hz}$, Mean |

1 | $22\phantom{\rule{0.166667em}{0ex}}\mathrm{kV}$ | January 2017 | ${V}_{\phantom{\rule{0.277778em}{0ex}}1,2,3}^{0\cdots 96}$ | $1/50\phantom{\rule{0.166667em}{0ex}}\mathrm{Hz}$, Mean |

2 | $66\phantom{\rule{0.166667em}{0ex}}\mathrm{kV}$ | January 2017 | ${V}_{\phantom{\rule{0.277778em}{0ex}}1,2,3}^{0\cdots 96}$ | $1/50\phantom{\rule{0.166667em}{0ex}}\mathrm{Hz}$, Mean |

3 | $300\phantom{\rule{0.166667em}{0ex}}\mathrm{kV}$ | January 2017 | ${V}_{\phantom{\rule{0.277778em}{0ex}}1,2,3}^{0\cdots 96}$ | $1/50\phantom{\rule{0.166667em}{0ex}}\mathrm{Hz}$, Mean |

**Table 2.**Summary statistics for the distribution of total harmonic distortion (THD) per site and phase.

Site | Phase | 1 Percentile | Median | 99 Percentile |
---|---|---|---|---|

$22\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$ | ${V}_{1}$ | $0.15$ | $0.49$ | $1.48$ |

${V}_{2}$ | $0.19$ | $0.51$ | $1.48$ | |

${V}_{3}$ | $0.19$ | $0.51$ | $1.48$ | |

$66\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$ | ${V}_{1}$ | $0.04$ | $0.29$ | $1.01$ |

${V}_{2}$ | $0.05$ | $0.31$ | $1.13$ | |

${V}_{3}$ | $0.05$ | $0.32$ | $1.26$ | |

$300\phantom{\rule{0.277778em}{0ex}}\mathrm{kV}$ | ${V}_{1}$ | $0.05$ | $0.24$ | $0.73$ |

${V}_{2}$ | $0.05$ | $0.28$ | $0.73$ | |

${V}_{3}$ | $0.05$ | $0.25$ | $0.77$ |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Hoffmann, V.; Torsæter, B.N.; Rosenlund, G.H.; Andresen, C.A.
Lessons for Data-Driven Modelling from Harmonics in the Norwegian Grid. *Algorithms* **2022**, *15*, 188.
https://doi.org/10.3390/a15060188

**AMA Style**

Hoffmann V, Torsæter BN, Rosenlund GH, Andresen CA.
Lessons for Data-Driven Modelling from Harmonics in the Norwegian Grid. *Algorithms*. 2022; 15(6):188.
https://doi.org/10.3390/a15060188

**Chicago/Turabian Style**

Hoffmann, Volker, Bendik Nybakk Torsæter, Gjert Hovland Rosenlund, and Christian Andre Andresen.
2022. "Lessons for Data-Driven Modelling from Harmonics in the Norwegian Grid" *Algorithms* 15, no. 6: 188.
https://doi.org/10.3390/a15060188