Towards Convergence in Federated Learning via NonIID Analysis in a Distributed Solar Energy Grid
Abstract
:1. Introduction
 This work defines the viable casebycase scenarios of the locally collected nonIID dataset. It suggests a quantification scheme of the nonIID degree in a regressionbased FL network with both local and global perspectives. To the best of my knowledge, this is the first study that covers the domain of local nonIID attributes in the FL regression task.
 This work suggests theoretical convergence analysis in FL regression optimization based on the quantitative degree of IID and nonIID attributes at the local training dataset, proposing a practical training approach to enhance the convergence rate of the global model.
 This work validates the suggested analysis through multiple experiments in the FL model with the distributed solar energy generation dataset in specific regions of South Korea within the period of January 2017 to August 2021 to empirically vindicate the usability and performance of the suggested FL regression and update periods in future applications in the Smart Grid system.
2. Related Works
3. FL Regression: Trained with NonIID and IID
3.1. Federated Learning
3.2. NonIID Cases in RegressionBased FL
3.2.1. NonIID Cases with a Structural View
 Case A. $n\left({D}_{\left(\exists i,train\right)}^{\left(T,t\right)}\right)\ne n\left({D}_{\left(\exists {i}^{\prime},train\right)}^{\left(T,t\right)}\right)s.t.i\ne {i}^{\prime}and{D}_{i}=\left({{\displaystyle \cup}}_{\forall j}{x}_{j},{{\displaystyle \cup}}_{\forall j}{y}_{j}\right)$.
 Case B. $\psi \left(\mathbb{D}\left({{\displaystyle \cup}}_{\forall j}{y}_{\left(\exists i,j\right)}\right),\mathbb{D}\left({{\displaystyle \cup}}_{\forall j}{y}_{\left(\exists {i}^{\prime},j\right)}\right)\right)\gg 0\text{}s.t.\text{}i\ne {i}^{\prime}\text{}and\text{}n\left({{\displaystyle \cup}}_{\forall j}{y}_{\left(\exists i,j\right)}\right)=n\left({{\displaystyle \cup}}_{\forall j}{y}_{\left(\exists {i}^{\prime},j\right)}\right)$ in (5).
 Case C. $\frac{1}{{{\displaystyle \cup}}_{\forall j}1}{\displaystyle \sum}_{\forall j}\left(\delta \left({y}_{\left(i,j\right)},{y}_{\left({i}^{\prime},j\right)}\right)\right)\gg 0$.
3.2.2. NonIID Cases in FLStructural View
 Case 1. $\left({M}_{\exists i}^{\left(\exists T\right)},{M}_{\exists {i}^{\prime}}^{\left(\exists T\right)}\right)\to caseAorBorC$.
 Case 2. $\left({M}_{\exists i}^{\left(\exists T\right)},{M}_{\exists {i}^{\prime}}^{\left(\exists T\right)}\right)\nrightarrow caseAorBorC$.
 Case 3. $\left({M}_{\forall i}^{\left(\exists T\right)},{M}_{\forall i}^{\left(\exists {T}^{\prime}\right)}\right)\to caseAorBorC$.
 Case 4. $\left({M}_{\forall i}^{\left(\exists T\right)},{M}_{\forall i}^{\left(\exists {T}^{\prime}\right)}\right)\nrightarrow caseAorBorC$.
 Case 5. $\left({M}_{\forall i}^{\left(\forall T\right)},\mathbb{M}\right)\to caseAorBorC$.
 Case 6. $\left({M}_{\forall i}^{\left(\forall T\right)},\mathbb{M}\right)\nrightarrow caseAorBorC$.
3.3. IID Dataset in FL Regression
4. Property and Performance Analysis of FL Regression
4.1. Dataset Analysis
4.2. Training RNNBased Models in Local Client
4.3. Merging Heterogeneous Local Models in FL
4.4. Federated Learning on Periodical Update
 Case $\mathcal{A}$. Daily update period, such that $\mathsf{\mathbb{t}}=24$, and $\mathbb{T}=\frac{c}{\mathsf{\mathbb{t}}}$ in (3).
 Case $\mathcal{B}$. Monthly period, such that $\mathsf{\mathbb{t}}=24\cdot Days$, and $\mathbb{T}=\frac{c}{\mathsf{\mathbb{t}}\cdot Days}$ in (3).
 Case $\mathcal{C}$. Yearly period, such that $\mathsf{\mathbb{t}}=24\cdot Days\cdot Months$, and $\mathbb{T}=\frac{c}{\mathsf{\mathbb{t}}\cdot Days\cdot Months}$ in (3).
5. Conclusions
Funding
Data Availability Statement
Conflicts of Interest
Notations
Notation  Definition  Notation  Definition 
$\mathbb{G}$  Global model  $t$  Local epoch 
$\mathbb{W}$  Set of global parameters  $T$  Global epoch 
$W$  Tensor shape of parameters  $\mathsf{\mathbb{t}}$  Number of local epoch 
$w,b$  2d vector of weights and bias  $\mathbb{T}$  Number of global epoch 
$i,{i}^{\prime}$  Local client index ($i\ne {i}^{\prime}$)  $\mathbb{L}(\cdot )$  Loss function of input 
$j,{j}^{\prime}$  $i$’s parameter index ($j\ne {j}^{\prime}$)  $u,h$  Set of hidden units 
$c,\widehat{c}$  Constant  $\ell ,k$  Layer, Layer index 
$D$  Dataset  $\u03f5$  Error term 
$R$  Required number of i  ${\mathbb{R}}^{d}$  Loss space with $d$ dimensions 
$\mathbb{D}(\cdot )$  Distribution of input  $\eta $  Learning rate 
$\psi \left(a,b\right)$  Kullback Leibler divergence of $a,b$  $\mathbb{M}$  Ideal benchmark dataset 
$M$  Local model  ${\mathsf{\U0001d4c9}}_{sorh}$  Time unit s: sec, h: hour 
$\sigma (\cdot )$  Standard deviation of the input  $f$  Set of feature; $f\in f$ 
$\mu (\cdot )$  Average of the input  $x\in x$  Input 
$\tau $  sampling rate per second  $y\in y$  Ground truth 
$\varphi (\cdot )$  multivariate function  ${L}_{1}$  Sejongsi 
$q$  multivariate function  ${L}_{2}$  Youngnamgun 
${L}_{3}$  Yansansi 
References
 McMahan, B.; Moore, E.; Ramage, D.; Hampson, S.; Arcas, B.A. Communicationefficient learning of deep networks from decentralized data. In Artificial Intelligence and Statistics; PMLR: Cambridge, MA, USA, 2017; pp. 1273–1282. [Google Scholar]
 Li, X.; Huang, K.; Yang, W.; Wang, S.; Zhang, Z. On the convergence of fedavg on noniid data. arXiv 2019, arXiv:1907.02189. [Google Scholar]
 Gadekallu, T.R. Federated Learning for Big Data: A Survey on Opportunities, Applications, and Future Directions. arXiv 2021, arXiv:2110.04160. [Google Scholar]
 Zhan, Y.; Zhang, J.; Hong, Z.; Wu, L.; Li, P.; Guo, S. A Survey of Incentive Mechanism Design for Federated Learning. IEEE Trans. Emerg. Top. Comput. 2021, 10, 1035–1044. [Google Scholar] [CrossRef]
 Karimireddy, S.P.; Kale, S.; Mohri, M.; Reddi, S.; Stich, S.; Suresh, A.T. Scaffold: Stochastic controlled averaging for federated learning. In International Conference on Machine Learning; PMLR: Cambridge, MA, USA, 2020; pp. 5132–5143. [Google Scholar]
 Korea Public Data Portal. Available online: https://www.data.go.kr/data/15003553/fileData.do (accessed on 21 December 2022).
 You, S. A Cybersecure Framework for Power Grids Based on Federated Learning. Eng. Arch. 2020. [Google Scholar] [CrossRef]
 Massaoudi, M.; AbuRub, H.; Refaat, S.S.; Chihi, I.; Oueslati, F.S. Deep Learning in Smart Grid Technology: A Review of Recent Advancements and Future Prospects. IEEE Access 2021, 9, 54558–54578. [Google Scholar] [CrossRef]
 Taik, A.; Nour, B.; Cherkaoui, S. Empowering Prosumer Communities in Smart Grid with Wireless Communications and Federated Edge Learning. IEEE Wirel. Commun. 2021, 28, 26–33. [Google Scholar] [CrossRef]
 Zhai, S.; Jin, X.; Wei, L.; Luo, H.; Cao, M. Dynamic Federated Learning for Gmec With TimeVarying Wireless Link. IEEE Access 2021, 9, 10400–10412. [Google Scholar] [CrossRef]
 Liu, H.; Zhang, X.; Shen, X.; Sun, H. A federated learning framework for smart grids: Securing power traces in collaborative learning. arXiv 2021, arXiv:2103.11870. [Google Scholar]
 Wen, M.; Xie, R.; Lu, K.; Wang, L.; Zhang, K. FedDetect: A Novel PrivacyPreserving Federated Learning Framework for Energy Theft Detection in Smart Grid. IEEE Internet Things J. 2021, 9, 6069–6080. [Google Scholar] [CrossRef]
 Zhao, L.; Li, J.; Li, Q.; Li, F. A Federated Learning Framework for Detecting False Data Injection Attacks in Solar Farms. IEEE Trans. Power Electron. 2021, 37, 2496–2501. [Google Scholar] [CrossRef]
 Li, D.; Luo, Z.; Cao, B. Blockchainbased federated learning methodologies in smart environments. Clust. Comput. 2021, 25, 2585–2599. [Google Scholar] [CrossRef]
 AlQuraan, M.; Khan, A.; Centeno, A.; Zoha, A.; Imran, M.A.; Mohjazi, L. FedTrees: A Novel ComputationCommunication Efficient Federated Learning Framework Investigated in Smart Grids. arXiv 2022, arXiv:2210.00060. [Google Scholar]
 Wang, Y.; Bennani, I.L.; Liu, X.; Sun, M.; Zhou, Y. Electricity consumer characteristics identification: A federated learning approach. IEEE Trans. Smart Grid 2021, 12, 3637–3647. [Google Scholar] [CrossRef]
 Zhao, Y.; Xiao, W.; Shuai, L.; Luo, J.; Yao, S.; Zhang, M. A Differential Privacyenhanced Federated Learning Method for ShortTerm Household Load Forecasting in Smart Grid. In Proceedings of the 2021 7th International Conference on Computer and Communications (ICCC), Chengdu, China, 10–13 December 2021; pp. 1399–1404. [Google Scholar] [CrossRef]
 Fernandez, J.D.; Menci, S.P.; Lee, C.; Fridgen, G. Secure Federated Learning for Residential Short Term Load Forecasting. arXiv 2021, arXiv:2111.09248. [Google Scholar]
 Zhou, X.; Feng, J.; Wang, J.; Pan, J. Privacypreserving household load forecasting based on nonintrusive load monitoring: A federated deep learning approach. PeerJ Comput. Sci. 2022, 8, e1049. [Google Scholar] [CrossRef] [PubMed]
 Savi, M.; Fabrizio, O. Shortterm energy consumption forecasting at the edge: A federated learning approach. IEEE Access 2021, 9, 95949–95969. [Google Scholar] [CrossRef]
 Taik, A.; Cherkaoui, S. Electrical Load Forecasting Using Edge Computing and Federated Learning. In Proceedings of the ICC 2020–2020 IEEE International Conference on Communications (ICC), Dublin, Ireland, 7–11 June 2020; pp. 1–6. [Google Scholar] [CrossRef]
 Zhang, P.; Wang, C.; Jiang, C.; Han, Z. Deep Reinforcement Learning Assisted Federated Learning Algorithm for Data Management of IIoT. IEEE Trans. Ind. Inform. 2021, 17, 8475–8484. [Google Scholar] [CrossRef]
 Saputra, Y.M.; Hoang, D.T.; Nguyen, D.N.; Dutkiewicz, E.; Mueck, M.D.; Srikanteswara, S. Energy Demand Prediction with Federated Learning for Electric Vehicle Networks. In Proceedings of the 2019 IEEE Global Communications Conference (GLOBECOM), Waikoloa, HI, USA, 9–13 December 2019; pp. 1–6. [Google Scholar]
 Huang, X.; Li, P.; Yu, R.; Wu, Y.; Xie, K.; Xie, S. Fedparking: A federated learning based parking space estimation with parked vehicle assisted edge computing. IEEE Trans. Veh. Technol. 2021, 70, 9355–9368. [Google Scholar] [CrossRef]
 Zhang, Y.; Tang, G.; Huang, Q.; Wang, Y.; Wu, K.; Yu, K.; Shao, X. FedNILM: Applying Federated Learning to nilm Applications at the Edge. In IEEE Transactions on Green Communications and Networking; IEEE: Piscataway Township, NJ, USA, 2022; p. 1. [Google Scholar] [CrossRef]
 Hudson, N.; Hossain, J.; Hosseinzadeh, M.; Khamfroush, H.; RahnamayNaeini, M.; Ghani, N. A Framework for Edge Intelligent Smart Distribution Grids via Federated Learning. In Proceedings of the 2021 International Conference on Computer Communications and Networks (ICCCN), Athens, Greece, 19–22 July 2021; pp. 1–9. [Google Scholar] [CrossRef]
 Nightingale, J.S.; Wang, Y.; Zobiri, F.; Mustafa, M.A. Effect of Clustering in Federated Learning on NonIID Electricity Consumption Prediction. In Proceedings of the 2022 IEEE PES Innovative Smart Grid Technologies Conference Europe (ISGTEurope), Novi Sad, Serbia, 10–12 October 2022; pp. 1–5. [Google Scholar] [CrossRef]
 Briggs, C.; Fan, Z.; Andras, P. Federated learning for shortterm residential energy demand forecasting. arXiv 2021, arXiv:2105.13325. [Google Scholar]
 Yan, G.; Hao, W.; Jian, L. Seizing Critical Learning Periods in Federated Learning. In Proceedings of the AAAI Conference on Artificial Intelligence; AAAI Press: Palo Alto, CA, USA, 2022. [Google Scholar]
 Lu, Y.; Huang, X.; Zhang, K.; Maharjan, S.; Zhang, Y. Blockchain Empowered Asynchronous Federated Learning for Secure Data Sharing in Internet of Vehicles. IEEE Trans. Veh. Technol. 2020, 69, 4298–4311. [Google Scholar] [CrossRef]
 Lim, W.Y.B.; Xiong, Z.; Miao, C.; Niyato, D.; Yang, Q.; Leung, C.; Poor, H.V. Hierarchical Incentive Mechanism Design for Federated Machine Learning in Mobile Networks. IEEE Internet Things J. 2020, 7, 9575–9588. [Google Scholar] [CrossRef]
 Lee, H.; Liu, Y.; Kim, D.; Li, Y. Robust Convergence in Federated Learning through Labelwise Clustering. arXiv 2021, arXiv:2112.14244. [Google Scholar]
 Zhou, T.; Konukoglu, E. FedFA: Federated Feature Augmentation. arXiv 2023, arXiv:2301.12995. [Google Scholar]
$$\widehat{\mathit{\delta}}\left({\mathit{L}}_{1},{\mathit{L}}_{2}\right)$$

$$\widehat{\mathit{\delta}}\left({\mathit{L}}_{1},{\mathit{L}}_{3}\right)$$

$$\widehat{\mathit{\delta}}\left({\mathit{L}}_{2},{\mathit{L}}_{3}\right)$$
 

2017  1.60 ± 0.85  1.55 ± 1.03  1.86 ± 0.90 
2018  1.66 ± 0.95  1.60 ± 0.99  1.86 ± 0.91 
2019  1.57 ± 0.88  1.60 ± 0.99  1.83 ± 0.91 
2020  1.71 ± 0.75  1.61 ± 1.00  1.97 ± 0.86 
August 2021  1.87 ± 0.87  1.64 ± 0.97  1.98 ± 0.86 
Overall  1.67 ± 0.87  1.60 ± 1.00  1.89 ± 0.89 
Experiment 1  
Local 1  Local 2  Local 3  
Train  Val  Train  Val  Train  Val  
LSTM  60,298.26  58,202.96  64,383.79  62,592.67  662,806.25  635,606.56 
GRU  60,169.04  58,801.27  64,059.05  64,715.78  640,914.86  641,708.94 
BiLSTM  59,735.69  58,777.66  64,374.08  65,708.52  646,646.01  651,371.92 
Experiment 2  
Local 1  Local 2  Local 3  
Train  Val  Train  Val  Train  Val  
LSTM  39.41  9.66  93.16  71.93  374.10  75.47 
GRU  47.73  36.67  30.86  27.75  443.16  310.01 
BiLSTM  41.74  58.56  35.78  12.73  56.57  59.75 
Experiment 3  
Local 1  Local 2  Local 3  
Train  Val  Train  Val  Train  Val  
LSTM  21.56  1.28  27.41  12.20  159.37  27.50 
GRU  29.86  19.76  32.15  6.80  204.22  30.82 
BiLSTM  22.43  1.81  14.13  8.16  119.58  16.97 
Local 1  Local 2  Local 3  

FedAvg (${L}_{1},{L}_{2},{L}_{3}$)  472,308.44  516,164.47  1,334,568.25 
FedAvg (${L}_{1},{L}_{2}$)  183,788.52  220,938.30  1,605,387.38 
FedAvg (${L}_{1},{L}_{3}$)  11,498.82  30,333.07  1,790,837.25 
FedAvg (${L}_{2},{L}_{3}$)  228,898.06  201,307.98  2,012,914.50 
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. 
© 2023 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Lee, H. Towards Convergence in Federated Learning via NonIID Analysis in a Distributed Solar Energy Grid. Electronics 2023, 12, 1580. https://doi.org/10.3390/electronics12071580
Lee H. Towards Convergence in Federated Learning via NonIID Analysis in a Distributed Solar Energy Grid. Electronics. 2023; 12(7):1580. https://doi.org/10.3390/electronics12071580
Chicago/Turabian StyleLee, Hyeongok. 2023. "Towards Convergence in Federated Learning via NonIID Analysis in a Distributed Solar Energy Grid" Electronics 12, no. 7: 1580. https://doi.org/10.3390/electronics12071580