Adaptive Energy Management of Big Data Analytics in Smart Grids
Abstract
:1. Introduction
2. Literature Review
2.1. Energy Management
2.2. Technologies Used in Big Data Implementation
3. Big Data Issues Addressing Adaptive Energy Management
4. Data Mining and Machine Learning Techniques of Big Data in Smart Grid
5. Prediction Analysis and Implementation of Techniques
5.1. Inputs
5.2. Outputs
5.3. Machine Learning Models
5.3.1. Linear Regression
5.3.2. Logistic Regression
Link Function | Logit |
Rows used | 60,000 |
Variable | Value | Count | |
stabf | stable | 21,720 | (Event) |
unstable | 38,280 | ||
Total | 60,000 |
Term | Coef | SE Coef | VIF |
Constant | −0.072 | 0.109 | |
stab | −7546 | 302 | 1.00 |
Odds Ratio | 95% CI | |
stab | 0.0000 | (0.0000, 0.0000) |
Deviance R-Sq | Deviance R-Sq (adj) | AIC | AICc | BIC |
99.77% | 99.77% | 186.17 | 186.17 | 204.17 |
Source | DF | Adj Dev | Adj Mean | Chi-Square | p-Value |
Regression | 1 | 78,365.1 | 78,365.1 | 78,365.06 | 0.000 |
stab | 1 | 78,365.1 | 78,365.1 | 78,365.06 | 0.000 |
Error | 59,998 | 182.2 | 0.0 | ||
Total | 59,999 | 78,547.2 |
5.3.3. K-Nearest Neighbors (KNN)
K | Count | RSquare | Misclassification Rate | Misclassifications |
1 | 60,000 | 0.72146 | 0.00000 | 0 * |
2 | 60,000 | 0.83904 | 0.00000 | 0 |
3 | 60,000 | 0.88678 | 0.00000 | 0 |
4 | 60,000 | 0.91268 | 0.00000 | 0 |
5 | 60,000 | 0.92893 | 0.00000 | 0 |
6 | 60,000 | 0.94003 | 0.00000 | 0 |
7 | 60,000 | 0.94811 | 0.00000 | 0 |
8 | 60,000 | 0.95425 | 0.00000 | 0 |
9 | 60,000 | 0.95908 | 0.00000 | 0 |
10 | 60,000 | 0.96298 | 0.00012 | 7 |
Confusion Matrix for Best K = 1. * based on conditions. |
Actual | Predicted Count | |
stabf | stable | unstable |
stable | 21720 | 0 |
unstable | 0 | 38280 |
Actual | Predicted Rate | |
stabf | stable | unstable |
stable | 1.000 | 0.000 |
unstable | 0.000 | 1.000 |
True Label | Predicted Label | Linear Regression | Logistic Regression | K-Nearest Neighbors |
Positive | Positive | True Positive (TP) | True Positive (TP) | True Positive (TP) |
Positive | Negative | True Positive (TP) | False Negative (FN) | False Negative (FN) |
Negative | Positive | True Negative (TN) | True Negative (TN) | False Positive (FP) |
Negative | Negative | True Negative (TN) | True Negative (TN) | True Negative (TN) |
6. Results and Discussion
7. Conclusions and Future Works
- (i)
- Customers should be directly involved in grid activities, like entering their own data, and for doing this, special concessions in billing should be given. This will help in maintaining the big data of the grid, and cost effectiveness can be achieved.
- (ii)
- New regulations should be implemented. With the help of IoT and other BDA tools, the customers should be given incentives to use less energy during peak hours.
- (iii)
- The collection of data in real time needs to be accomplished in order to utilize the full capability of BDA.
- (iv)
- Renewable energy source integration needs to be achieved with real-time data for efficient usage.
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Zhang, W.; Liu, S.; Xia, Z. A distributed privacy-preserving data aggregation scheme for smart grid with fine-grained access control. J. Inf. Secur. Appl. 2022, 66, 103118. [Google Scholar] [CrossRef]
- Chang, Y.-C.; Huang, T.-C. An interactive smart grid communication approach for big data traffic. Comput. Electr. Eng. 2018, 67, 170–181. [Google Scholar] [CrossRef]
- Diamantoulakis, P.D.; Kapinas, V.M.; Karagiannidis, G.K. Big Data Analytics for Dynamic Energy Management in Smart Grids. Big Data Res. 2015, 2, 94–101. [Google Scholar] [CrossRef]
- Li, J.; Herdem, M.S.; Nathwani, J.; Wen, J.Z. Methods and applications for Artificial Intelligence, Big Data, Internet of Things, and Blockchain in smart energy management. Energy AI 2023, 11, 100208. [Google Scholar] [CrossRef]
- Hasan, M.K.; Habib, A.A.; Shukur, Z.; Ibrahim, F.; Islam, S.; Razzaque, A. Review on cyber-physical and cyber-security system in smart grid: Standards, protocols, constraints, and recommendations. J. Netw. Comput. Appl. 2023, 209, 103540. [Google Scholar] [CrossRef]
- Mohajeri, M.; Ghassemi, A.; Gulliver, T.A. Fast Big Data Analytics for Smart Meter Data. IEEE Open J. Commun. Soc. 2020, 1, 1864–1871. [Google Scholar] [CrossRef]
- Wen, L.; Zhou, K.; Yang, S.; Li, L. Compression of smart meter big data: A survey. Renew. Sustain. Energy Rev. 2018, 91, 59–69. [Google Scholar] [CrossRef]
- Biabani, M.; Golkar, M.A.; Kasiry, A.H.Z.; Akbari, M. Smart Grid in Iran: Driving Factors, Evolution, Challenges and Possible Solutions. In Proceedings of the 2011 10th International Conference on Environment and Electrical Engineering, EEEIC.EU 2011—Conference Proceedings, Rome, Italy, 8–11 May 2011. [Google Scholar] [CrossRef]
- Archana; Shankar, R.; Singh, S. Development of smart grid for the power sector in India. Clean. Energy Syst. 2022, 2, 100011. [Google Scholar] [CrossRef]
- Ouafiq, E.M.; Raif, M.; Chehri, A.; Saadane, R. Data Architecture and Big Data Analytics in Smart Cities. Procedia Comput. Sci. 2022, 207, 4123–4131. [Google Scholar] [CrossRef]
- Dehalwar, V.; Kolhe, M.L.; Deoli, S.; Jhariya, M.K. Blockchain-based trust management and authentication of devices in smart grid. Clean Eng. Technol. 2022, 8, 100481. [Google Scholar] [CrossRef]
- Xue, X.; Tsai, P.-W. Integrating Energy Smart Grid’s ontologies through multi-objective particle swarm optimization algorithm with competitive mechanism. Sustain. Energy Technol. Assess. 2022, 53, 102442. [Google Scholar] [CrossRef]
- Bolurian, A.; Akbari, H.; Mousavi, S.; Aslinezhad, M. Bi-level energy management model for the smart grid considering customer behavior in the wireless sensor network platform. Sustain. Cities Soc. 2023, 88, 104281. [Google Scholar] [CrossRef]
- Haq, E.U.; Pei, C.; Zhang, R.; Jianjun, H.; Ahmad, F. Electricity-theft detection for smart grid security using smart meter data: A deep-CNN based approach. Energy Rep. 2023, 9, 634–643. [Google Scholar] [CrossRef]
- Khan, A.A.; Laghari, A.A.; Rashid, M.; Li, H.; Javed, A.R.; Gadekallu, T.R. Artificial intelligence and blockchain technology for secure smart grid and power distribution Automation: A State-of-the-Art Review. Sustain. Energy Technol. Assess. 2023, 57, 103282. [Google Scholar] [CrossRef]
- Kamil, I.A.; Ogundoyin, S.O. A big data anonymous batch verification scheme with conditional privacy preservation for power injection over vehicular network and 5G smart grid slice. Sustain. Energy Grids Netw. 2019, 20, 100260. [Google Scholar] [CrossRef]
- Alavikia, Z.; Shabro, M. A comprehensive layered approach for implementing internet of things-enabled smart grid: A survey. Digit. Commun. Netw. 2022, 8, 388–410. [Google Scholar] [CrossRef]
- Raghavendra, S.; Neelakandan, S.; Prakash, M.; Geetha, B.; Asha, S.M.R.; Roberts, M.K. Artificial humming bird with data science enabled stability prediction model for smart grids. Sustain. Comput. Inform. Syst. 2022, 36, 100821. [Google Scholar] [CrossRef]
- Sivarajan, S.; Jebaseelan, S.S. Efficient adaptive deep neural network model for securing demand side management in IoT enabled smart grid. Renew. Energy Focus 2022, 42, 277–284. [Google Scholar] [CrossRef]
- Zhang, X.; Huang, C.; Gu, D.; Zhang, J.; Xue, J.; Wang, H. Privacy-preserving statistical analysis over multi-dimensional aggregated data in edge computing-based smart grid systems. J. Syst. Archit. 2022, 127, 102508. [Google Scholar] [CrossRef]
- Mostafa, N.; Ramadan, H.S.M.; Elfarouk, O. Renewable energy management in smart grids by using big data analytics and machine learning. Mach. Learn. Appl. 2022, 9, 100363. [Google Scholar] [CrossRef]
- Akkad, A.; Wills, G.; Rezazadeh, A. An information security model for an IoT-enabled Smart Grid in the Saudi energy sector. Comput. Electr. Eng. 2023, 105, 108491. [Google Scholar] [CrossRef]
- Alomar, M.A. An IOT based smart grid system for advanced cooperative transmission and communication. Phys. Commun. 2023, 58, 102069. [Google Scholar] [CrossRef]
- Rouzbahani, H.M.; Karimipour, H.; Lei, L. Multi-layer defense algorithm against deep reinforcement learning-based intruders in smart grids. Int. J. Electr. Power Energy Syst. 2023, 146, 108798. [Google Scholar] [CrossRef]
- Oad, A.; Ahmad, H.G.; Talpur, M.S.H.; Zhao, C.; Pervez, A. Green smart grid predictive analysis to integrate sustainable energy of emerging V2G in smart city technologies. Optik 2023, 272, 170146. [Google Scholar] [CrossRef]
- Panthi, M.; Das, T.K. Intelligent Intrusion Detection Scheme for Smart Power-Grid Using Optimized Ensemble Learning on Selected Features. Int. J. Crit. Infrastruct. Prot. 2022, 39, 100567. [Google Scholar] [CrossRef]
- Ghiasi, M.; Niknam, T.; Wang, Z.; Mehrandezh, M.; Dehghani, M.; Ghadimi, N. A comprehensive review of cyber-attacks and defense mechanisms for improving security in smart grid energy systems: Past, present and future. Electr. Power Syst. Res. 2023, 215, 108975. [Google Scholar] [CrossRef]
- Bitirgen, K.; Filik, B. A hybrid deep learning model for discrimination of physical disturbance and cyber-attack detection in smart grid. Int. J. Crit. Infrastruct. Prot. 2023, 40, 100582. [Google Scholar] [CrossRef]
- Kalfarisi, R.; Chew, A.; Cai, J.; Xue, M.; Pok, J.; Wu, Z.Y. Predictive modeling framework accelerated by GPU computing for smart water grid data-driven analysis in near real-time. Adv. Eng. Softw. 2022, 173, 103287. [Google Scholar] [CrossRef]
- Qays, O.; Ahmad, I.; Abu-Siada, A.; Hossain, L.; Yasmin, F. Key communication technologies, applications, protocols and future guides for IoT-assisted smart grid systems: A review. Energy Rep. 2023, 9, 2440–2452. [Google Scholar] [CrossRef]
- Rouzbahani, H.M.; Karimipour, H.; Lei, L. Optimizing scheduling policy in smart grids using probabilistic Delayed Double Deep Q-Learning (P3DQL) algorithm. Sustain. Energy Technol. Assess. 2022, 53, 102712. [Google Scholar] [CrossRef]
- Kabalci, E.; Kabalci, Y. Big Data, Privacy and Security in Smart Grids. In From Smart Grid to Internet of Energy; Elsevier: Amsterdam, The Netherlands, 2019; pp. 309–333. [Google Scholar] [CrossRef]
- Guan, Z.; Si, G. Achieving privacy-preserving big data aggregation with fault tolerance in smart grid. Digit. Commun. Netw. 2017, 3, 242–249. [Google Scholar] [CrossRef]
- Mortaheb, R.; Jankowski, P. Smart city re-imagined: City planning and GeoAI in the age of big data. J. Urban Manag. 2023, 12, 4–15. [Google Scholar] [CrossRef]
- Bai, Y.; Chou, L.; Zhang, W. Industrial innovation characteristics and spatial differentiation of smart grid technology in China based on patent mining. J. Energy Storage 2021, 43, 103289. [Google Scholar] [CrossRef]
- Meegahapola, L.G.; Bu, S.; Wadduwage, D.P.; Chung, C.Y.; Yu, X. Review on Oscillatory Stability in Power Grids with Renewable Energy Sources: Monitoring, Analysis, and Control Using Synchrophasor Technology. IEEE Trans. Ind. Electron. 2021, 68, 519–531. [Google Scholar] [CrossRef]
- Si, C.; Xu, S.; Wan, C.; Chen, D.; Cui, W.; Zhao, J. Electric Load Clustering in Smart Grid: Methodologies, Applications, and Future Trends. J. Mod. Power Syst. Clean Energy 2021, 9, 237–252. [Google Scholar] [CrossRef]
- Zidi, S.; Mihoub, A.; Qaisar, S.M.; Krichen, M.; Abu Al-Haija, Q. Theft detection dataset for benchmarking and machine learning based classification in a smart grid environment. J. King Saud Univ.—Comput. Inf. Sci. 2023, 35, 13–25. [Google Scholar] [CrossRef]
- Baskar, S.; Dhote, S.; Dhote, T.; Akila, D.; Arunprathap, S. Surge detection for smart grid power distribution using a regression-based signal processing model. Comput. Electr. Eng. 2022, 104, 108424. [Google Scholar] [CrossRef]
- Kotsiopoulos, T.; Sarigiannidis, P.; Ioannidis, D.; Tzovaras, D. Machine Learning and Deep Learning in smart manufacturing: The Smart Grid paradigm. Comput. Sci. Rev. 2021, 40, 100341. [Google Scholar] [CrossRef]
- Alemazkoor, N.; Tootkaboni, M.; Nateghi, R.; Louhghalam, A. Smart-Meter Big Data for Load Forecasting: An Alternative Approach to Clustering. IEEE Access 2022, 10, 8377–8387. [Google Scholar] [CrossRef]
- Al-Badi, A.; Khan, A.I. A Sustainable Development Neural Network Model for Big Data in Smart Cities. Procedia Comput. Sci. 2022, 202, 408–413. [Google Scholar] [CrossRef]
- Ahmad, T.; Madonski, R.; Zhang, D.; Huang, C.; Mujeeb, A. Data-driven probabilistic machine learning in sustainable smart energy/smart energy systems: Key developments, challenges, and future research opportunities in the context of smart grid paradigm. Renew. Sustain. Energy Rev. 2022, 160, 112128. [Google Scholar] [CrossRef]
- Rashid, M.; Ahmad, A.J.; Prashar, D. Integration of IoT with Big Data Analytics for the Development of Smart Society. In Artificial Intelligence and Machine Learning in Smart City Planning; Elsevier: Amsterdam, The Netherlands, 2023; pp. 13–27. [Google Scholar] [CrossRef]
S. No. | Challenges | Solution to Challenges |
---|---|---|
1 | Data volume and velocity | The solution to this challenge can be reached with the following measures: (a) use cloud computing and distributed storage, (b) use real-time analytics, (c) use data compression techniques, (d) use data sampling techniques, and (e) use parallel processing techniques. |
2 | Data quality | The solution to this challenge can be reached with the following measures: (a) use a data quality management tool, (b) involve stakeholders in the data quality process, and (c) continuously monitor data quality. |
3 | Privacy and security [33] | The solution to this challenge can be reached with the following measures: (a) use a privacy and security management tool, (b) involve stakeholders in the privacy and security process, and (c) continuously monitor privacy and security. |
4 | Cost | The solution to this challenge can be reached with the following measures: (a) careful planning of the project, (b) working with expertise, and (c) continuously monitoring costs. |
5 | Skills | For dealing with this challenge, some skills in relevant fields are needed: electrical engineering, power systems, information security, data visualization, and business intelligence. |
Stab 1 | Measures | Value | Measures | Value | Stab 2 |
RSquare | 0.0791332 | RSquare | 0.0792677 | ||
RASE | 2.6365864 | RASE | 2.6325246 | ||
Mean Abs Dev | 2.259736 | Mean Abs Dev | 2.2565207 | ||
−LogLikelihood | 95,541.72 | −LogLikelihood | 95,480.047 | ||
SSE | 278,077.42 | SSE | 277,221.29 | ||
Sum Freq | 40,002 | Sum Freq | 40,002 | ||
Stab 3 | Measures | Value | Measures | Value | Stab 4 |
RSquare | 0.0784685 | RSquare | 0.0817522 | ||
RASE | 2.6332954 | RASE | 2.6328099 | ||
Mean Abs Dev | 2.2575801 | Mean Abs Dev | 2.2572644 | ||
−LogLikelihood | 95,491.758 | −LogLikelihood | 95,484.382 | ||
SSE | 277,383.65 | SSE | 277,281.38 | ||
Sum Freq | 40,002 | Sum Freq | 40,002 |
Model | Stability Analysis | Explanation |
---|---|---|
Linear Regression | Moderate stability | Linear regression is sensitive to data ambiguities, which can lead to large changes in model parameters. However, if the distribution of the data remains relatively constant, the predictions of the model are stable. |
Logistic Regression | Moderate stability | Logistic regression has a similar sensitivity as linear regression, which affects the coefficients of the model. With a stable data set, logistic regression provides consistent probabilities and predictions. |
K-Nearest Neighbors | Low stability | Since the KNN records the training instances directly, the performance of the KNN is highly dependent on the training data. Small changes in the data and other additions/deletions of data points can significantly change the decision limits. This makes KNNs less stable compared to regression models for dynamic or noisy data sets. |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Gupta, R.; Chaturvedi, K.T. Adaptive Energy Management of Big Data Analytics in Smart Grids. Energies 2023, 16, 6016. https://doi.org/10.3390/en16166016
Gupta R, Chaturvedi KT. Adaptive Energy Management of Big Data Analytics in Smart Grids. Energies. 2023; 16(16):6016. https://doi.org/10.3390/en16166016
Chicago/Turabian StyleGupta, Rohit, and Krishna Teerth Chaturvedi. 2023. "Adaptive Energy Management of Big Data Analytics in Smart Grids" Energies 16, no. 16: 6016. https://doi.org/10.3390/en16166016
APA StyleGupta, R., & Chaturvedi, K. T. (2023). Adaptive Energy Management of Big Data Analytics in Smart Grids. Energies, 16(16), 6016. https://doi.org/10.3390/en16166016