# Adaptive Machine Learning for Automated Modeling of Residential Prosumer Agents

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

- The definition of the main criteria when choosing an adaptation algorithm in the context of prosumer agents. Here, the issues that the adaptation techniques address are examined to determine the best solution for models management, future concept assumptions, mixed drifts, and selection of training strategies.
- The identification of suitable algorithms for adapting the prosumer agents models to overcome environment changes. The algorithms are estimated with different adaptation strategies and forgetting mechanisms, such as Adaptive Windowing (ADWIN) [20], FISH [21], and Drift Detection Method (DDM) [22], in order to identify their required features in the prosumer agent context.
- Proposition of a new adaptation algorithm based on triggered adaptation techniques and performance-based forgetting mechanisms. The proposed method is non-iterative; thus, it is less computationally complex than other methods in the literature. The suggested approach is capable of training several appliances’ models in relatively fast data streams for the prosumer agent context, using a single data window.

## 2. Problem Statement

#### 2.1. Drift Magnitude

#### 2.2. Drift Duration

#### 2.3. Drift Subject

- Real concept drift: This type occurs when the posterior probability, $\mathcal{P}\left(y\right|X)$ changes over time and requires a retraining of the model. The change can occur in either a portion of the domain of X (sub-concept drift) or all of it (Full-concept drift).$${\mathcal{P}}_{t}\left(y\right|X)\ne {\mathcal{P}}_{t+m}\left(y\right|X).$$
- Virtual concept drift: It happens when, instead, it is the distribution of the features $\mathcal{P}\left(X\right)$, which changes over time while the posterior probability, $\mathcal{P}\left(y\right|X)$ remains the same. In that case, it is not always necessary to update the model.$${\mathcal{P}}_{t}\left(y\right|X)={\mathcal{P}}_{t+m}\left(y\right|X)\phantom{\rule{1.em}{0ex}}{\mathcal{P}}_{t}\left(X\right)\ne {\mathcal{P}}_{t+m}\left(X\right).$$

#### 2.4. Drift Predictability

## 3. Adaptive Algorithms

**The appearance of gradual drifts makes it impractical to assume that the concept of future data is always closer to the latest data.**Therefore, instead of assuming the concept of future data, it will be useful to implement an algorithm that recognizes the distribution of the features of arriving data. Besides, in some applications of prosumers, the robustness of the methods is essential to differentiate outliers from concept drifts. However, in this study, the agent trusts the external information he receives from measurement systems and weather information services.**In the data stream, there could be different kinds of drifts mixed and outliers data samples.**Another important concern with the concept drift in the specific context of the prosumer agent is the concurrence of the drift types. Thus the agent could be facing sudden drifts, gradual drifts, and incremental drifts within one timeframe [27]. For that reason, adaptation algorithms that were made to solve problems related to specific cases of drift are not the best option for the agent. Here, we test some of those algorithms to validate this affirmation.**It is impractical for a prosumer agent to have different models trained with different data sets and ensemble the forecasts.**Therefore, considering the available mechanisms to update the agents knowledge, when using a single model, the only strategy is to adapt the parameters. Nevertheless, in addition to parameters’ adaptation, it is also possible to combine the models by weighting them as in ensemble learning [28]. The choice of the method should be made by taking into account the residential agent’s restrictions related to the processing time and hardware limitation. Normally, the models of three main groups are used to provide information for other processing systems, such as a Home Energy Management Systems (HEMS), with a practical objective, like either minimizing energy cost or maximizing comfort. These systems usually provide results in five to fifteen minutes intervals, thus limiting convenient exploitation of ensemble learning.**Not all adaptation techniques may work for all the models.**The adaptation of parameters depends on the model type since some models can be trained incrementally, for example, by using adaptive linear neuron rule (ADALINE) or recursive least squares (RLS) [29], while others have to start from zero every time. The drawback of incremental learning is that outliers are directly included in the model’s knowledge. Notwithstanding, it can reduce memory usage and processing time. Besides, the choice of the learning strategy depends on the nature of data and the rate of data collection. Thus, the residential agent could be receiving and processing the data under both forms of single or batch measurements. For instance, the labels of the models could be given every time while the features could be queried in batches each several hours.

#### 3.1. Drift Detection Method

- Warning level: confidence level is 95%, so it is reached when ${\mathcal{P}}_{t}+{s}_{t}\ge {\mathcal{P}}_{min}+2{s}_{min}$.
- Drift detection: confidence level is 99%, so it is reached when ${\mathcal{P}}_{t}+{s}_{t}\ge {\mathcal{P}}_{min}+3{s}_{min}$.

Algorithm 1: Drift Detection method. |

#### 3.2. Gold Ratio Method

#### 3.3. Klinkenberg and Joachims’ Algorithm

Algorithm 2: Gold ratio method. |

Algorithm 3: Klinkenberg and Joachims’ Algorithm. |

#### 3.4. Fish Method

Algorithm 4: Fish method. |

#### 3.5. ADWIN

Algorithm 5: ADWIN algorithm. |

## 4. Proposed Algorithm

## 5. Numerical Results

**Power Generation:**The base learner is a feed-forward neural network that forecasts the power output of a generation system ${P}_{gen}$, as shown in Figure 4. The hidden layers of the network have 100 × 5 neurons; the activation function is a hyperbolic tangent, the step size is fixed at 0.0001, the initial state of weights is 1, and the method used to train the model is stochastic gradient descent. The data used in this case was synthetically created by simulating a photovoltaic array in PVlib [39] library with random cloud coverage (from 0 to 100% with transmittance offset of 0.75) and real temperature of Trois-Rivières, QC, in 2018.

**Fixed Load:**The model to forecast the fixed load consumption ${P}_{fix}$ is a decision tree where the quality of a split is measured using the mean squared error, and nodes are expanded until all leaves are pure [40]. The variables considered in this model are a cosine signal with a period of 24 h, the number of the day (from 1 to 7), the temperature, and the previous consumption (since the agent obtains data every 5 min, 288 samples correspond to 24 h). This model is presented in Figure 5. The data used in this case is real measurements of the power demand of a house in Trois-rivières, QC, during 2018.

**Controllable Load (Thermal model):**This model s based on the equivalent circuit 5R1C proposed in the standard ISO 13790:2008 [41]. The inputs in this case are the external temperature ${T}_{ext}$, the fixed load consumption ${P}_{fix}$, the solar irradiance ${P}_{irr}$, and the power demand of the heating system ${P}_{heat}$. The output will be the internal air temperature ${T}_{air}$. We assume that there is no special ventilation system, so there is only one external temperature as shown in Figure 6.

#### 5.1. Spring Day

^{2}/h is reached at 12 h of the first day. The heating system of the house has a capacity of 15 kW, and the internal temperature goes from 17.6 °C to 21.1 °C, with an average of 19 °C.

#### 5.2. Summer Day

^{2}/h; and the internal temperature goes from 24 °C to 28.3 °C, with an average of 25.8 °C.

#### 5.3. Results Analysis

## 6. Conclusions

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- U.S. EIA. Annual Energy Outlook 2019 with projections to 2050. Available online: https://www.eia.gov/aeo (accessed on 20 December 2019).
- GWAC. GridWise Transactive Energy Framework. GridWise Archit. Counc. Trans. Energy
**2013**, 1, 1–23. [Google Scholar] - Lian, J.; Zhang, W.; Sun, Y.; Marinovici, L.; Kalsi, K.; Widergren, S. Transactive System. Part I: Theoretical Underpinnings of Payoff Functions, Control Decisions, Information Privacy, and Solution Concepts; Technical Report December; Pacific Northwest National Laboratory: Richland, WA, USA, 2017.
- Yoon, J.H.; Baldick, R.; Novoselac, A. Dynamic demand response controller based on real-time retail price for residential buildings. IEEE Trans. Smart Grid
**2014**, 5, 121–129. [Google Scholar] [CrossRef] - Nagi, K. Transactional Agents: Towards a Robust Multi-Agent System; Springer Berlin Heidelberg: Karlsrule, Germany, 2001. [Google Scholar]
- Odell, J.; Giorgini, P.; Müller, J. Agent-Oriented Software Engineering V; Springer: Berlin/Heidelberg, Germany, 2004. [Google Scholar]
- Damisa, U.; Nwulu, N.I.; Sun, Y. Microgrid energy and reserve management incorporating prosumer behind-the-meter resources. IET Renew. Power Gener.
**2018**, 12, 910–919. [Google Scholar] [CrossRef] - Zhang, Y.; Huang, T.; Bompard, E.F. Big data analytics in smart grids: A review. Energy Inform.
**2018**, 1, 1–24. [Google Scholar] [CrossRef] - Khamphanchai, W.; Saha, A.; Rathinavel, K.; Kuzlu, M.; Pipattanasomporn, M.; Rahman, S.; Akyol, B.; Haack, J. Conceptual Architecture of Building Energy Management Open Source Software (BEMOSS). In Proceedings of the 2014 5th IEEE PES Innovative Smart Grid Technologies Europe (ISGT Europe), Istanbul, Turkey, 12–15 October 2014; pp. 1–6. [Google Scholar] [CrossRef]
- Bifet, A.; Gavaldà, R. Adaptive Learning from Evolving Data Streams. In Advances in Intelligent Data Analysis VIII; Springer: Berlin, Germany, 2009; pp. 249–260. [Google Scholar] [CrossRef]
- Loeffel, P.X. Adaptive Machine Learning Algorithms for Data Streams Subject to Concept Drifts. Ph.D. Thesis, Iniversité Pierre et Marie Curie, Paris, France, 2018. [Google Scholar]
- Zaknich, A. Principles of Adaptive Filters and Self-learning Systems; Springer: Murdoch, Australia, 2005. [Google Scholar]
- Balasubramanian, V.; Ho, S.S.; Vovk, V. Conformal Prediction for Reliable Machine Learning; Elsevier: Waltham, MA, USA, 2016. [Google Scholar] [CrossRef]
- Gama, J.; Zliobaité, I.; Bifet, A.; Pechenizkiy, M.; Bouchachia, A. A Survey on Concept Drift Adaptation. ACM Comput. Surv.
**2013**, 46. [Google Scholar] [CrossRef] - Al-Ali, A.; Zualkernan, I.A.; Rashid, M.; Gupta, R.; Alikarar, M. A smart home energy management system using IoT and big data analytics approach. IEEE Trans. Consum. Electron.
**2017**, 63, 426–434. [Google Scholar] [CrossRef] - Keshtkar, A.; Arzanpour, S. An adaptive fuzzy logic system for residential energy management in smart grid environments. Appl. Energy
**2017**, 186, 68–81. [Google Scholar] [CrossRef] - Farzan, F.; Jafari, M.A.; Gong, J.; Farzan, F.; Stryker, A. A multi-scale adaptive model of residential energy demand. Appl. Energy
**2015**, 150, 258–273. [Google Scholar] [CrossRef] - Yoo, J.; Park, B.; An, K.; Al-Ammar, E.A.; Khan, Y.; Hur, K.; Kim, J.H. Look-ahead energy management of a grid-connected residential PV system with energy storage under time-based rate programs. Energies
**2012**, 5, 1116–1134. [Google Scholar] [CrossRef] - Bouchachia, A.; Gabrys, B.; Sahel, Z. Overview of some incremental learning algorithms. In Proceedings of the IEEE International Conference on Fuzzy Systems, London, UK, 23–26 July 2007. [Google Scholar] [CrossRef]
- Bifet, A.; Gavaldà, R. Learning from Time-Changing Data with Adaptive Windowing. In Proceedings of the 2007 SIAM International Conference on Data Mining, Minneapolis, MN, USA, 26–28 April 2007; Society for Industrial and Applied Mathematics: Philadelphia, PA, USA, 2007; pp. 443–448. [Google Scholar] [CrossRef][Green Version]
- Zliobaite, I. Adaptive Training Set Formation. Ph.D. Thesis, Vilnius University, Vilnius, Lithuania, 2010. [Google Scholar]
- Gama, J.; Medas, P.; Castillo, G.; Rodrigues, P. Learning with Drift Detection. In Advances in Artificial Intelligence-SBIA 2004; Springer: Berlin, Germany, 2004; Volume 3171, pp. 286–295. [Google Scholar] [CrossRef]
- Murphy, K. Machine Learning: A Probabilistic Perspective; The MIT Press: Cambridge, MA, USA, 2012; pp. 27–71. [Google Scholar] [CrossRef]
- Moreno-Torres, J.G.; Raeder, T.; Alaiz-Rodríguez, R.; Chawla, N.V.; Herrera, F. A unifying view on dataset shift in classification. Pattern Recognit.
**2012**, 45, 521–530. [Google Scholar] [CrossRef] - Webb, G.I.; Hyde, R. Characterizing Concept Drift. Data Min. Knowl. Discov.
**2016**, 30, 964–994, arXiv:1511.03816v6. [Google Scholar] [CrossRef][Green Version] - Cieslak, D.A.; Hoens, T.R.; Chawla, N.V.; Kegelmeyer, W.P. Hellinger distance decision trees are robust and skew-insensitive. Data Min. Knowl. Discov.
**2012**, 24, 136–158. [Google Scholar] [CrossRef] - Widmer, G.; Kubat, M. Learning in the presence of concept drift and hidden contexts. Mach. Learn.
**1996**, 23, 69–101. [Google Scholar] [CrossRef][Green Version] - Zhou, Z.H. Ensemble Methods: Foundations and Algorithms; CRC Press: Boca Raton, FL, USA, 2012. [Google Scholar]
- Palnitkar, R.; Cannady, J. A Review of Adaptive Neural Networks. In Proceedings of the IEEE SoutheastCon, Greensboro, NC, USA, 26–29 March 2004; pp. 38–47. [Google Scholar] [CrossRef]
- Kuncheva, L.I.; Gunn, I.A. A concept-drift perspective on prototype selection and generation. In Proceedings of the International Joint Conference on Neural Networks, Vancouver, BC, Canada, 24–29 July 2016; pp. 16–23. [Google Scholar] [CrossRef][Green Version]
- Bifet, A.; Gavaldà, R.; Holmes, G.; Pfahringer, B. Machine Learning for Data Streams: With Practical Examples in MOA; The MIT Press: Cambridge, MA, USA, 2018. [Google Scholar]
- Koychev, I.; Lothian, R. Tracking Drifting Concepts by Time Window Optimisation. In Research and Development in Intelligent Systems XXII; Springer: London, UK, 2006; pp. 46–59. [Google Scholar] [CrossRef]
- Klinkenberg, R.; Joachims, T. Detecting Concept Drift with Support Vector Machines. In Proceedings of the Seventeenth International Conference on Machine Learning, Stanford, CA, USA, 29 June–2 July 2000; pp. 487–494. [Google Scholar]
- Aggarwal, C.C. Towards systematic design of distance functions for data mining applications. In Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining-KDD ’03, Washington, DC, USA, 24–27 August 2003; ACM Press: New York, NY, USA, 2003; p. 9. [Google Scholar] [CrossRef]
- Aggarwal, C.C.; Hinneburg, A.; Keim, D.A. On the Surprising Behavior of Distance Metrics in High Dimensional Space. Am. J. Dermatopathol.
**2001**, 30, 420–434. [Google Scholar] [CrossRef][Green Version] - Chai, T.; Draxler, R.R. Root mean square error (RMSE) or mean absolute error (MAE)? – Arguments against avoiding RMSE in the literature. Geosci. Model Dev.
**2014**, 7, 1247–1250. [Google Scholar] [CrossRef][Green Version] - Touzani, S.; Granderson, J.; Fernandes, S. Gradient boosting machine for modeling the energy consumption of commercial buildings. Energy Build.
**2018**, 158, 1533–1543. [Google Scholar] [CrossRef][Green Version] - Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Müller, A.; Nothman, J.; Louppe, G.; et al. Scikit-learn: Machine Learning in Python. Psychol. Sci.
**2012**, 25, 1682–1690. [Google Scholar] [CrossRef] - Holmgren, W.; Hansen, C.; Mikofski, M. pvlib python: A python package for modeling solar energy systems. J. Open Source Softw.
**2018**, 3, 884. [Google Scholar] [CrossRef][Green Version] - Alpaydin, E. Introduction to Machine Learning Second Edition, 3rd ed.; The MIT Press: Cambridge, MA, USA, 2010; pp. 350–380. [Google Scholar] [CrossRef][Green Version]
- Jokisalo, J.; Kurnitski, J. Performance of EN ISO 13790 utilisation factor heat demand calculation method in a cold climate. Energy Build.
**2007**, 39, 236–247. [Google Scholar] [CrossRef] - Gelman, A.; Carlin, J.; Stern, H.; Rubin, D. Bayesian Data Analysis; Chapman & Hall: London, UK, 2004. [Google Scholar]
- Chong, E.; Zak, S. An Introduction to Optimization, 4th ed.; Wiley and Sons: Hoboken, NJ, USA, 2013. [Google Scholar]

Power Generation [kW] | Fixed Load [kW] | Thermal Load [C] | |
---|---|---|---|

Full memory (8064 samples) | 0.68479 (20.89%) | 2.22717 (35.71%) | 0.65367 (21.57%) |

Drift Detection Method | 0.70753 (21.65%) | 2.36283 (37.90%) | 0.74063 (24.37%) |

Gold Ratio Method | 0.68168 (20.77%) | 2.08994 (33.47%) | 0.7432 (24.45%) |

Klinkenberg Method | 0.70976 (21.64%) | 1.99740 (32.02%) | 0.653353 (22.01%) |

FISH Method | 0.69418 (21.15%) | 2.22050 (35.63%) | 0.64908 (21.59%) |

Adwin Method | 0.69573 (21.19%) | 1.89000 (30.31%) | 0.70723 (23.33%) |

Proposed Method | 0.67787 (20.67%) | 1.97518 (31.68%) | 0.62385 (20.52%) |

Power Generation [kW] | Fixed Load [kW] | Thermal Load [C] | |
---|---|---|---|

Full memory (8064 samples) | 0.44479 (17.32%) | 2.80095 (34.55%) | 0.47161 (19.82%) |

Drift Detection Method | 0.44250 (17.24%) | 2.55943 (31.58%) | 0.44122 (18.775%) |

Gold Ratio Method | 0.45597 (17.80%) | 2.43693 (30.04%) | 0.47094 (19.79%) |

Klinkenberg Method | 0.53091 (20.64%) | 2.39652 (29.55%) | 0.44375 (19.04%) |

FISH Method | 0.46619 (18.17%) | 2.95768 (36.87%) | 0.79645 (32.82%) |

Adwin Method | 0.46240 (18.05%) | 2.69773 (33.26%) | 0.43769 (18.81%) |

Proposed Method | 0.43797 (17.06%) | 2.71268 (33.52%) | 0.4423 (18.68%) |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Toquica, D.; Agbossou, K.; Malhamé, R.; Henao, N.; Kelouwani, S.; Cardenas, A. Adaptive Machine Learning for Automated Modeling of Residential Prosumer Agents. *Energies* **2020**, *13*, 2250.
https://doi.org/10.3390/en13092250

**AMA Style**

Toquica D, Agbossou K, Malhamé R, Henao N, Kelouwani S, Cardenas A. Adaptive Machine Learning for Automated Modeling of Residential Prosumer Agents. *Energies*. 2020; 13(9):2250.
https://doi.org/10.3390/en13092250

**Chicago/Turabian Style**

Toquica, David, Kodjo Agbossou, Roland Malhamé, Nilson Henao, Sousso Kelouwani, and Alben Cardenas. 2020. "Adaptive Machine Learning for Automated Modeling of Residential Prosumer Agents" *Energies* 13, no. 9: 2250.
https://doi.org/10.3390/en13092250