REAL-TIME FRAMEWORK FOR ENERGY MANAGEMENT SYSTEM OF A SMART MICROGRID USING MULTIAGENT SYSTEMS

This paper deals with the problem of real-time management of Smart Grids. For this sake, 1 the energy management is integrated with the power system through a telecommunication system. 2 The use of Multiagent Systems leads the proposed algorithm to find the best-integrated solution, 3 taking into consideration the operating scenario and the system characteristics. The proposed 4 technique is tested with the help of an academic microgrid, so the results may be replicated. 5


Introduction
Electrical power systems in the world are going through a modernization period with drastic improvements.This raises environmental consequences and concerns about the sustainability and resilience of these systems.Particularly, the penetration of renewable sources and communication features increases the complexity of these systems, known as active systems.Microgrids fit in active systems and depending on the level of communication and control structures, may become smart grids.A microgrid is a network in small scale to supply small community loads [1].At [2], it is possible to define a microgrid as a small power system, able to balance its generation and demand, in order to maintain stable power supply inside a defined area, as shown in Figure 1.
Microgrids offer the possibility of integration of different resources, so the operation may help one to supply critical loads under emergency conditions.The main objectives of this integration are the increase in efficiency, reliability, and reduction of losses and greenhouse gases emission, among others [3].One of the strategies and technologies proposed to achieve these objectives is the concept of Sustainable Microgrids.A microgrid must be capable of properly handling two basic operation states: normal (grid connected) and emergency (islanded).To achieve an islanded operation capacity for a long period, a microgrid must fulfill requirements of energy storage and continuous microgeneration for supplying all loads, or in critical operation cases, supplying the highest priority loads [4].
To efficiently operate and thus make the power grid more reliable, increasing availability and safety, and reducing environmental impact, a smart telecommunication system [5] is required.It has the function of integrating all microgrid components and enables the insertion of dispatchable distributed generation (DG), energy trading, integrated system protection, power quality management and real-time microgrid monitoring [1].Some data protocols like IEC 61850, IEEE C37.118 and IEEE 2030.5 were being developed to help and provide support for all of these functions.
However, the monitoring, management, and operation, even for a small network, can be a complex task [1].Then, with the use of a Multiagent System (MAS) it is possible to split this complex problem into several small and simple ones [6].Some recent works such as [7][8][9] present solutions to partially handle this problem, but none of them evaluate the complete integration of the available resources.
The concern of the response time of a simulation could be seen in [10] by using PSCAD\NS2 (Network Simulator 2), in [11] by using PSFL\NS2 [12][13][14][15] it is possible to see an effort and concern for this integration or co-simulation and thus validate the results in the new context of the intelligent network.You can still find other researchers of multiagent system applied to microgrid [16][17][18][19][20][21][22] where the telecommunications system was not included.As it is responsible for obtaining the data in the Intelligent Eletronic Device (IED) and delivering it to the Information Technology (IT) system, include it in the simulation have a great impact on the evaluation of the solution.
Another important aspect that should be noted is how MAS were developed to solve problems in a microgrid.Some researchers such as [23][24][25][26][27][28][29][30] have proposed frameworks and guidelines for the design, modeling and development of MAS with a correct view on how they should act, interact, socialize and pursue the goal, which is the power management solution of their power system.It is not intended to classify solutions in the microgrids as in [31][32][33] and for MAS as presented at [34][35][36].
In this new context of Smart Grids, solutions that can be used for the problems that are about to arise are necessary.Thus, in visualizing the whole and not only the energy management algorithm, understanding that the time spent is important, what information is needed to reach the goal, contribute to the data communication protocols being developed and evaluate if methodologies have the ability to respond to situations within the time required.
This work propose a framework with these three system integrated to allow a better analysis in real time of the impact of modeling a MAS using two different triggers and see the time impact of this modeling.Further, analyzing all systems, we verify the impact of a good data protocol development, so the information necessary to define the expected behavior of the agents could be sent to their recipients.As described in [37], there are some control methodologies for microgrid during islanded operation, like voltage and frequency management, power quality, droop control and balance between supply and demand.The focus of this paper is in the last methodology.
This paper is then organized as follows: Section 2 presents the Multiagent System; Section 3 presents the real time framework that will be used in Section 4, that shows the results of real time simulation; finally Section 5 presents the conclusions.

Multiagent System
Agents and Multiagent Systems (MAS) enable a new way to analyze, design and implement complex control systems.A MAS can be defined as a loosely coupled network of problem solvers working together to solve problems that are beyond their individual capacities or knowledge to solve a problem [6].Thus, a multiagent system should be able to operate without direct human intervention or of other agents and should have control of their own behavior and internal state.So, it must have a flexible autonomous action, showing the following characteristics:

•
Responsive: agents must perceive their environment and respond in a timely manner to the dynamic changes that occur in it; • Proactive: agents must not only act in response to the environment, they are able to exhibit goal-directed behavior by taking the initiative in order to satisfy their design objectives.

•
Social: agents should be able to interact, when appropriate, with other agents or humans looking to complete their tasks, and assist others with their activities [6,38].
The reasons for choosing MAS are: the ability to provide robustness and efficiency; ability to allow interoperation with existing legacy systems and ability to solve problems where data, knowledge or control are distributed [6].
Therefore, the use of a MAS is an option for Smart Grids as in Smart Homes applications [39], in microgrids [17,27], either for control of Distributed Generation (DG) [40], energy management [41][42][43] or microgrid control [16,24,[44][45][46].The development of MAS can benefit the intelligent operation of electrical power systems in many ways, but there are still some challenges that need to be addressed as problem formulation, standardization, communication, coherency, robustness, implementation [34].
At [26] it is described that the current trend of the use of MAS in Power Systems can be divided into two modes: one as a methodology for developing flexible and extensible computer/hardware systems; and the other as a problem modeling methodology.The flexibility denotes the capacity to correctly respond to dynamic situations and support for replications in different situations (environments).The extensibility denotes the capacity to easily add a new functionality to a system, enhancing or updating any type of functionality.As a modeling methodology, Multi-agent systems are more than a system integration method, by offering a different way to see the world.An agent system can intuitively be used to represent a real situation of interacting entities and give a way to test how complex behaviors can arise.

RealTime Framework
To perform the tests, and to execute them in real time, the RTDS R was used for the simulation of the microgrid.This is because it has the capacity to send/receive information to/from the system modeled in real time.This is an important factor and a contribution of this work since despite simulations of electrical systems integrated to computational systems with bi-directional information flows have already been studied in [17,47,48], the response time was not obtained in the execution time, which caused a certain delay.Some solutions were implemented for the time synchronization between the simulated electrical systems, the communication system, and the data processing system, as seen in [11,[49][50][51], but the type of data sent from the power system simulator is not analyzed.So a framework was developed using the RTDS system coupled to its simulation and monitoring software of the electrical system, the RSCAD, installed in one PC, which was connected by a 100 Mbps switch to the second PC, where the Multiagent System was running.The topology can be seen in Figure 2. The traffic of data on the switch was very small and did not compromise the performance in question because the used equipment works in wire-rate i.e., it is able to deliver the maximum bandwidth on each port without having packet loss.

Microgrid Simulation
The proposed power system model for this test case simulation is composed of one wind power generator, one Photovoltaic (PV) generator, and one dispatchable diesel generator, as presented in Table 1.There are also 7 loads described in Table 2.The priority of the loads can be defined according to specific criteria for each particular microgrid.
The renewable generators have a wind speed and solar irradiance profiles as shown in Figure 3.The data regarding these typical profiles are based on the Belgium electricity transmission system operator (ELIA) website.Any other test case could be used in this study, as well as a more complex case.This simple case was selected in this paper in order to prove the concepts and enable one to reproduce the results.

Multiagent System Design
The test case model presented here is useful to demonstrate the multiagent system interacting with the electrical power system in the islanded mode, pursuing the goal to supply all the loads as long as possible following the goal described at Equation (3) where the Source is defined by (1) and the Load is defined by (2).The load of group 1 belongs to the Emergency state, loads of group 2 to the Critical state and the loads of group 3 to the survival.This modeling platform also shows the potentials of this type of evaluation in real time in integrating the three systems.Weaknesses and unforeseen constraints can then be directly verified as in the real time frame.
P Loads = ∑ P Priority 1 + ∑ P Priority 2 + ∑ P Priority 3 (2) For the implementation of the multiagent system, there are several platforms or development frameworks such as Zeus, Cybele, Cougar and Net.In this work, we chose the platform JADE (Java Agent Development) [52].JADE is widely used because it is a middleware that abstracts the distributed architecture required for the creation of MAS, thus allowing that the development can be focused on the logical aspects of the system because it controls communication between different hosts and messages exchanged between agents [53].
All the communications between the agents are made using FIPA (Foundation for Intelligent Physical Agents) protocol, implemented in JADE, and the communication between the agents and the power system in the RTDS (Real Time Digital Simulator) is done using TCP/IP protocol.For this simulation, the following agents are proposed: The development of the MAS was based on the Gaia [54] methodology, that helped and guided with the definitions of the roles that will be exercised, of the agents that will participate and execute the roles, and of the services that each agent will have to perform to reach the desired goal.The following roles were defined: Equipment, Manager, and Translator, where the Equipment represents the agents responsible for the management of the IEDs, whether they are coupled to the loads or to the sources of power generation.The Manager is responsible for verifying the current situation of the microgrid and define a state (emergency, critical or survival) and the Translator makes the connection between the electrical or microgrid system and the MAS.
The agent models are based on the roles to be performed at the environment in which they are immersed.Thus, four agents are defined: LOAD (load) responsible for monitoring the loads of the microgrid and has "Equipment" as the role to be prosecuted; Source (generation) responsible for the equipment of generation of energy and that has the same role as the Load; Middle (translator) responsible for the exchange of information and translation of commands between the microgrid and the MAS by performing the role "Translator"; and the Manager, responsible for collecting the information of those who perform the role "Equipment" for the definition of a priority status when performing the role "Manager".
By performing the tests on a real-time electrical system simulation platform and accessing the data through a SCADA (Supervisory Control and Data Acquisition) monitoring interface, it is possible to determine which data will actually become available to send to the management software system, which, in this case, is the MAS.Much of the information used in other simulations using an electrical system and MAS make use of information that is not necessarily available by the Intelligent Electronic Device (IED).

Simulation and Results
The interaction between the MAS and the power system has to be executed in real time.The microgrid described in Figure 4 is used to test the interaction between the proposed methodology and the Real Time Digital Simulator R (RTDS).
The JADE framework has a Directory Facilitator (DF) that enables one to know all the agents that are active on the multiagent platform.There is also a sniffer agent that can be used to review how the communication between agents work.In this work, no index of communication performance was considered, which can be addressed in future works.
The middle agent is capable of interacting with the microgrid by using scripts commands to obtain information about the power meter and send commands to close or open the circuit breakers of the loads.This emulates a power management solution for a microgrid in a real telecommunication context.Such a communication system receives and processes information in real time, and then send responses.Therefore, the decision also occurs in real time.
The simulation starts with the microgrid in the grid-connected mode, i.e., there is no concern of the agents to meet the demand of the smart microgrid (SMG).As soon as the SMG is disconnected, the information is sent to the MAS to enter in the islanded mode, triggering the generation/load balance policy within the microgrid.In this research, the MAS follows the algorithm described in Figure 5 to operate the microgrid in the islanded mode based on the methodology presented in [55].The communication of the agents in this system determines all the loads and power sources available.This enables the manager agent to define the state of the SMG in order to guarantee the power supply to the priority loads.All the agents are registered in the DF so that the identification of them all could be possible in any state of the SMG. Figure 6 illustrates the process.
Once the middle agent is connected to the power system, such information is sent to the other agents (load and source agents).Both of them receive a stream of information in a multicast message sent by the middle agent that use the DF to know all the load agents and build a list.The same action is executed for the source agents.These ACL INFORM carry the information of all devices in the microgrid, which is monitored by the load or source agents.
Each agent verifies the message, find its respective information and update its status.This step occurs in 5 s time intervals and this information is not sent directly to the manager agent.The manager agent makes a request for both groups of agents, load, and source, using an ACL REQUEST message with the "Demand" and "Supply" conversation ID, respectively, in an interval of 5 s.When the source agents identify the request, they send an ACL INFORM message with the information of their status to the manager, which processes the data to determine what is the state that the microgrid is in that exact moment, based on the available variables.Then, the manager informs all the load agents about of the actual microgrid status, (EMERGENCY, CRITICAL or SURVIVAL) using an ACL INFORM message with the ID "microgrid-state", and so, each load agent verifies its priority, and if necessary, send an ACL REQUEST message to the middle agent to reject a respective load.
The middle agent receives the request and then, it sends to RSCAD a script command to open the branch using the TCP/IP connection and sends back to load agent an ACL INFORM message with the "OK" status.Then, the load agent changes its state to OFF.Therefore, the next update of all loads available will be different, and so will the microgrid state.Figure 7 illustrates the process.The Status EMERGENCY, CRITICAL or SURVIVAL can be defined according to distributed generation priority levels of the load for a particular microgrid.Hence, there is a new microgrid perspective from the demand and power supply that should be monitored by the manager agent.When the state of this microgrid is changed, the manager agent sends another message to all registered loads in the DF for each one to execute their own actions based on their state and priority.This will be repeated until all the power source is not able to supply the loads connected to the microgrid in the survival state.
On the other hand, if the manager agent detects that the supply is greater than the actual demand, it will send an ACL REQUEST to all loads, since they do not know the priority of each load.Each load agent receives the message and checks if the group or priority is the same of them.Then, they send the information back to the manager using an ACL CFP message with the maximum demand data or zero, if the group request is not the same of them, as shown in Figure 8.

Results
The simulation uses a client-server model to read the information from the IED.A polling time for the load and source agents to receive data from the middle agent is defined.The power profile data is in intervals of 15 min in order to generate a complete simulation, which is updated in the RTDS every 25 s, and the polling time of the middle agent and manager agent is 5 s.Thus, every 25 s in the graph of the manager (GUI) represents 15 min of data profile.The time used in the interaction of the power system with the multiagent system and vice-versa through the telecommunication system is on a scale 1 s:1 s.A 24 h-time span of the data profile of power generation and loads is employed.The results are shown in Figure 9.In Figure 9 the SMG starts in the grid-connected mode, so there is no problem with the demand to be greater than the generation.When the SMG changes the mode to islanded, for any reason, such as a faulted line, the SMG is in the emergency status.When the MAS detects that the actual generation in not able to supply all the loads in the microgrid, the manager changes the MSG state to critical in a first attempt to supply the loads.The manager has a new scenario of the SMG with only four loads and, in this moment the power generation inside the microgrid is still less than the necessary to supply the demand.Again, the manager changes the state of SMG, but now to survival.In this new scenario, only the two high priority loads can be supplied with the inside power generation.
Until this point, where loads were rejected using priority, it seems that there is no reason to use MAS to manage the microgrid energy when it is in islanded mode.However, when the generation is raising, the ability of MAS to exchange information shows its differentiated methodology that helps to prioritize the loads at the current power generation.The MAS has to discover the new demand of the new state before reconnecting the loads, so the Manager Agent has a picture of the actual generation and the next demand, to make the decision of raising the state or keeping the state.
The manager still keeps checking the balance of power generation inside the SMG, since the mode is still islanded.When the manager notes that there is more power than demand, it asks for the loads of next priority what are their maximum demand.If the power at this moment is greater than the actual demand plus the new demand of the next priority load, then the manager changes the state to critical, and the load is connected again to the SMG as seen in Figure 10.
When the power generation is decreasing, as shown in Figure 11, the manager agent still keeps checking the power balance inside the SMG, and when it becomes insufficient to supply the loads, it changes the state of SMG again to survival, so only the high priority loads are able to remain connected.It is assumed here that the total power generation inside the microgrid is sufficient to supply the high priority loads.
By using the first method to model the MAS, it is possible to see when the demand is higher than the supply, as depicted in in Figure 12.This happens for few seconds, but it is undesirable since it results in voltage and frequency drop issues.Since the methodology applied to MAS is based only on a pooling time, even when the manager identifies an unbalanced situation, this is reflected in the next pooling only.Therefore, to optimize the operation and obtain a better power management, an event-driven solution with the pooling mechanism was developed.Then, when the manager detects that the power is unbalanced, it sends a message for state changing and requests an update.The obtained result is now is a better fit of the balance as shown in Figure 11.Another issue that was discovered during the tests is that just reading the power of the meters will not lead MAS to the goal that was designed.The information that the load agent firstly was designed to receive is the information of the power meter.So, when the LOAD is connected to the microgrid, the information sent is the actual power consumption and it is useful to the moment when the power generation is decreasing.However, this information is useless when the power generation is raising.When the manager asks for the status of the loads to verify if is possible to supply the new demand, the meter is showing 0 W (zero).Thus, the manager will not raise the microgrid status, and the LOADs will not be reconnected to the grid.
During the test, it was noted the need for a more elaborated language for the MAS.As we can see at Figure 13 the Diesel generator was following the load in the microgrid, so when there was less load to supply, the model of the generator in the RTDS R reduced the power delivered.The maximum power capacity of this power resource should be sent to allow the Manager agent receive information that real reflect the place that it is inserted.
Without this information, the Manager could not decide to raise the microgrid state, because even when the PV generator achieve a power that plus the maximum power of the diesel generator and the Wind generator is capable to supply the next state, it will not change because the information is the actual power generation of the diesel power source.It is necessary a new behavior of the Source agent to lead the diesel generator to the desired power that could supply the new state.
So, a new simulation was made to test and verify this new behavior of the MAS, and the result is presented at Figure 14 Like presented at [56], the priority is the balance between load and generation and having the priority of dispatch for wind and photovoltaic sources, but all the decision for load shedding is made in real time by the agents.We just adjust the load LD02 to 0.80 MW to have a better visualization of the MAS.
The voltage profile of some buses and the frequency during island mode is presented at Figures 15 and 16.
The SMG with the energy balance methodology presented good results on the voltage profile and frequency, during the simulation of the MAS and SMG in the island mode (40 s), and we have the first change of state.The generator follows the load and reduces the power generated.When the DERs achieve power enough to join the maximum capacity of the diesel generator, the SMG raise the state (600 s) and this state remains until the actual power generation is below the actual demand (1150 s).At this time, the SMG lower the state and again the diesel generator follows the load and reduces the generation too.

Conclusions
This paper presented a framework in real time to test a solution for energy management in a smart microgrid in an islanded mode using the multiagent system.The integration of all agents was possible by using TCP/IP communication, but it was necessary a middleware to exchange and translate information from the MAS to the power system.Since was not possible to use an OpenSource JAVA API for the IEC 61850 in the JADE Framework, it was necessary to create a bridge between the data from the power system and the data for the MAS.This is a critical point because if the middle agent eventually goes down, the communication with the power system is lost.In a mission-critical system, like the power system, it is not desirable.
The use of MAS and its performance analysis in real time has shown to be a good solution for complex problems, but the correct use and modeling of MAS demands a great communication design to provide the sociability behavior expected from this type of methodology.When the MAS uses only the polling mechanism it was observed some problem related to the timing and synchronization for control of the loads in an unbalanced state.By adding an event-driven mechanism to the pooling, the problem was solved.Therefore, the simulation of multiagent systems in smart microgrids in real-time and with real communication links can reveal some issues that only happens in this type of simulation, like the kind of information an agent should be able to report and what type of methodology is better to fit a real situation.This enhances the relevance of the models and research framework presented in this paper.

Figure 3 .
Figure 3. Power Generation Profile for Wind and Photovoltaic Sources.

Figure 7 .
Figure 7. Data Exchange updated in MAS.

Figure 8 .
Figure 8. New Demand Request and Reply.

Figure 9 .
Figure 9. Energy Management for Power Profile.

Figure 10 .
Figure 10.Energy Management for Power Profile Increasing.

Figure 11 .
Figure 11.Energy Management for Power Profile Decreasing.

Figure 13 .
Figure 13.Multiagent System without Event Right Information.

Figure 15 .
Figure 15.Voltage Profile in Island Mode.

Figure 16 .
Figure 16.Frequency in the Microgrid in Island Mode.

Table 2 .
Description of Load at Microgrid.