Multiagent Based Distributed Control with Time-Oriented SoC Balancing Method for DC Microgrid

: Based on the droop control, voltage regulation at the secondary control is required to eliminate the deviation of the average voltage across the microgrid. Meanwhile, to prevent any of energy storage (ESs) from over-charging or over-discharging, State-of-Charge (SoC) balancing should be added in the secondary control. This paper proposes a distributed secondary control in the DC microgrid based on the multiagent system (MAS). This controller consists of voltage regulation and time-oriented SoC balancing. In voltage regulation, a PI controller adjusts the droop parameters according to the discrepancy between the average voltage and the reference voltage. In SoC balancing, controller operates in charging mode or discharging mode according to changes of the global average SoC. Being di ﬀ erent from the conventional method, the time-oriented SoC balancing method is designed to balance charge / discharge time rather than to balance SoC directly. Thus, SoCs reach a consensus only at the last moment when all ES nodes charge or discharge completely. Furthermore, characteristics, global dynamic model, and steady-state analysis of the proposed control method are studied. Finally, MATLAB / Simulink simulations are performed to verify the e ﬀ ectiveness of the proposed control.


Introduction
In recent years, microgrid has received great attention as an important solution to utilize renewable energy resources such as wind, solar, and tidal energy. Microgrid can effectively integrate distributed generations (DGs), converters, battery energy storage systems (BESSs), and loads. With the advent of more DC loads like electrical vehicles (EVs), DC microgrid has attracted extensive interest and related researches are developing rapidly. Compared to AC microgrid, DC microgrid has considerable advantages. For example, power loss is less without DC/AC converters, moreover, reactive power, frequency, and phase are not problems any more [1,2].
Hierarchical control has been generally accepted since it was put forward [3]. Hierarchical control is composed of primary control, secondary control, and tertiary control. Primary control is the droop control that realizes reference voltage and power sharing. Secondary control eliminates the deviation of voltage and power sharing. Tertiary control is the energy dispatch. According to the configuration of the communication system, secondary control can be divided into 3 types: centralized control, decentralized control, and distributed control. For the centralized control, information like voltages and currents of all nodes including power generation nodes and energy storage (ES) nodes is delivered through network communication to the central controller, then PI controllers calculate droop correction terms based on the global information. The most critical disadvantage of the centralized control is that any fault of the network will cause breakdown of the whole secondary control. For the decentralized control, coordination strategy relies on local information without communication links, and its key is to find an appropriate control method to satisfy the requirements of voltage regulation and power sharing. Distributed control is based on multiagent system (MAS), where central controller is not needed anymore and every node only communicates with its adjacent nodes. Controller at every node collects local and neighbors' information to estimate the global average voltage and other information across the whole microgrid by some consensus protocols [4,5].
To prevent any of ESs from over-charging or over-discharging, State-of-Charge (SoC) balancing is necessary. During the discharging process, the ES with higher SoC should supply more power than the others. While during the charging process, the ES with lower SoC should absorb more power than the others. Based on the hierarchical control, SoC balancing methods can be divided into 3 types: centralized, decentralized, and distributed. Through some modified droop control, SoC balancing can be realized in a decentralized way [6][7][8][9][10][11]. Reference [6] proposed a decentralized control method with SoC balancing. In this method, the droop coefficients are proportional to SoC n during the charging process but are inversely proportional to SoC n during the discharging process. It is obvious that the speed of SoC balancing relies on the parameter n. However, there are 3 drawbacks. First, the change ranges of the droop coefficients are too large, which will influence the system stability. Particularly, the system will be unstable when SoC is very small. In addition, the droop coefficients are still changing even after SoC balancing has been achieved. Finally, the method cannot work well when capacities of ESs are different. In Reference [7], a global cost function is proposed based on the effective load and SoC. Then, an optimal droop surface is derived to determine the droop coefficients to balance SoC and optimize the global cost function. In Reference [8], the reference frequency of the droop control is regulated in a linear way according to the changes of local SoC for an AC microgrid, but there will be still frequency deviation during the process of SoC balancing. In Reference [9], the droop coefficients are proportional to e SoC in the charging process but are inversely proportional to e SoC in the discharging process to balance SoC for the DC microgrid. Reference [10] compared various SoC balancing functions in the decentralized control and found that the power function SoC n and the exponential function e SoC were in fact the best choice to promote SoC equalization. Particularly, the power function exhibited the fast speed of SoC equalization and the lowest DC voltage deviation.
For the centralized control, most literatures focused on SoC balancing by regulating the droop parameters based on the average SoC, like regulating reference currents [11], regulating droop coefficients [12], regulating reference voltages [13], and so on. Generally, control methods used in the centralized control can be also used in the distributed control with some modification. In References [14,15], the droop parameters are adjusted according to the discrepancy between local SoC and the global average SoC, and the latter can be estimated by the dynamic consensus protocol. Furthermore, the droop control can be also adjusted according to the discrepancies among local SoC and neighbors' SoC in References [16,17]. Reference [18] involved sliding mode control in the distributed control of DC microgrid to set the ES's level of participation in the droop control for SoC balancing.
In this paper, a multiagent based distributed control method with voltage regulation and time-oriented SoC balancing for the islanded DC microgrid is proposed. In the novel SoC balancing method, direct control objective is charge/discharge time rather than SoC consensus. In fact, for the SoC balancing, there is no need to make SoCs reach a consensus too early, and it is enough to ensure that ESs charge or discharge completely at the same time. Hence, the charging and discharging time is used as the feedback signals in this paper. Compared to the conventional SoC balancing methods, the proposed time-oriented SoC balancing method can regulate ESs in a more progressive way, which can avoid the ESs with large charging or discharging currents, especially for the extremely unbalanced situations. In this way, the ESs with higher SoC or lower SoC can be protected from overcurrent, and the pressure on filter inductors to avoid saturation is alleviated. Besides, the novel method can work well in The rest of this paper is organized as follows. First, the preliminaries of MAS and graph theory are introduced in Section 2. Then, Section 3 discusses the global controller including voltage regulation and time-oriented SoC balancing. Besides, Section 4 studies characteristics, global dynamic model, and steady-state analysis of the proposed controller. Furthermore, Section 5 verifies the proposed controller by the islanded DC microgrid simulation. Finally, the paper is concluded in Section 6.

Preliminaries of MAS and Graph Theory
In MAS, agents cooperate with each other to complete a task. Agents just communicate with their neighbors directly and this sparse communication network can be represented by an undirected graph. In the graph, the nodes are agents, and edges represent the communication links for information exchange. Every node just communicates with its neighbors and makes use of local and neighbors' data to realize global consensus through the average consensus protocol.
The nodes, edges, communication weights, etc. can be described by matrices in the graph. The weighted adjacency matrix is A G = a ij ∈ R n×n , where n is the number of nodes and a ij denotes communication weight from node j to node i. a ij > 0 when node j is adjacent to node i, or a ij = 0 otherwise. The Laplacian matrix is defined as L = l ij ∈ R n×n , where l ii = n j=0 a ij and l ij = −a ij , i j. Furthermore, the Laplacian matrix is balanced if the in-degree of each node is equal to its out-degree. For the undirected graph, it has many properties. First, Laplacian matrix of the undirected graph is balanced. Second, if the undirected graph is connected, its L has a simple zero eigenvalue and all nonzero eigenvalues are positive. Right eigenvector and left eigenvector associated with zero eigenvalue are 1 = [1, 1, . . .  Figure 1 shows the layout of proposed multiagent based distributed control in the DC microgrid, which consists of physical, cyber, and control layers. In the physical layer, there are nodes and remote loads connected to bus. A node consists of ES, power electronic converter including controller like DSP, local loads, and transmission line. In the cyber layer, there is a sparse communication network among all nodes where every node communicates directly to its neighbors. Note that the communication network is nothing to do with the topology of physical microgrid. Importantly, the communication network should be designed such that in case of any link failure, the remaining network is still connected. Ring-like network is a simple but useful example. Generally, links are undirected.

Control Layout
The control layer consists of primary control and secondary control including voltage regulation and time-oriented SoC balancing, which are based on local and neighbors' data. Primary control is mainly composed by the droop control and its control law can be expressed as where δv 1 i and δv 2 i mean the first voltage correction term and the second voltage correction term, and they are output of the PI controller H i in voltage regulation and G i in SoC balancing, respectively. Like common droop control, v * i , v re f , r i , and i i are local voltage setpoint, reference voltage, droop coefficient, and current, respectively. voltage at node j and node i, respectively. By differentiating (2), it can be obtained that Rewrite (3) in the matrix form, the following expression can be derived = -(4)

Voltage Regulation
As shown in Figure 1, controller at each node estimates the average voltage of the whole microgrid based on local dynamic voltage and neighbors' estimated average voltage, then PI controller H i calculates the first voltage correction term δv 1 i according to the discrepancy between this estimate v i and the global reference voltage v re f . The protocol to estimate the average voltage is designed as where v i (t) is the dynamic voltage at node i. v j (t) and v i (t) denote the estimation of average voltage at node j and node i, respectively. By differentiating (2), it can be obtained that Energies 2020, 13, 2793

of 17
Rewrite (3) in the matrix form, the following expression can be derived where V and _ V are the Laplace transformations of v and _ v, respectively. According to (2), where I n ∈ R n×n and H obs = s(sI n + L) −1 are identify matrix and transfer function from V to _ V, respectively. If L is balanced, then all elements of _ v converge to a consensus value, which is the true average voltage, i.e., the average of all elements in v [5]. In other words where Q ∈ R n×n is the averaging matrix, whose elements are all equal to 1/n. x ss expresses the steady-state value of the column vector x ∈ R n×1 . x represents the average of all elements in the column vector x. 1 ∈ R n×1 is a column vector whose elements are all equal to one.

Time-Oriented SoC Balancing Method
As shown in Figure 1, first, controller at each node estimates the average SoC among all nodes based on local SoC and neighbors' estimated average SoC. According to whether the estimated average SoC is increasing or decreasing, controller operates in charging mode or discharging mode.
The protocol for estimation of average SoC is designed as where SoC i (t) is the real-time SoC at node i. SoC j (t) and SoC i (t) denote the estimate of global average SoC at node j and node i, respectively. If SoC i (t) is increasing, controller will operate in the charging mode, while if SoC i (t) is decreasing, controller will operate in the discharging mode.
Generally, according to battery discharge characteristic, SoC of ES should be controlled between the lower limit of SoC (20%) SoC L and the upper limit of SoC (90%) SoC U .

Discharging Mode
First, the derivative of SoC i (t) can be calculated as In discharging mode, k i (t) is negative. If k i (t) is constant, then the time for ES at node i to discharge completely can be obtained Energies 2020, 13, 2793 6 of 17 When SoC i (t) = SoC L = 20%, it means that ES at node i needs to be cut now. If t d i > t d j , it shows that ES at node i is supplying less power than that at node j, and it will spend more time for ES at node i to discharge completely. The average completely discharge time among all nodes can be calculated by the average consensus protocol, namely Then can represent the discrepancy of completely discharge time between node i and 'other nodes' shows that ES at node i is supplying less power than others, and the second voltage correction term δv 2 i at node i should increase. Therefore where g ip and g ii are the proportional and integral gains, respectively, in the PI controller G i . However, in practice, k i (t) maybe very small and close to 0. t d i will be very large and may be out of limit. In order to use the reciprocal of t d i as the control objective, swap the numerator and denominator in (11) and remove absolute value sign considering k i (t) < 0. Therefore, (11), (12), and (13) are replaced by Note that when SoC is close to SoC L (20%), the denominator is close to 0. So, ES should be cut a little early, for example, when SoC = 21%.

Charging Mode
Protocol in charging mode is very similar to that in discharging mode.
This can represent the time for ES at node i to charge completely and SoC i (t) = SoC U = 90% at that time. Then, the average complete charge time among all nodes can be calculated by the average consensus protocol shows that ES at node i is absorbing less power than others, and the second voltage correction term at node i, i.e., δv 2 i , should decrease. Therefore Energies 2020, 13, 2793 7 of 17 In practice, in order to use the reciprocal of t c i as the control objective, swap the numerator and denominator in (17) and remove absolute value sign considering k i (t) > 0. Therefore, (17), (18), and (19) are replaced by Note that when SoC is close to SoC U (90%), the denominator is close to 0. So, ES should be switched to floating charge mode a little early, for example, when SoC = 89%.
The integral of integrator in PI controller G i remains unchanged when controller switches between discharging mode and charging mode, in other words, controller uses the same integrator in both modes. That is because ES with higher SoC always needs to supply more power or to absorb less power, and its δv 2 i is always bigger than that of ES with lower SoC. So, there is no need to clear the integral to zero when controller switches mode.

Characteristics
As shown in Figure 2, the time-oriented SoC balancing method is designed to balance charge/discharge time rather than to balance SoCs directly, in other words, all ES nodes should charge or discharge completely at the same time. Discrepancies among all SoCs decrease very slowly and SoCs reach a consensus only at the last moment when all ES nodes charge or discharge completely. So, the changing curves of SoCs in this method are milder than those in the traditional method. Therefore, the SoC balancing method can protect ES and its inductors from overcurrent. Apparently, the novel method can work well without knowledge of capacities when capacities of ESs are different.
Energies 2020, 13, x FOR PEER REVIEW 7 of 16 As shown in Figure 2, the time-oriented SoC balancing method is designed to balance charge/discharge time rather than to balance SoCs directly, in other words, all ES nodes should charge or discharge completely at the same time. Discrepancies among all SoCs decrease very slowly and SoCs reach a consensus only at the last moment when all ES nodes charge or discharge completely. So, the changing curves of SoCs in this method are milder than those in the traditional method. Therefore, the SoC balancing method can protect ES and its inductors from overcurrent. Apparently, the novel method can work well without knowledge of capacities when capacities of ESs are different.

Global Dynamic Model
According to (1), there is where correction term vector, and the second voltage correction term vector, respectively. r = diag{r i } is the droop coefficient matrix.
∆V 1 and ∆V 2 can be calculated as where H = diag{H i } and G = diag{G i } are PI controller matrices of voltage regulation and SoC balancing method, respectively. K =

Steady-State Analysis
Here, steady-state analysis is studied to ensure that the proposed controller meets two requirements: the global voltage regulation and the SoC balancing.
First, assume where v re f is the reference voltage value. Suppose that dynamic voltage vector V is stable and the final value theorem is effective. According to (31), there is It is known that the DC gains of the closed-loop converters are equal to one [5], i.e., where H P and H I denote diagonal matrices carrying proportional and integral gains of the voltage controllers. Similarly, G P and G I denote diagonal matrices carrying proportional and integral gains of the time-oriented SoC balancing controllers. By substituting (7) All the integrator gains for the SoC balancing controllers are positive, so, G −1 I exists. Pre-multiply at both sides of (38) by QG −1 . Because Q is the averaging matrix, whose elements are all equal to 1/n, there is QQ = Q Therefore It shows that the average voltage across the microgrid can be regulated at the reference voltage value as expected.
Therefore, it can be derived from Equation (38) that Pre-multiply at both sides of (44) by LG −1 Because L1 = 0, LQ = 0. Therefore, (45) can be written as Therefore, K ss is the right eigenvector of L associated with zero eigenvalue, and where m is a scalar factor. It implies that the time for all ESs to charge or discharge completely is identical.

Simulation Results
An islanded 400V DC microgrid comprised of 8 ES nodes is simulated in MATLAB/Simulink to verify the effectiveness of proposed controller. Eight ES nodes and a remote load are connected in parallel with bus physically, and a communication network of 8 nodes is shown in Figure 3. Battery at each node is connected to bus through bidirectional Buck-Boost converter. Transmission line impedances at both sides are identical. Public and private parameters of 8 ES nodes are shown in Tables 1 and 2, respectively. identical.

Simulation Results
An islanded 400V DC microgrid comprised of 8 ES nodes is simulated in MATLAB/Simulink to verify the effectiveness of proposed controller. Eight ES nodes and a remote load are connected in parallel with bus physically, and a communication network of 8 nodes is shown in Figure 3. Battery at each node is connected to bus through bidirectional Buck-Boost converter. Transmission line impedances at both sides are identical. Public and private parameters of 8 ES nodes are shown in Table 1 and Table 2 Where denotes the number of neighbors at node i.    where n i denotes the number of neighbors at node i. A total of 5 cases are simulated and results are analyzed. The first case is charging case, the second case is discharging case, the third case is switching case, the fourth case is fast discharging and charging case, and the last case is the comparative case using the conventional SoC balancing method proposed in Reference [6].

Case 1: Charging simulation
Charging simulation is performed for 3000 s. The average voltage across the microgrid is shown in Figure 4a. Dynamic voltages of 8 ES nodes are shown in Figure 4b. SoCs of 8 ES nodes are shown in Figure 4c. Output currents of 8 nodes are shown in Figure 4d. At first, droop controller is implemented, remote load is 8 Ω, and an extra constant current source supplies 150A current. At t = 1 s, proposed controller is performed. At t = 1200 s, remote load changes in step from 8 Ω to 4 Ω. At t = 2400 s, remote load changes in step from 4 Ω to 8 Ω.
Lower limit of SoC 20% Upper limit of SoC 90% A total of 5 cases are simulated and results are analyzed. The first case is charging case, the second case is discharging case, the third case is switching case, the fourth case is fast discharging and charging case, and the last case is the comparative case using the conventional SoC balancing method proposed in Reference [6].
• Case 1: Charging simulation Charging simulation is performed for 3000s. The average voltage across the microgrid is shown in Figure 4a. Dynamic voltages of 8 ES nodes are shown in Figure 4b. SoCs of 8 ES nodes are shown in Figure 4c. Output currents of 8 nodes are shown in Figure 4d. At first, droop controller is implemented, remote load is 8Ω, and an extra constant current source supplies 150A current. At = 1 s, proposed controller is performed. At = 1200 s, remote load changes in step from 8Ω to 4Ω. At = 2400 s, remote load changes in step from 4 Ω to 8 Ω.
It can be seen that SoCs of 8 ES nodes have been different until all SoCs are equal to 90%. It can be seen that SoCs of 8 ES nodes have been different until all SoCs are equal to 90%. Discharging simulation is performed for 2900s. The average voltage across the microgrid is shown in Figure 5a. Dynamic voltages of 8 ES nodes are shown in Figure 5b. SoCs of 8 ES nodes are shown in Figure 5c. Output currents of 8 nodes are shown in Figure 5d. At first, droop controller is implemented and remote load is 4 Ω. At = 1 s, proposed controller is performed. At = 1200 s, remote load changes in step from 4 Ω to 8Ω. At = 2400 s, remote load changes in step from 8Ω to 4 Ω.
It can be seen that SoCs of 8 ES nodes have been different until all SoCs are equal to 20%. It can be seen that, according to whether the average SoC is increasing or decreasing, controllers switch between discharging mode and charging mode smoothly. It can be seen that, according to whether the average SoC is increasing or decreasing, controllers switch between discharging mode and charging mode smoothly. It can be seen that the time-oriented SoC balancing method can work well in fast discharging and charging situation.   It can be seen that the time-oriented SoC balancing method can work well in fast discharging and charging situation.

•
Case 5: Comparative simulation using the conventional SoC balancing method The all basic parameters for simulation in the case are the same as case 4. The protocol for SoC balancing is chosen as The voltage at the remote load is shown in Figure 8a. Dynamic voltages of 8 ES nodes are shown in Figure 8b. SoCs of 8 ES nodes are shown in Figure 8c. Output currents of 8 nodes are shown in Figure 8d.
The voltage at the remote load is shown in Figure 8a. Dynamic voltages of 8 ES nodes are shown in Figure 8b. SoCs of 8 ES nodes are shown in Figure 8c. Output currents of 8 nodes are shown in Figure 8d.
It can be seen that the SoC balancing method proposed in Reference [6] has 3 drawbacks. First, voltage fluctuates widely in the discharging process because the droop coefficients are too big, which is caused by . Second, the speed of SoC balancing is too slow compared to time-oriented SoC balancing method, though = 5 in = 20 × . This is because the conventional SoC balancing method cannot work well when the variations of line loads are very high. Third, the conventional SoC balancing method cannot work well when capacities of ESs are different.
Last but not least, if the speed of SoC balancing in the conventional method is faster than that in the time-oriented SoC balancing method, the peak currents in the conventional method will be higher. Five cases can verify the effectiveness of the proposed controller very well. Apparently, voltage deviation between the reference voltage and the average voltage has been evidently eliminated after the proposed controller is performed. There are always differences among dynamic voltage of 8 ES nodes, but the average voltage across the microgrid is stabilized at reference voltage. Furthermore, all SoCs have been different until they are all equal to 90% in charging mode or 20% in discharging It can be seen that the SoC balancing method proposed in Reference [6] has 3 drawbacks. First, voltage fluctuates widely in the discharging process because the droop coefficients are too big, which is caused by 1 SoC n . Second, the speed of SoC balancing is too slow compared to time-oriented SoC balancing method, though n = 5 in r i = 20 × SoC n i . This is because the conventional SoC balancing method cannot work well when the variations of line loads are very high. Third, the conventional SoC balancing method cannot work well when capacities of ESs are different.
Last but not least, if the speed of SoC balancing in the conventional method is faster than that in the time-oriented SoC balancing method, the peak currents in the conventional method will be higher.
Five cases can verify the effectiveness of the proposed controller very well. Apparently, voltage deviation between the reference voltage and the average voltage has been evidently eliminated after the proposed controller is performed. There are always differences among dynamic voltage of 8 ES nodes, but the average voltage across the microgrid is stabilized at reference voltage. Furthermore, all SoCs have been different until they are all equal to 90% in charging mode or 20% in discharging mode. Besides, proposed controllers switch between charging mode and discharging mode smoothly.
It can be seen that the time-oriented SoC balancing method can work well in 3 extreme situations where fast discharging and charging is needed, capacities of ESs are different, and variations of line loads are high.
It is worth mentioning that output currents are almost constant without step load change during SoC balancing. However, output currents will change when SoC balance is achieved in the conventional SoC balancing methods.

Conclusions
A multiagent based distributed controller in DC microgrid is proposed. The controller consists of voltage regulation and time-oriented SoC balancing. Voltage regulator estimates average voltage, and the discrepancy between this estimation and reference voltage is then used to calculate the first voltage correction term by PI controller. In the SoC balancing, the average SoC needs to be estimated first. If the average SoC is increasing, the controller will be in charging mode, or the controller will be in discharging mode otherwise. The time-oriented SoC balancing method balances charge/discharge time rather than SoCs directly; in other words, all ES nodes should charge or discharge completely at the same time. Global dynamic model and steady-state analysis are studied, and MATLAB/Simulink simulations verify the proposed controller. The advantages of the novel method are summarized as follows. First, the method can work well without knowledge of capacities when capacities of ESs are different. Second, the method does a good job in fast discharging and charging process. Third, the method is not sensitive to line loads. Fourth, the method can protect the ES and filter inductor from overcurrent.