Multi-Party Energy Management for Networks of PV-Assisted Charging Stations: A Game Theoretical Approach

Abstract

Motivated by the development of electric vehicles (EVs), this paper addresses the energy management problem for the PV-assisted charging station (PVCS) network. An hour-ahead optimization model for the operation of PVCS is proposed, considering the profit of the PVCS, the local consumption of the photovoltaic (PV) energy and the impacts on the grid. Moreover, a two-level feasible charging region (FCR) model is built to guarantee the service quality for EVs and learning-based decision-making is designed to assist the optimization of the PVCS in various scenarios. The multi-party energy management problem, including several kinds of energy flows of the PVCS network, is formulated as a non-cooperative game. Then, the strategies of the PVCSs are modeled as the demand response (DR) activities to achieve their own optimization goals and a two-level distributed heuristic algorithm is introduced to solve the problem. The simulation results show that the economic profit of the network is increased by 6.34% compared with the common time of use (TOU) prices approach. Besides, the percentage of the PV energy in total charging load (PPTCL) and load rate are promoted by 28.93% and 0.3125, respectively, which demonstrates the validity and practicability of the proposed method.

1. Introduction

With the continuous worldwide shortage of fossil fuels and environmental pollution problems, development of renewable energy resources and electric vehicles (EVs) is regarded as an effective way to reduce carbon emissions by many countries [1,2,3]. However, EVs need to be connected with the utility grid for charging and there are still many barriers to be overcome [4,5]. Firstly, the indirect emissions of EVs cannot be ignored. If the power generation of the electric power system is dominated by coal-fired power plants, the emission reduction advantage of EVs is not obvious, which means EVs are not very helpful to the environment. Secondly, more investment is needed to expand the capacities of generation, transmission and distribution system due to the boosting charging requirements of EVs [6]. As a form of renewable and clean energy, photovoltaic (PV) energy can be produced anywhere, including in urban areas for EV applications. Therefore, the integration of PV resources with EV charging infrastructure is a possible way to effectively improve the emission reduction of EVs, meet the daytime charging demand and reduce the dependence of EVs on the utility grid [7,8]. Due to the above advantages, the number of PV-assisted charging stations (PVCSs) has increased quickly and service networks of PVCSs will gradually emerge.

At present, the issues surrounding PVCSs are widely discussed, including charging strategies [9,10], energy management methods [11,12] and optimal planning [13,14,15]. Moreover, some supporting technologies such as the thermography-based virtual maximum power point tracking (MPPT) scheme [16], PV array fault diagnosis [17] and new switched reluctance (SR) drive for PHEVs [18] are proposed to improve the efficiency of PV generation and EV charging. Therefore, great efforts have been made to increase the economic profit, efficiency and service quality of the individual stations. However, until recently, we have not found any usable models for the operation of the PVCS network.

Meanwhile, for the common charging stations (CSs) without PV sources, several papers are focused on the network of CSs, which can be roughly categorized into two groups: (1) locating and sizing; and (2) energy management. For the first, several different methods and algorithms were proposed, i.e., the life cycle cost (LCC) criterion [19], the primal-dual interior point algorithm [20], and the cross-entropy method [21], etc., to achieve optimal locating and sizing of the CSs. The main objective of the planning is to increase the service ability [20] of the CS network and minimize the construction cost of the CSs and the reinforcement cost of the distribution network [19,20]. In addition, the minimizations of the power losses and voltage deviations as well as the EV traveling distance are also considered [21].

As for the latter, it has been proven that game theory is an effective way to deal with the multi-party energy problem of the CS network [22]. A Stackelberg game-based control mechanism was introduced in [23] to manage a population of self-interested mobile EVs. The aim of the leader (network operator) is to serve more customers with the same amount of grid resources, while the goal of the EV followers (EV drivers) is to get the charging service at a minimum cost. Moreover, in [24], a game model was proposed to help the EV drivers select an appropriate CS, working toward a time/price tradeoff. Besides, the geographical information system (GIS) developed in [25] is also a useful tool which can be considered to appropriately manage the interactions between different PVCSs and the coordinator in a limited area.

However, the multi-party energy management of the CS network cannot be suitable for the network of PVCS, for the following two reasons. First, the PVCS is actually a type of prosumer, which can act as an energy buyer or seller alternatively during the operation. The optimization model of a PVCS would be totally different from the conventional CS, considering the impact of PV energy. Second, a new type of energy exchange flow between the selling role PVCS and the buying role PVCS will emerge in the network. As the PVCSs are equipped with different capacities of PVs, the trading among the adjacent PVCSs is a possible way to increase the utilization of PV energy and lower their operation cost.

To this end, this paper aims to address the multi-party energy management problem of the PVCS network. The main contributions can be summarized as follows:

(1)

An hour-ahead optimization model for the operation of PVCS is proposed, considering the profit of the PVCS, the local consumption of the PV energy and the impacts on the grid. Moreover, to guarantee the service quality for EVs, a two-level feasible charging region (FCR) model is built to work as a real-time constraint for the optimization. Considering the EV charging demands and PV energy might be quite different in various scenarios, learning-based decision-making is designed to assist the optimization of the PVCS.

(2)

The multi-party energy management problem of the PVCS network is formulated as a non-cooperative game. The energy flow includes the buying of energy from the conventional power plants, the trading among the PVCSs, and the surplus PV energy fed back to the utility grid. The strategy of each player (PVCS) is modeled as a demand response (DR) activity to achieve their own optimization goal. The existence and uniqueness of Nash equilibrium of the game is proved, and a two-level distributed heuristic algorithm is introduced to solve the problem.

2. Basic Structure and Models

2.1. Structure of the PVCS Network

As shown in Figure 1, the PVCS network consists of a number of PVCSs which are located in different areas in a distributed network, and it is connected with the power plant and the utility grid. In addition, each station in this network includes charging load, PV resource and user energy management system (UEMS).

As we can see, all the PVCSs are connected with the coordinator via information lines to upload information and receive messages or instructions. The coordinator is in charge of gathering/transferring information. Moreover, the PVCSs can exchange information with the coordinator and collect the data of the charging load as well as the PV resource with the help of their UEMS. Besides, they also have the ability for cost calculation and optimization of local controls.

2.2. Model of EV’s Charge

The relationship between the state of charge ($SOC$) and the current of the charged EV is given by [26]:

$$\begin{array}{c}\hfill SOC\left(t\right)=SOC\left(t_0\right)+\frac{I\left(t\right)}{Q_n}(t-t_0)\end{array}$$

(1)

where $SOC\left(t\right)$ is the SOC of the EV’s battery at time slot t; $I\left(t\right)$ is the charging current at time slot t; and $Q_n$ is the rated capacity of the EV’s battery.

The charging rate of the EV is defined by:

$$\begin{array}{c}\hfill C\left(t\right)=\frac{I\left(t\right)}{Q_n}\end{array}$$

(2)

Hence, (1) can be transformed to:

$$\begin{array}{c}\hfill SOC\left(t\right)=SOC\left(t_0\right)+C\left(t\right)(t-t_0)\end{array}$$

(3)

Thus, the charging power of the EV is given by:

$$\begin{array}{c}\hfill P\left(t\right)=U_n·C\left(t\right)·Q_n,C\left(t\right)\in [C_{min},C_{max}]\end{array}$$

(4)

where $U_n$ is the rated voltage; and $C_{min}$ and $C_{max}$ are the lower and upper bound of the charging rate.

2.3. Direct Power Supply for Large Consumer

In the electric energy market, the large consumers which reach a certain scale have the qualifications to directly purchase energy from power plants instead of the electric power company [27]. In this way, the large consumers can buy the energy at a lower price and they just need to pay the transmission and distribution fees to the electric power company.

The network of the PVCSs is assumed to be qualified for direct power supply for large consumers, so the network can directly purchase energy from power plants. Generally, the generation cost of the fossil fuel unit mainly depends on the fuel consumption characteristic which is usually given by a quadratic function:

$$\begin{array}{c}\hfill C_g=a·E^2+b·E+c\end{array}$$

(5)

where E is the amount of the generated energy; a,b,c are the coefficients of the cost function.

Besides the generation cost, the cost of power transmission and distribution should be considered as well and it is given by a linear function:

$$\begin{array}{c}\hfill C_{td}=c_{td}·E\end{array}$$

(6)

where $c_{td}$ is the unit cost of power transmission and distribution.

If the PVCS network possesses surplus energy in a time slot, this part of energy will be sold to the utility grid to obtain some income, i.e., negative cost. Therefore, the total cost of the PVCS network should be calculated through a piecewise function:

$$C_{total}=\left\{\begin{array}{c}{C}_g+C_{td},E\ge 0\\ \gamma ·E,E<0\end{array}\right.$$

(7)

where $\gamma$ is the unit price of the energy which is sold to the utility grid.

The above piecewise function can be approximated into a quadratic function when E is within a certain range (the approximation is illustrated in Appendix A). The approximated quadratic function can be represented as:

$$\begin{array}{c}\hfill C_{total}^{\prime }=a^{\prime }·E^2+b^{\prime }·E\end{array}$$

(8)

Thus, the unit price for the PVCSs to trade (buy or sell) energy with the power plant and the utility grid is given by:

$$\begin{array}{c}\hfill P{r}_{buy}=\frac{C_{total}^{\prime }}{E}=a^{\prime }·E+b^{\prime }\end{array}$$

(9)

3. Two-Level FCRs

As we know, most of the vehicles which are parked around office buildings belong to the staff who work in these buildings, and thus they always arrive early in the morning and leave at dusk due to work. Therefore, the parking time is usually much longer than the time they need to for charging if these vehicles are all EVs. The charging activities of these EVs are adjustable to a great extent, so this paper mainly focuses on the PVCSs which provide a charging service for these controllable EVs.

A real-time energy management method is proposed in this paper to solve the energy trading problem of the PVCS network. Here, one day is divided into H time slots which are denoted by T ($T=1,2,...,H$) and the optimal decisions are made at the beginning of each time slot. Before the optimization of the PVCSs and the EVs, the feasible regions at the station level and vehicle level must be known to obtain the constraints of the optimization.

3.1. FCR at the Station Level

The owners of the EVs should input their departure time and objective $SOC$ of the EVs on the interactions panel as soon as they arrive at the charging station. Hence, the operator of the PVCS can obtain the optimal decisions according to the submitted information.

At the the beginning of time slot T, the charging requirements of all the EVs are examined. Taking EV i for example, the difference between its present $SOC$ and objective $SOC$ is given by:

$$\begin{array}{c}\hfill SO{C}_d^i=SC{O}_{obj}^i-SO{C}^i(T-1)\end{array}$$

(10)

where $SC{O}_{obj}^i$ is the objective $SOC$ of EV i; $SO{C}^i(T-1)$ is the $SOC$ of EV i at the end of time slot $T-1$.

The EVs which need to be charged in time slot T are divided into two types for expression convenience.

Type 1: 

This type of EV will leave before the end of time slot T, so they must be charged in time slot T to reach the objective $SOC$ before their departure time.

Type 2: 

This type of EV will not leave before the end of time slot T, so they can be charged or not in time slot T depending on the amount of the charging load and the PV energy.

According to the classification of EVs, the upper limit of the charging load of PVCS k at time slot T can be obtained by meeting the charging requirements of all the EVs: Please ensure all abbreviations are defined the first time they appear in main text throughout.

$$\begin{array}{c}\hfill C{E}_{max}^k\left(T\right)=\sum _{i\in D}min(SO{C}_d^i,C{r}_{max}·\Delta T)·Q_n^i·U_n\\ \hfill +\sum _{i\in M}min(SO{C}_d^i,C{r}_{max}·[t_{dep}^i-(T-1)])·Q_n^i·U_n\end{array}$$

(11)

where M is the set of the Type 1 EVs and D is the set of Type 2 EVs; $Q_n^i$ is the rated capacity of the battery of EV i; $t_{dep}^i$ is the departure time of EV i; $C{r}_{max}$ is the maximum charging rate of each charger; and $\Delta T$ is the time duration of time slot T.

As for the lower limit, the charging load in time slot T must meet at least the charging requirements of Type 1 EVs in each PVCS. Therefore, the lower limit of the charging load of PVCS k at time slot T is:

$$\begin{array}{c}\hfill C{E}_{min}^k\left(T\right)=\sum _{i\in M}min(SO{C}_d^i,C{r}_{max}·[t_{dep}^i-(T-1)])·Q_n^i·U_n\end{array}$$

(12)

3.2. FCR at the EV Level

The lower limits of all the EVs’ charging rates are set to 0, i.e., $C_{min}^i=0,\forall i\in M,D$. However the upper limits of the EVs’ charging rate are divided into two situations according to the types of EVs.

For Type 1 EVs, as mentioned before, they will leave before the end of time slot T so they cannot be charged through the whole hour. Thus the upper limit of Type 1 EVs is given by:

$$\begin{array}{c}\hfill C_{max}^i\left(T\right)=min(C{r}_{max},\frac{SO{C}_d^i}{t_{dep}^i-(T-1)})\end{array}$$

(13)

For Type 2 EVs, they will not leave before the end of time slot T so they will be charged through the whole hour. Thus, the upper limit of Type 2 EVs is:

$$\begin{array}{c}\hfill C_{max}^i\left(T\right)=min(C{r}_{max},SO{C}_d^i)\end{array}$$

(14)

4. Modeling of the PVCSs Network

4.1. Load Model of the Network

In order to maximize the self-consumption of PV energy and minimize the cost of purchasing energy, internal energy trading occurs prior to external energy trading. To be specific, if some PVCSs possess surplus energy at time slot T, this part of energy is sold to the other PVCSs in this network which have an energy deficit at first. After this kind of internal trading, the network will purchase energy from the power plant if there is still energy deficit or sell energy to the utility grid if there is still surplus energy.

For PVCS k, the netload of this PVCS in time slot T is:

$$\begin{array}{c}\hfill N{L}^k\left(T\right)=C{E}^k\left(T\right)-p{v}^k\left(T\right)\end{array}$$

(15)

where $C{E}^k\left(T\right)$ is the charging load of PVCS k at time slot T; and $p{v}^k\left(T\right)$ is the PV output of PVCS k at time slot T.

Thus taking the transmission losses into consideration, the total netload of the PVCSs network is given by:

$$\begin{array}{c}\hfill NL\left(T\right)=\sum _{k\in S_s}(1-\delta )N{L}^k\left(T\right)+\sum _{k\in S_d}N{L}^k\left(T\right)\end{array}$$

(16)

where $S_s$ and $S_d$ represent the set of the PVCSs which have energy surplus and deficit, respectively; and $\delta$ is the rate of transmission losses.

4.2. Objective of the PVCS

The economic profit of each PVCS can be described as:

$$\begin{array}{c}\hfill pr{o}_{eco}^k\left(T\right)=C{E}^k\left(T\right)P{r}_s+p{v}^k\left(T\right)S_{pv}-N{L}^k\left(T\right)P{r}_b\left(T\right)\end{array}$$

(17)

where $P{r}_s$ is the unit price of the charging service for EVs; $S_{pv}$ is the unit subsidy for PV generation; and $P{r}_b\left(T\right)$ is the unit price of energy trading, including the internal energy trading and the external energy trading, which means all the PVCSs just buy or sell the energy at the same price and are not concerned about the who the sellers (power plant or other PVCSs) and buyers (utility grid or other PVCSs) are.

According to (9), the unit price of energy trading should be:

$$\begin{array}{c}\hfill P{r}_b\left(T\right)=a^{\prime }·NL\left(T\right)+b^{\prime }\end{array}$$

(18)

Besides the economic profit, the PVCS must take two aspects of risks into consideration. One is the risk of low-level local consumption of the PV energy, which cannot take full advantage of the EVs, and the indirect carbon emissions of the EVs are still non-negligible. The other is the risk of severe negative impacts on the grid, which means the charging activities of the PVCSs put much pressure on the transmission and distribution network. These two risks are evaluated in this paper through the differences between the charging load and the PV energy:

$$\begin{array}{c}\hfill d^k\left(T\right)={(C{E}^k\left(T\right)-r^k\left(T\right)·p{v}^k\left(T\right))}^2\end{array}$$

(19)

where $r^k\left(T\right)$ is the risk coefficient of PVCS k at time slot T, which is obtained from the real-time and historical data to reflect the best ratio between the charging load and the PV energy.

If the charging load is much larger than the PV energy, i.e., $C{E}^k\left(T\right)\gg r^k\left(T\right)·p{v}^k\left(T\right)$, that means most of the energy consumed by the EVs is from the power plant, i.e., the fossil fuel, and this situation is obviously not desired. If the PV energy is much larger than the charging load, i.e., $C{E}^k\left(T\right)\ll r^k\left(T\right)·p{v}^k\left(T\right)$, that means most of the PV energy is just sold and little PV energy is locally consumed. In addition, the absolute value of the PVCS netload is large whether $C{E}^k\left(T\right)\gg r^k\left(T\right)·p{v}^k\left(T\right)$ or $C{E}^k\left(T\right)\ll r^k\left(T\right)·p{v}^k\left(T\right)$, which implies the PVCS will put much pressure on the grid in both situations. Therefore, the most desired situation for the PVCSs is that $C{E}^k\left(T\right)$ is close to $r^k\left(T\right)·p{v}^k\left(T\right)$.

Combining the economic profit and the other two factors, the objective function of all the PVCSs should be:

$$\begin{array}{c}\hfill maxpr{o}^k\left(T\right)=pr{o}_{eco}^k\left(T\right)-w·d^k\left(T\right)\end{array}$$

(20)

where w is the weight factor of the two risks.

4.3. Learning-Based Decision-Making

The risk coefficient $r^k\left(T\right)$ is applied to adopt off-line learning and optimization to make better real-time decisions [11]. After the PVCSs operate for some days, the operation data, such as the arrival/departure time of EVs, the initial/objective SOC of EVs, and the PV output, are all collected and the off-line optimal risk coefficients of these days can be obtained by off-line optimization. To be specific, the following off-line data in these days should be figured out and the knowledge base can be established using the off-line data in these days.

$$\begin{array}{c}D{t}_{q,off}^k=[{\mathbf{r}}_{q,opt}^k,{\mathbf{CE}}_{q,min}^k,{\mathbf{CE}}_{q,max}^k,{\mathbf{pv}}_q,Pr{o}_q^k]\\ K{B}^k=D{t}_{q,off}^k,q=1,2,...,Q.\end{array}$$

(21)

where ${\mathbf{r}}_{q,opt}^k$ is the optimal risk coefficients of PVCS k in day q; ${\mathbf{CE}}_{q,min}^k,{\mathbf{CE}}_{q,max}^k$ are the lower and upper bounds of PVCS k in day q; ${\mathbf{pv}}_q$ is the PV energy of PVCS k in day q; $Pr{o}_q^k$ is the set of some other properties of PVCS k in day q; and Q is the total number of the similar days.

Therefore, a learning model based on extreme learning machine with Kernel (ELMK) is built to get the $r^k\left(T\right)$ for the current day using the data from similar days.

$$\begin{array}{c}\hfill r^k\left(T\right)=ELMK(K{B}_s^k,p{v}^k\left(T\right),C{E}_{min}^k\left(T\right),C{E}_{max}^k\left(T\right))\end{array}$$

(22)

where $K{B}_s^k$ is knowledge base of PVCS k in similar days.

The properties of the similar days $Pr{o}_q^k$ and current day $Pr{o}_c^k$ must meet the following requirements:

(1)

$|Pr{o}_q^k.P_1-Pr{o}_c^k.P_1|/Pr{o}_q^k.P_1<K_{P1},q=1,2,...,Q$, which means the PV energy difference ratio of the two days is less than $K_{P1}$.

(2)

$Pr{o}_q^k.P_2=Pr{o}_c^k.P_2,q=1,2,...,Q$, which means the two days have the same weather type (sunny, rainy, cloudy, and foggy).

(3)

$Pr{o}_q^k.P_3=Pr{o}_c^k.P_3,q=1,2,...,Q$, which means the two days are both weekdays, weekends or festivals.

The parameter $K_{P1}$ can be set empirically based on different conditions.

4.4. Allocation of the Charging Load

Each PVCS needs to allocate the charging load to the EVs when $C{E}^k\left(T\right)$, i.e., the charging load of time slot T is determined. The specific strategy for energy allocation is as follows [26]:

step 1: 

For Type 1 EVs, set $C^i\left(T\right)=C_{max}^i\left(T\right)$; for Type 2 EVs, set $C^i\left(T\right)=C_{min}^i\left(T\right)$.

step 2: 

Calculate the sum of the EVs’ present charging load in each PVCS:

$$\begin{array}{c}\hfill P^k\left(T\right)=\sum _{i\in D,M}{C}^i\left(T\right)·U_n·Q_n\end{array}$$

(23)

step 3: 

If $P^k\left(T\right)<C{E}^k\left(T\right)$, increase the charging rate of all the Type 2 EVs until $P^k\left(T\right)=C{E}^k\left(T\right)$:

$$\begin{array}{c}\hfill C^i\left(T\right)=C^i\left(T\right)+\epsilon *C_{max}^i\left(T\right),i\in D,C^i\left(T\right)<C_{max}^i\left(T\right)\end{array}$$

(24)

where $\epsilon$ is the step size to increase the charging rate.

5. Distributed EM Algorithm Based on Non-Cooperative Game

5.1. Formulation of the Non-Cooperative Game

As the unit price of buying energy depends on the netload of the whole network, the decision of each PVCS has an influence on the decisions of all the other PVCSs. Thus, each PVCS in the energy trading network needs to find its best decision when the decisions of all the other PVCSs are given. This kind of energy management problem can be described as a non-cooperative game in which the PVCSs optimize their charging load as response to the electricity price [28]. The game is defined by its strategic form as:

$$G=\left\{\begin{array}{c}C{E}^1\left(T\right),C{E}^2\left(T\right),...\\ pr{o}^1\left(T\right),pr{o}^2\left(T\right),...\end{array}\right\}$$

(25)

Definition 1.

Consider the non-cooperative game G defined in (25); a set of strategies ${\mathit{CE}}^{*}\left(T\right)$ constitutes the Nash equilibrium, if and only if it satisfies the following inequality:

$$\begin{array}{c}pr{o}^k\left(T\right)\left({\mathit{CE}}^{*}\left(T\right)\right)\ge pr{o}^k\left(T\right)(C{E}^k\left(T\right),{\mathit{CE}}^{-k*}\left(T\right))\\ \forall k\in \mathit{K},\forall C{E}^k\left(T\right)\in [C{E}_{min}^k\left(T\right),C{E}_{max}^k\left(T\right)]\end{array}$$

(26)

where ${\mathit{CE}}^{*}\left(T\right)=[C{E}^{1*}\left(T\right),C{E}^{2*}\left(T\right),...C{E}^{k*}\left(T\right),...]$, ${\mathit{CE}}^{-k*}\left(T\right)=[C{E}^{1*}\left(T\right),C{E}^{2*}\left(T\right),...C{E}^{k-1*}\left(T\right),C{E}^{k+1*}\left(T\right),...]$ and $\mathit{K}$ is the set of all the PVCSs.

Theorem 1.

In a game, if the strategy spaces are nonempty compact convex subsets of an Euclidean space, and the payoff functions are continuous and quasi-concave, there exists a unique pure-strategy Nash equilibrium [29].

The payoff functions of all the PVCSs, as shown in (20), can be proved to be continuous and quasi-concave, so there must exist a unique Nash equilibrium in the game G.

Proof. 

Firstly, the payoff functions of the PVCSs are obviously continuous.

Then, for any random $C{E}_1^k\left(T\right)$, $C{E}_2^k\left(T\right)$ and $0<\lambda <1$,

$$\begin{array}{c}pr{o}^k[\lambda C{E}_1^k\left(T\right)+(1-\lambda )C{E}_2^k\left(T\right)]-[\lambda pr{o}^k\left(C{E}_1^k\left(T\right)\right)+(1-\lambda )pr{o}^k\left(C{E}_2^k\left(T\right)\right)]\\ =(a+w)·\lambda (1-\lambda ){(C{E}_1^k\left(T\right)-C{E}_2^k\left(T\right))}^2\end{array}$$

(27)

For $a+w>0$, so,

$$\begin{array}{c}pr{o}^k[\lambda C{E}_1^k\left(T\right)+(1-\lambda )C{E}_2^k\left(T\right)]>[\lambda pr{o}^k\left(C{E}_1^k\left(T\right)\right)+(1-\lambda )pr{o}^k\left(C{E}_2^k\left(T\right)\right)]\end{array}$$

(28)

Thus the payoff functions of all the PVCSs are concave. As all concave functions are quasi-concave [30], the payoff functions are continuous and quasi-concave. Therefore, the Nash equilibrium in game G has been proven to be unique according to Theorem 1. ☐

5.2. Distributed Solving Method for the Game

The procedures of the demand response in the energy trading network are as follows:

step 1: 

The coordinator sends the information about the cost of buying energy ($a^{\prime }$ and $b^{\prime }$) to all the PVCSs in the network;

step 2: 

The PVCSs determine their $C{E}^k\left(T\right)$ and calculate their $N{L}^k\left(T\right)$, then send $N{L}^k\left(T\right)$ to the coordinator;

step 3: 

The coordinator calculates the total netload $NL\left(T\right)$ and sends the it to all the PVCSs;

step 4: 

All the PVCSs figure out their own optimal charging load $C{E}^{k\prime }\left(T\right)$ according to $NL\left(T\right)$ assuming that the charging loads of all the other PVCSs ${\mathbf{CE}}^{-k}\left(T\right)$, i.e., the netload of all the other PVCSs $N{L}^{-k}\left(T\right)=NL\left(T\right)-N{L}^k\left(T\right)$, are not changed;

step 5: 

Repeat step2 to step4 until $C{E}^{k\prime }\left(T\right)=C{E}^k\left(T\right),\forall k\in \mathbf{K}$.

A two-level distributed demand response (DR) algorithm (see details in Algorithm 1) is developed to achieve the above procedures. And the objective functions of the two-level algorithm are given by:

$$\left\{\begin{array}{c}min{F}_{outer}=\sum _{k\in K}|pr{o}_{outer}^k\left(T\right)-pr{o}_{inner}^k\left(T\right)|\\ max{F}_{inner}=pr{o}_{inner}^k\left(T\right)\end{array}\right.$$

(29)

where $pr{o}_{outer}^k\left(T\right)$ is the profit of the outer-level and it is calculated by $C{E}^k\left(T\right)$ in step2; $pr{o}_{inner}^k\left(T\right)$ is the profit of the inner-level and it is calculated by $C{E}^{k\prime }\left(T\right)$ in step4.

Algorithm 1 Two-level distributed DR algorithm

Initialization:  1. Set $j=1$ and randomly generate $C{E}_j^k\left(T\right)$ by: $$\begin{gathered} \begin{array}{c}C{E}_j^k\left(T\right)=random[C{E}_{min}^k\left(T\right),C{E}_{max}^k\left(T\right)],\forall k\in \mathbf{K} \\ {\mathbf{CE}}^{opt}\left(T\right)={\mathbf{CE}}_j\left(T\right)\end{array} \end{gathered}$$  2. Given a very small termination error $\epsilon$.  3. Calculate initial $pr{o}_{j,outer}^k\left(T\right),\forall k\in \mathbf{K}$ according to (20) using $C{E}_j^k\left(T\right)$.  4. Get initial $pr{o}_{j,inner}^k\left(T\right),\forall k\in \mathbf{K}$ using inner level.  5. Calculate initial $F_{j,outer}$ according to (29).  Iteration:loop  Outer level:  1. Update $C{E}_{j+1}^k\left(T\right),\forall k\in \mathbf{K}$ using DE according to: $$\begin{array}{c}{\mathbf{CE}}_{j+1}\left(T\right)=\underset{{\mathbf{CE}}_{j+1}\left(T\right)\in [{\mathbf{CE}}_{min}\left(T\right),{\mathbf{CE}}_{max}\left(T\right)]}{argmax}{F}_{j+1,outer}\end{array}$$  2. Update $pr{o}_{j+1,outer}^k\left(T\right),\forall k\in \mathbf{K}$.  3. Update $pr{o}_{j+1,inner}^k\left(T\right),\forall k\in \mathbf{K}$ using inner level.  4. Update $F_{j+1,outer}$ according to (29) and: $$\begin{array}{c}if{F}_{j+1,outer}<F_{j,outer},{\mathbf{CE}}^{opt}\left(T\right)={\mathbf{CE}}_{j+1}\left(T\right)\end{array}$$  5. If $F_{j+1,outer}<\epsilon$, break.  Inner level:  1. Update $C{E}_{j+1}^{k\prime }\left(T\right),\forall k\in \mathbf{K}$ according to: $$\begin{array}{c}C{E}_{j+1}^{k\prime }\left(T\right)=\underset{C{E}_{j+1}^k\left(T\right)\in [C{E}_{min}^k\left(T\right),C{E}_{max}^k\left(T\right)]}{argmax}{F}_{j+1,inner}\end{array}$$  where ${\mathbf{CE}}_j^{-k}$ is constant.  2. Update $pr{o}_{j+1,inner}^k\left(T\right),\forall k\in \mathbf{K}$ using ${\mathbf{CE}}_{j+1}^{k\prime }\left(T\right)$.

6. Case Study

6.1. Basic Data

For numerical case studies, we consider 60 PVCSs in a neighboring region which are interested in participating in the network to purchase energy together as a large consumer. Each station in this network is equipped with a PV resource and 50 chargers, i.e., 50 parking lots. The capacities of the PV resources in these stations vary from 30 kWp to 50 kWp and the PV output of five stations in a typical day is shown in Figure 2. It is important to note that the length and width of common parking lots are 2.5 and 5.3 m, respectively, thus the area of one parking lot is 13.25 m${}^2$. That means all the PVCSs have at least a 662.5 m${}^2$ rooftop to install photovoltaic battery panels. At present we generally think that 10 kWp photovoltaic battery panels will roughly cover an area of 100 m${}^2$ considering some interspaces between battery panels. Hence, the photovoltaic battery panels of each PVCS need to cover an area of 300 m${}^2$ to 500 m${}^2$. Therefore, there are enough spaces for each PVCS to build up PV generation and other facilities such as power electronics convertors and so on.

The arrival/departure time of the EVs is randomly generated according to the probability distribution function (PDF) in [31]. The daily driving distances of the EVs are also randomly generated based on the National Household Travel Survey (NHTS) [32]. Then, the initial SOC of EVs can be figured out according to the daily driving distances. The arrival/departure time and initial SOC of 50 EVs in one station are illustrated in Figure 3. In addition, the parameters of the EV battery is presented in

Table 1. Parameters of the EV battery.

ObjectiveSOC	RatedCapacity	Maximum Charge Rate	RatedVoltage	EnduranceMileage
0.95	178 Ah	0.5 C	320 V	240 km

.

6.2. Comparisons of the Effect with the Time of Use (TOU) Prices Model

Based on the basic data, numerical simulations are performed to analyze the effect of the optimization model. In order to highlight the advantage of the proposed model in pricing and EM, the common time of use (TOU) prices model [33] is chosen to draw some comparisons. The total netloads of the network obtained from these two models are illustrated in Figure 4. Note that the weight factor w in objective function is set to be 0.07 (we assume that all the PVCSs have the same weight factor for simplicity). Besides, the charging service price $P{r}_s$ and the unit subsidy for PV generation $S_{pv}$ in (17) are set to be 2.5 CNY/kWh and 0.42 CNY/kWh, respectively. As shown in Figure 4, all the PVCSs tend to intensively charge the EVs in valley period by using the TOU prices model. Thus, the new peak periods are generated because the electricity prices are at high level when PV energy is adequate so a great part of PV energy is sold to the utility grid. This kind of scheduling leads to low-level self-consumption of PV energy and consequently has great negative effects on the grid. However, the proposed demand response model can take full advantage of the PV energy and hence the netload of the proposed model is better proportioned, which can obviously lessen the undesired impacts on grid.

The prices of the two models are illustrated in Figure 5, which shows that the prices of the proposed model are lower in almost all the time slots. The result indicates that the proposed model can help the PVCSs with their further cost reduction compared with the TOU prices model.

Several indexes of the network and the economic profits of five individual stations are shown in

Table 2. Indexes of the network.

Indexes	PPTCL	Average Price (CNY/kWh)	Economic Profit (CNY)	Load Rate
TOU model	46.63%	0.8761	59,994	0.1061
Proposed model	75.56%	0.7153	63,798	0.4186
Increase/decrease value	28.93%	0.1608	3804 (6.34%)	0.3125

PPTCL refers to the percentage of PV energy in the total charging load.

and

Table 3. Economic profits of five stations (CNY).

Number of the Stations	1	2	3	4	5
TOU model	844.9	972.5	1050.8	952.1	828.9
Proposed model	907.8	1018.6	1118.6	994.3	856.5
Increase rate	7.44%	4.74%	6.45%	4.43%	3.33%

to make the comparisons of the two models clearer.

According to the above results in the two tables, it is obvious that the economic benefits, including the total profit of the network and the profits of individual PVCSs, are increased with the help of the proposed method. In addition, the load rate and PPTCL are both improved because the environmental benefit is taken consideration into the model.

6.3. Sensitivity Analysis

Considering that different values of the weight factor w in objective function will lead to different optimization results, sensitivity analyses are conducted to see the impacts of varying w on simulation results. We perform numerical simulations by changing the values of w from 0.05 to 0.09 and the w values of all the PVCSs in the network are assumed to be the same. The situation where $w=0.07$ has already been discussed in the previous section and the optimization results of the other four situations are as follows: the total netload of these different situations are illustrated in Figure 6 and other indexes are compared in

Table 4. Comparisons of indexes with different w values.

Item	w = 0.05	w = 0.06	w = 0.07	w = 0.08	w = 0.09
Average price (CNY)	0.7756	0.7368	0.7153	0.7189	0.7229
Total economic benefit (CNY)	62,833	63,358	63,798	63,726	63,593
Peak netload (kW)	1148.6	1043.8	951.2	1061.3	1301.5
PPTCL (%)	73.08	74.24	75.56	76.68	77.59
Load rate	0.3357	0.3748	0.4186	0.3675	0.2985

.

From the results of sensitivity analyses, we can see that the values of PPTCL increase with the weight factor w, while the other indexes all reach their extreme values when $w=0.07$. The reason is explained as follows. When w is small, the economic profit plays a leading role in the objective function so the PVCSs tend to charge more EVs in every time slot to get higher economic profit. Therefore, the netload is relatively high in the early time slots and then there is not much of a charging requirement in the subsequent time slots. On the contrary, when w is large, the proportion of the risk factors increases with w and thus the charging power needs to be reduced to keep pace with PV output according to the objective function, which leads to higher-level self-consumption of PV energy. However, due to the limited charging load in the early time slots, the PVCSs must purchase plenty of energy at dusk to guarantee that the EVs can achieve their objective $SOC$ before they leave, which results in an undesired netload peak at dusk and the netload peak has severe negative impacts on both load rate and economic profit of the network. As a consequence, the weight factor w should be set to be 0.07 under this circumstance, but in reality, different PVCSs in the network may have quite different capacities and the weight factors of each PVCS could also be different instead of being the same as our assumption. Hence, it is significant for all the PVCSs to choose appropriate weight factor according to their own load requirement and PV output.

6.4. Analysis of Convergence and Practical Feasibility

In order to verify the validity of the proposed model, we choose to analyze the convergence process in several hours. As shown in Figure 7, the algorithm can achieve convergence within tens of iterations in these hours, which indicates that the algorithm is feasible and efficient. In addition, we use a computer with Intel Core i7-4790 CPU 3.6 GHZ, 8 G memory, and Matlab 2014a (version 8.3.0.532, MathWorks, U.S.) as the testing environment for the algorithm to analyze the practical feasibility of the proposed method. By using the proposed method in this paper, each PVCS just needs to submit their planned netload to the coordinator, and in response, the coordinator needs to send the total planned netload of all the PVCSs to each individual PVCS. Therefore, the data interchanged between the individual PVCSs and the coordinator is only several bytes of data, which can be easily achieved with basic configuration of the smart grid. The average computation time for achieving the Nash Equilibrium of each time slot is 0.9446 s, and the computation complexity is O(1), which means the increase of the number of PVCSs will not enhance the computation complexity. Hence, the proposed method can be easily implemented within the context of the smart grid.

7. Conclusions

In this paper, a real-time optimization method is proposed to solve the energy management problem of the PVCS network, and the economic profit of the PVCS, the local consumption of the PV energy, and the impacts on the grid are all taken into consideration. Moreover, the FCR model at the station and EV level is built to guarantee the service quality for EVs and learning-based decision-making is designed to obtain the optimal real-time decision with the help of off-line optimization. The energy management problem with multi-party and multi-energy flow is formulated as a non-cooperative game, and then the existence and uniqueness of the Nash equilibrium was proved. In addition, the strategies of the PVCSs are modeled as the demand response (DR) activities to achieve their own optimization goals and a two-level distributed heuristic algorithm is introduced to solve the problem. The numerical results in the case study show that the economic profit of the network is increased by 6.34% compared with the common TOU prices model and the PPTCL is also promoted by 28.93%, which demonstrates that the proposed model can increase both the economic and environmental profits of the network. In addition, the load rate of the network is increased by 0.3125 as well, which indicates that the demand response of the PVCSs can also offer much help for the load shifting and load rate increasing.

Our results demonstrate that the proposed energy management method has significant potential to serve as an effective means of improving the profitability of the PVCS network and reducing the negative impacts of the EVs. As for practical application, the operation mode based on the proposed non-cooperative game can be used to help the PVCS operators to improve the operation effectiveness. Particularly, the average computation time for achieving the Nash Equilibrium of each time slot is 0.9446 s, i.e., the complexity of the proposed algorithm is low and the required computational resource is very small, which makes the algorithm suitable to be integrated into the embedded system in the PVCSs. Moreover, the algorithm is implemented in a distributed way, which has fewer communication costs.