Analysis of the Impact of Plug-In Hybrid Electric Vehicles on Residential Distribution Grids by using Quadratic and Dynamic Programming

The charging of batteries of plug-in hybrid electric vehicles at home at standard outlets has an impact on the distribution grid which may require serious investments in grid expansion. The coordination of the charging gives an improvement of the grid exploitation in terms of reduced power losses and voltage deviations with respect to uncoordinated charging. The vehicles must be dispatchable to achieve the most efficient solution. As the exact forecasting of household loads is not possible, stochastic programming is introduced. Two main techniques are analyzed in this paper: quadratic and dynamic programming. Both techniques are compared in results and storage requirements. The charging can be coordinated directly or indirectly by the grid utility or an aggregator who will sell the aggregated demand of PHEVs at the utility. PHEVs can also discharge and so inject energy in the grid to restrict voltage drops. The amount of energy that is injected in the grid depends on the price tariffs, the charging and discharging efficiencies and the battery energy content. The impact of a voltage controller embedded in a PHEV charger is regarded in this paper. A day and night tariff are applied. The charging and discharging of vehicles can respond to real-time pricing or on a price-schedule as well. Voltage control is the first step in the utilization of distributed resources like PHEVs for ancillary services.


Introduction
Hybrid electric vehicles (HEVs), battery electric vehicles (BEVs) and plug-in hybrid electric vehicles (PHEVs) are gaining popularity.HEVs and PHEVs have an extra electric motor added at their power train for propulsion purposes.Because of this electric motor, the combustion engine can be downsized which reduces the greenhouse gases.A PHEV is defined by [1] as any hybrid electric vehicle which contains at least 1) a battery storage system of 4 kWh or more used to power the motion of the vehicle, 2) a means of recharging that battery system from an external source of electricity and 3) an ability to drive at least 10 miles (16 km) in all electric mode consuming no gasoline.PHEVs may have a larger battery and a more powerful motor compared to a HEV, but their electric range is still very limited [2] and can not compete with the drive range of a conventional vehicles.
PHEVs can be charged by on-board electricity generation or plugging into standard electric outlets, so they have a connection to the grid.There are several places where the batteries of PHEVs can be recharged, e.g. on a car park, at work or at home.The focus here lies on the latter.The electrical consumption for charging PHEVs may rise up to 5% of the total electrical consumption in Belgium by 2030 [3].For a PHEV with a range of 60 miles (100 km), this amount can increase up to 8%.From the distribution system operator (DSO) point of view in a performance based regulation, there are strong incentives to minimize the power losses during charging and to avoid transformer and feeder overload.Not only power losses, but also power quality (e.g.voltage dips, unbalance, harmonics, etc . . . ) is essential to the DSO as to grid customers as well.Overnight charging can also increase the loading of base-load power plants and smoothen their daily cycle or avoid additional generator start-ups thus enhancing general efficiency [4].From the PHEV owner point of view, the batteries of the PHEV have to be charged within a certain time period, possibly at the lowest cost, so the driver can drive off with a fully-charged battery.
In [5], the uncoordinated and coordinated charging of the batteries of PHEVs are discussed.For uncoordinated charging, the vehicles start to charge immediately or after a fixed start delay.In this article, the coordinated charging is investigated.The charge profile of the batteries of PHEVs can be adopted and the profile can be determined by electricity price, frequency, power losses, voltage and owner preferences [6].This creates opportunities for intelligent or smart charging.The coordination of the charging could shift the demand to periods of lower load consumption and thus avoid high peaks in electric consumption.The idea of coordinated charging is to achieve optimal charging and grid utilization by minimizing the power losses for both deterministic and stochastic data of the household loads.For deterministic household profiles, there is a perfect knowledge of the future data.The stochastic data reflect an error in the forecasting of the daily load profiles.Two program techniques are presented to improve the grid quality for the deterministic and the stochastic approach: quadratic programming (QP) and dynamic programming (DP).Both techniques are compared in results, storage requirements and computational effort.
The connection to the electric power grid, mainly for purpose of charging the batteries for driving needs, offers more opportunities.The power flow of the charger can be bidirectional, meaning that the vehicle can also discharge and thus inject electrical energy from the battery back in the grid [6].In that way, PHEVs can support the grid.This is the vehicle to-grid-concept (V2G).For the V2G concept, a lot of vehicles must be connected to the power grid.The battery capacity of each vehicle individually is small, but the large number of vehicles make that PHEVs have enormous energy storage capacity.More than 90% of the vehicles are always available for V2G [1], [7], [8] and must be connected to the grid when idle.This can be encouraged by giving incentives to stay plugged in.The V2G-concept offers opportunities for both the vehicle owners and the distribution and transmission system operator (TSO).There is almost no storage available in the power grid nowadays so the demand and generation must be perfectly matched and continuously managed to absorb fluctuations [8].The electrical storage of PHEVs could provide grid services via V2G concept and add a surplus value to the vehicle owner [9], although, it is not clear if this would be economically viable.Grid operators and vehicle owners have complementary needs [7].The PHEV-owner needs energy for driving at more or less predictable times and the grid operator is responsible for power balance at each time instant.In this research is shown that vehicles can each individually support the grid by implementing a voltage controller embedded in the charger.

Assumptions and modeling 2.1 Load scenarios
From an available set of residential load measurements, two large groups of daily winter and summer load profiles are selected.The load profiles cover 24 hours and the instantaneous power is given on a 15 minute time base as shown in Fig. 1 for an arbitrary day during winter.

Specifications of PHEVs
Each of the PHEVs has a battery with a maximum storage capacity of 11 kWh [4].Only 80% of the capacity of the battery can be used to optimize life expectancy.This gives an available capacity of 8.8 kWh. 10 kWh is required from the grid, assuming an 88% energy conversion efficiency from AC power absorbed from the grid to DC power in the battery of the vehicle [11].The batteries can be charged and discharged, meaning that the energy flow is bidirectional.The charger has a maximum output power of 4 kW for both directions.The charger of 4 kW is chosen because the maximum power output of a standard single phase 230 V outlet is 4.6 kW.So this is the largest charger that can be used for a standard outlet at home without reinforcing the wiring.The maximum penetration degree is 30% by 2030 for Belgium as predicted by the Tremove model [12].

Charging periods
It is not realistic to assume that PHEVs could be connected at any place where a standard outlet is present.Therefore in this article, the batteries of the vehicles are assumed to be connected at home.Fig. 2 shows the percentage of all trips by vehicle each hour on average.At that moment, they are not available for connection.An important period is during the evening and night.Most of the vehicles are at home from 21h00 until 06h00 in the morning.Other possible periods are not considered in this paper because the focus lies on a connection at home, in distribution grids, but the proposed methods are also valid for other periods.

Grid topology
The radial network used for this analysis is the IEEE 34 node test feeder [14] shown in Fig 3.This network is downscaled to 230 V so this grid topology represents a residential radial network.The line impedances are adapted to achieve tolerable voltage deviations and power losses.Each node is a connection with a residential load and some of the connections which are randomly chosen, will have PHEVs charging or discharging.

Quadratic programming
The coordinated charging handles a unidirectional energy flow from the grid to the batteries.Discharging is not implemented, so in this case, the vehicles are not able to support the grid by injecting energy into the grid.The idea of coordinated charging by minimizing power losses is to optimize the grid utilization and quality.This optimization problem can be tackled by quadratic programming.This technique optimizes a quadratic function of several variables, in this case the power of the PHEV chargers at all time steps, which are subjected to linear constraints.The QP technique is applied to handle deterministic and stochastic household load profiles.

Optimization problem
By optimizing grid utilization, the owners of PHEVs will no longer be able to control the charging profile.The only degree of freedom left for the owners is to postulate the point in time when the batteries must be fully charged.For the sake of convenience, the end of the indicated charging period is taken as the point in time when the vehicles must be fully charged.The charging power varies between zero and maximum charger output.

Methodology
The objective is to minimize the power losses which are treated as a reformulation of the non-linear power flow equations.This non-linear minimization problem can be tackled as a sequential quadratic optimization [15].The charging power obtained by the quadratic programming can not be larger than the maximum power of the charger P max .The batteries must be fully charged at the end of cycle, so the energy which flows to the batteries must equal the capacity of the batteries C max .x n is zero if there is no PHEV placed and is one if there is a PHEV at node n.The goal is to minimize power losses while taking into account these constraints.The quadratic programming uses equations ( 1) and (2).The vehicles are randomly placed after the selection of a daily load profile and the number of PHEVs.A flat voltage profile is assumed and the node voltages are computed with the backward-forward sweep method assuming that there are no PHEVs.The backward and forward sweep are formulated as a matrix multiplication.

Deterministic programming
The quadratic optimization is performed in order to determine the optimal charging profile.Next, the node voltages are computed again.This process is repeated until the power loss based stopping criterion is reached.

Stochastic programming
The analysis of the previous paragraph are based on deterministic or historical data for the daily load profiles.So the essential input parameters, such as the household load profiles, are fixed.For this approach, sufficient measurement data must be available.Most of the time, however, these measurements are not adequate to do a perfect forecasting of the data.A stochastic approach, in which an error in the forecasting of daily load profiles is considered, is therefore more realistic.
The daily load profiles are the essential input parameters.The uncertainties of these parameters can be described in terms of probability density functions.In that way, the fixed input parameters are converted into random input variables with normal distributions assumed at each node.N independent samples of the random input variable ω i , the daily load profile, are selected.Equation (3) gives the estimation for the stochastic optimum vn .The function g P n,t , ω i gives the power losses and P n,t is the power rate of the charger for all the PHEVs and time steps.fN is a sample-average approximation of the objective of the stochastic programming problem.
The mean value of the power losses, E (v n ), is a lower bound for the real optimal value of the stochastic programming problem, v * [16], as shown in (4).
E (v n ) can be estimated by generating M independent samples ω i,j of the random input variable each of size N .M optimization runs are performed based on (3).vj n is the mean optimal value of the problem for each of the M samples as shown in (5).The optimal values of the M samples constitute a normal distribution.
From equation ( 6), L N,M is an unbiased estimator of E (v n ).
Simulations indicate that in this type of problem, the lower bound converges to the real optimal value when N is sufficiently high.A forecasting model for the daily load profile for the next 24 hours is required.The daily load profiles of the available set are varied by a normal distribution function.The standard deviation σ is determined in such a way that 99.7 % of the samples vary, at maximum, 5 or 25 % of the average μ.
For 2000 independent samples of the daily load profile, one optimal charging profile is calculated.This optimal charging profile is used to determine the power losses for the 2000 individual load profiles.This is the stochastic optimum.For each of these 2000 load profiles, the optimal charging profile and the corresponding power losses are also computed, which is the deterministic optimum.The power losses of the deterministic optimum are subtracted from the power losses of the stochastic optimum and divided by the deterministic optimum, defined as ΔP .This is shown for a variation of the household loads of 5 and 25% in Fig. 5 and 6 respectively.The value of this difference is always positive.The forecasting of the daily load profiles introduces an efficiency loss because the charge profiles of the PHEVs are not optimal for this specific daily load profile.If the standard deviation of the normal distribution and thus the variation of the household load is reduced, the 2000 charge profiles of the deterministic optimum will converge to the optimal charge profile.The efficiency loss will also reduce indicating that the power losses of the differences will go down by a factor 25 as shown in Fig. 5  In general, the difference between the power losses of the stochastic and the deterministic optimum is rather small.It is clear that the error in forecasting does not have a large impact on the power losses.The daily household load profiles during the winter season are showing the same trend each day during winter season resulting in an optimal charge profile which resembles a deterministic charge profile of a specific day as shown in Fig. 7 for the last node of the test grid.Both charge profiles have the same trend.Therefore, the contrast in terms of power losses between the deterministic and stochastic optimum is not large.However, the difference between the uncoordinated and coordinated charging is much larger because the charge profiles are more different.The uncoordinated charging has a constant charge profile for a specific amount of time and the power of the charger can not be varied.In Fig. 5 and 6, a specific household load profile is assumed which is varied by a normal distribution function.In Fig. 8, the load profiles are randomly selected out of a database of household load profiles.This database contains profiles that differ more each day and are more peaked which increases the efficiency losses significantly.

Dynamic programming
The optimal coordination of charging PHEVs can also be tackled by the dynamic programming technique (DP).The QP and DP techniques are compared with respect to results, storage requirements and computational time.The DP technique decomposes the original optimization problem into a sequence of subproblems which are solved backward over each stage.A classical implementation of the DP technique is the shortest path problem.For the application of this paper, the model is represented as a series of plug-in hybrid electric vehicles.

Optimization
There are Q vehicles with charging batteries and the maximum value of Q corresponds to a penetration degree of 30%.The battery content of these Q vehicles at each stage are the Q state variables S t,i .The number of stages T is the number of hours of the charging period multiplied by four because the household loads are available on a 15 minute time base.The backward recursive equations for the conventional dynamic programming technique are given in ( 7) and (8).
The function f t represents the total optimal power losses from period t to the last period T .The vector S t is a Q-dimensional vector.Each storage level can take R discrete values at time t.L t are the power losses during period t and S t,i is the battery content of the i th vehicle at time stage t.The power of the chargers is represented by P t and is also a Q-dimensional vector.So the first component of this vector gives the power of the charger for the first PHEV.The output of the charger is not continuous, but has a step size of 400 W.This is relatively large, but smaller step sizes would lead to too much computational time which is proportional to R T [17].The constraints of the problem remain the same and are shown in (9).
The power loss objective function is to minimize.The storage vector S t is a Q-dimensional vector and thus "the curse of dimensionality" [18] arises which is handled by modifying the original dynamic programming technique.The dynamic programming successive approximation (DPSA) technique decomposes the multidimensional problem in a sequence of one-dimensional problems which are much easier to handle [19].The optimizations occur one variable at a time while holding the other variables at a constant value.All the variables are evaluated that way.This technique converges to the global optimum for convex problems.This method will be used for the deterministic and stochastic programming.

Deterministic programming
A daily load profile of the selected season is chosen and the vehicles are placed randomly.The DPSA technique needs initial values of the state variables to start the iteration.These values are generated by calculating the optimal charge trajectory for each PHEV separately without considering the other PHEVs.These optimal trajectories are put together into one temporary optimal trajectory and thus one Q-dimensional state vector.All the components of the state vector are held constant except the first one.The optimal charge trajectory for the first component of the state variable is defined.The new value is ascribed to the first component and the procedure continues until the last component of the state vector is optimized.This procedure is repeated until convergence is obtained.The problem is switched from a multidimensional problem to a sequence of one-dimensional problems.The algorithm of dynamic programming successive approximation is represented in Fig. 9.

Stochastic programming
The uncertainties of the household loads must also be implemented in the DP technique.2000 stochastic household load profiles are generated and the mean power losses of these loads are used to determine the total power losses f t as presented in (10).
The same stochastic load profiles as produced in the stochastic programming of the QP technique are applied to make the comparison more clear.
One optimal charge profile is generated for these 2000 stochastic household loads with the DPSA technique.The power losses are calculated separately for the 2000 household load profiles and the single optimal charge profile.This is the stochastic optimum.For the deterministic optimum, the optimal charge profile and power losses are determined for each of the 2000 stochastic household load profiles, giving 2000 optimal charge profiles.The power losses of the deterministic optimum are subtracted from the power losses of the stochastic optimum and divided by the deterministic optimum for a variation of the household loads of 5 and 25%.

Results
In general, the difference between the results of the DP and QP techniques is negligible although the QP technique gives more accurate results because the values of the charge profile are continuous, in contrast to the DP technique where a step size of 400 W is introduced for the power of the charger, giving a discrete charge profile.In Fig. 10, the charge profiles for the QP and DP technique are compared.Reducing the step to an infinitesimal value would give the same result as the QP technique.This step size is taken rather large to reduce the number of levels and with that the computational time and storage requirements.The storage requirements are heavier for the DP technique compared to the QP technique because every possible path over each stage must be stored.Since this leads to very large matrices and increased computational time, the DP technique is slower.Therefore, the QP technique preferred above the DP technique for the further analysis.

Voltage control by PHEVs
The aim of this section is to emphasize the impact on the grid of implementing a voltage controller.PHEVs with a bidirectional connection to the power grid can deliver ancillary services although PHEVs are not technical and economically suitable for all kind of ancillary services.The vehicles respond quickly, but they have a high cost per kWh and the battery capacity is rather limited, so the duration of the services must be short.The ancillary services are not handled in this paper but are studied in [8] and [20].The charging of PHEVs increases the total load of the distribution grid considerably [5].The extra loads cause an increase of power losses and voltage deviations.The voltage deviations can be reduced to meet partially the EN50160 standard by embedding a voltage controller in a PHEV charger.This could even be made mandatory because the grid reliability must be assured.In the test grid of this paper, no voltage deviations occur in the case with no PHEVs present in the grid.Therefore, this support is not considered as an ancillary service, but is a first step in the direction of supporting the grid by PHEVs.

Optimization problem
The utilization of the power grid is maximized by the optimization of the bidirectional charge profile.The objective function is now a linear cost function so a linear programming technique (LP) can be used.This optimization problem minimize the charging cost for the consumers or PHEV owners while the voltage in the nodes must stay within their limits.The test grid and the charging period stay the same.

Methodology
The objective function is a cost function which must be minimized as shown in (11).This function is very simple and has only two constants: one constant represents the tariff during the day C day and a one constant is the tariff overnight C night .The ratio of the day constant to the night constant is estimated to be about 1.6 [21].A night tariff starts between 21h00 and 23h00 and ends between 06h00 and 08h00.In this paper, the night tariff starts at 22h00 and ends at 07h00.The constraints of (2) are kept and new constraints are added as shown in (12).The vehicles are now also able to discharge so the charger output varies between -4000 and 4000 W. The discharge efficiency is also taken into account, which is 88%.The capacity of the batteries, C n,t , must be between zero and C max for each time step and equals C max at the end of the charging period, i.e. 06h00.The voltage must satisfy the EN50160 standard so the node voltages V n,t at each time step must be higher than 90% of 230 V, which is V limit .The goal is to minimize this cost function while fulfilling the constraints.

Results
The results are represented for a day of the winter season with high peak loads.The new constraints are added separately to distinguish their impacts.For Fig. 11, the vehicles are not able to discharge and no voltage constraint is implemented.The objective function is simplified and a single tariff is used, making no distinction between night and day.The charge profiles for a node at the end of the IEEE test grid are showed in Fig. 11 for three different penetration degrees.Because the objective is no longer to minimize power losses and a single tariff is assumed, the vehicles are charging randomly during this period.Fig. 12 shows the voltage profiles for the same node.Because no voltage constraint is implemented, the voltage goes well below the voltage limit.Therefore, a voltage constraint is added at the linear programming.The charge profile is shown in Fig. 13 also for a node at the end of the test grid.The vehicles will not be charging on the moment the voltage is already low due to the household loads.The cost function stays the same so the vehicles are randomly charging between 21h00 and 06h00, satisfying an extra constraint: the voltage constraint.The discharging of the vehicles is implemented EVS24 International Battery, Hybrid and Fuel Cell Electric Vehicle Symposium in the program and the objective function has two tariffs.However, the vehicles are not discharging as shown in Fig. 15.The charging period starts at 21h00 and thus there is only one hour left to discharge at peak tariff.This is not happening because the batteries of the PHEVs are assumed to be empty at the start of the charging period and charging and discharging at the same cost will be uneconomical because of the charge and discharge efficiencies.Because there is no other objective, and there are only two cost tariffs, the vehicles are further randomly charged at night tariff.There is no incentive to reduce the power losses.In the previous argumentation, it is assumed that the batteries are empty at the beginning of the charging period.For the next model, there is energy left in the batteries at the start of the charging period.This energy is stochastic determined by a Gauss curve with an avarage of zero and a σ of 1000 W. Fig. 16 shows the charge profiles of a node at the end of the test grid for different penetration degrees.The night tariff starts at 22h00, therefore the vehicles are discharging between 21h00 and 22h00 depending on the energy left in the battery.The batteries still must be fully charged at the end of the period.The impact of the energy left in the battery at the beginning of the charging period is shown in Fig. 17.The more energy left in the battery, the more the PHEVs are discharging between 21h00 and 22h00, when the peak tariff is valid.The amount of discharging is directly related to the energy left in the battery.This is shown for a penetration degree of 50%.

Conclusion
In general, coordinated charging of plug-in hybrid electric vehicles can lower power losses and voltage deviations by flattening out peak power.At the first stage, historical data is used so there is a perfect knowledge of the load profiles.In a second stage, stochastic programming is introduced to represent an error in the forecasting which increases the power losses.This efficiency loss is rather small if the trend of the household load profiles is known, so charging during the peak load of the evening can be avoided.These results are obtained by the quadratic programming technique.The dynamic programming technique is also implemented but does not improve the computational time nor the achieved accuracy.The applied techniques and methods can be extended to other objective functions.A voltage support could be implemented in the electric chargers to avoid too large voltage drops in the grid.If discharging is applied, it is only economically beneficial at the moment the peak tariff is valid.The vehicles will only discharge if some energy is left in the battery.The results are of course depending on the depicted charging period.

Figure 2 :
Figure 2: Percentage of vehicle trips at each hour on average [13].

Fig. 4
Fig.4represents the outline of the algorithm of coordinated charging.The vehicles are randomly placed after the selection of a daily load profile and the number of PHEVs.A flat voltage profile is assumed and the node voltages

compared to 6 .Fig. 5 :Fig. 6 :
Fig. 5: Histogram of the efficiency loss of an arbitrary day during winter for a variation of 5%.

Figure 7 :
Figure 7: The deterministic optimum and optimal charger profile for node 33.

Fig. 8 :
Fig. 8: Histogram of the efficiency loss of an arbitrary day during winter for other household profiles.

Figure 10 :
Figure 10: The charge profile for node 1 for the QP and the DP program technique.

Figure 11 :
Figure 11: Charge profile for different penetration degrees and no voltage constraint.

Figure 12 :
Figure 12: Voltage profile for different penetration degrees and no voltage constraint.

Figure 13 :
Figure 13: Charge profile for different penetration degrees with voltage constraint.

Fig. 14 Figure 14 :
Fig.14shows the voltage profiles if the voltage constraint is implemented.The voltage stays well above the voltage limit.

Figure 15 :
Figure 15: Charge profile for different penetration degrees.

Figure 16 :
Figure 16: Charge profile for battery capacity different from zero.

Figure 17 :
Figure 17: Charge profile for different battery capacity degrees.