Application of Artificial Intelligence for EV Charging and Discharging Scheduling and Dynamic Pricing: A Review

Abstract

The high penetration of electric vehicles (EVs) will burden the existing power delivery infrastructure if their charging and discharging are not adequately coordinated. Dynamic pricing is a special form of demand response that can encourage EV owners to participate in scheduling programs. Therefore, EV charging and discharging scheduling and its dynamic pricing model are important fields of study. Many researchers have focused on artificial intelligence-based EV charging demand forecasting and scheduling models and suggested that artificial intelligence techniques perform better than conventional optimization methods such as linear, exponential, and multinomial logit models. However, only a few research studies focused on EV discharging scheduling (i.e., vehicle-to-grid, V2G) because the concept of EV discharging electricity back to the power grid is relatively new and evolving. Therefore, a review of existing EV charging and discharging-related studies is needed to understand the research gaps and to make some improvements in future studies. This paper reviews EV charging and discharging-related studies and classifies them into forecasting, scheduling, and pricing mechanisms. The paper determines the linkage between forecasting, scheduling, and pricing mechanism and identifies the research gaps in EV discharging scheduling and dynamic pricing models.

1. Introduction

In recent years, the rollout of electric vehicles (EVs) has been accelerated in many countries to combat the energy crisis and environmental concerns such as high CO₂ emissions and climate change [1]. Around two million EVs were sold in the first quarter of 2022, an increase of 75% from the same period in 2021 [2]. The number of EVs will continue to grow with governments’ incentives and policies around the globe. On the one hand, the increasing and uncoordinated EV charging will burden the existing power grid. On the other hand, EVs’ batteries are mobile energy storage systems that can be used to provide ancillary services for power grids, such as peak-shaving and valley-filling, voltage and frequency regulations [3,4,5]. In addition, EVs’ batteries can be used as flexible load and supply to maximize renewable energy utilization, if EVs charging and discharging are properly coordinated to closely match renewable generation profiles [6,7].

The idea of supplying electricity from EVs’ batteries back to the power grid is known as vehicle-to-grid (V2G). This idea was first introduced by Kempton and Letendre in 1997 [8]. Using EVs as energy storage through the V2G program can bring many environmental, economic, and social-technological benefits to program participants, such as EV owners, grid operators, government, and aggregators [9]. For example, the V2G services can reduce the total ownership cost of EVs for EV owners by allowing them to sell stored energy back to power grids during peak hours [10]. Furthermore, the V2G program can reduce power grid congestion, emissions, and increase renewable energy utilization for grid operators by shifting EV charging to a high renewable generation period and discharging back to power grids during a low renewable energy generation and high load demand period [11,12].

However, the penetration of EVs is currently still low, and the concept of V2G is relatively new and evolving. Many V2G projects remain in the pilot project stages of development [13,14]. Some of the representative V2G pilot projects with different goals and services are summarized in

Table 1. Summary of some significant V2G pilot projects worldwide [14].

Project Name	Country	No. of Chargers	Timespan	Service
Realising Electric Vehicle to Grid Services	Australia	51	2020–2022	Frequency response, reserve
Parker	Denmark	50	2016–2018	Arbitrage, distribution services, frequency response
Bidirektionales Lademanagement—BDL	Germany	50	2021–2022	Arbitrage, frequency response, time shifting
Fiat-Chrysler V2G	Italy	600	2019–2021	Load balancing
Leaf to home	Japan	4000	2012–ongoing	Emergency backup, time shifting
Utrecht V2G charge hubs	Netherlands	80	2018–ongoing	Arbitrage
Share the Sun/Deeldezon Project	Netherlands	80	2019–2021	Distribution services, frequency response, time shifting
VGI core comp. dev. and V2G demo. using CC1	South Korea	100	2018–2022	Arbitrage, frequency response, reserve, time shifting
SunnYparc	Switzerland	250	2022–2025	Time shifting, pricing scheme testing, reserve
Electric Nation Vehicle to Grid	UK	100	2020–2022	Distribution services, reserve, time shifting
OVO Energy V2G	UK	320	2018–2021	Arbitrage
Powerloop: Domestic V2G Demonstrator Project	UK	135	2018–ongoing	Arbitrage, distribution services, emergency backup, time shifting
UK Vehicle-2-Grid (V2G)	UK	100	2016–ongoing	Support power grid
INVENT—UCSD/Nissan/Nuvve	US	50	2017–2020	Distribution services, frequency response, time shifting
SmartMAUI, Hawaii	US	80	2012–2015	Distribution services, frequency response, time shifting

. A complete list of V2G pilot projects around the world can be found in [14]. As shown in Table 1, most of the V2G pilot projects were only initiated in recent years, and the scale of these pilot projects is small. Moreover, most of the existing literature only focuses on the EV charging aspect. For example, the classification of EVs’ charging techniques is discussed in [10,15,16]. Authors in [17,18,19,20,21,22,23,24,25,26,27,28,29,30] designed EV charging strategies to minimize charging costs. In [31], Al-Ogaili et al., reviewed EVs’ charging scheduling, clustering, and forecasting techniques in the existing literature. Optimal charging strategy for electric vehicles under dynamic pricing schemes, including Real-Time Pricing (RTP), Time of Use (ToU), Peak Time Rebates (PTR), and Critical Peak Pricing (CPP), has been reviewed in [32]. A few studies focused on different aspects of V2G, such as customer acceptance of V2G [33], integration of renewable energy into transportation systems via V2G [11], the economic feasibility of V2G for a university campus [10], coordination of V2G with energy trade [12], and optimal energy management systems [34]. Encouraging EV owners to participate in the V2G program without monetary incentives is difficult because it costs energy and time to feed electricity back to power grids. Therefore, the economic and operational aspects of V2G will become very important when the penetration of EVs with V2G capability is high. Many EV charging scheduling approaches have been effective in following ToU and RTP signals for peak demand reduction [35], alleviating the impact of load fluctuation [36], and reducing EV charging costs [17,18,19,20,21,22,23,24,25,26]. However, the effectiveness of these pricing policies in reflecting power system conditions has barely been discussed in the literature.

Moreover, the economic and operational aspects of the V2G program, such as discharging scheduling and pricing mechanism, have only been proposed in a few papers [27,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51]. Al-Ogaili et al. [31] suggested that artificial intelligence models perform better than probabilistic models. With the availability of the dataset and increasing computational power, artificial intelligence algorithms such as neural networks and reinforcement learning have become very popular and effective for many applications, including forecasting and optimization problems. The learning ability from datasets of artificial intelligence models makes them superior to conventional optimization models, such as linear and exponential optimization models. Moreover, artificial intelligence models usually do not require expert knowledge of complex systems, which might be challenging to obtain. As a result, artificial intelligence models have been used in many EVs related studies ranging from EV battery design and management to V2G applications [52]. A review of the different roles that artificial intelligence plays in the mass adoption of EVs can be found in [52]. Shahriar et al. [53] reviewed machine learning-based EV charging behavior prediction and classification and pointed out the potential of reinforcement learning for EV scheduling. In addition, many artificial intelligence models have been implemented to solve EV charging scheduling-related tasks such as predicting EV charging electricity price [5,20,22,54,55,56,57], EV driving patterns [58], aggregated available capacity of batteries [59,60,61,62], charging load demand [63,64,65,66,67], charging pattern [68], and EV charging/discharging scheduling [28,68,69,70,71,72,73]. However, the linkage and gaps between each study have not been comprehensively discussed in the existing literature. Thus, this review aims to explore artificial intelligence-based forecasting, scheduling, and dynamic pricing models. In addition, the relationship between forecasting, scheduling, and dynamic pricing is also discussed in this paper. Moreover, the research gaps found in the existing literature are mentioned, and future research direction related to EV charging and discharging is discussed.

The main contribution of this paper is to compare, summarize and analyze the existing artificial intelligence-based algorithms of three critical EV charging/discharging components: forecasting, scheduling, and dynamic pricing. This paper also points out the research gaps in effective EV discharging scheduling and pricing policy based on the finding of the existing literature.

The rest of the paper is structured as follows. Section 2 briefly discusses the EV charging/discharging control techniques, the concept of V2G, and battery degradation. Section 3 summarizes different artificial intelligence-based models for predicting EV charging-related tasks such as electricity price and EV charging load demand. Artificial intelligence-based EV charging/discharging scheduling models are reviewed in Section 4. Section 5 discusses dynamic pricing and peer-to-peer energy transaction strategies. Discussion of the current advanced technology and research gaps are presented in Section 6. Finally, conclusions and future research directions are presented in Section 7.

2. EV Charging/Discharging and Battery Degradation

2.1. EV Charging and Discharging Techniques

There are four charging/discharging techniques, namely, uncontrolled charging-discharging [10,15], controlled charging-discharging [10,16], smart charging [10], and indirectly controlled charging [10].

Table 2. Summary of different EV charging and discharging techniques.

Techniques	Benefits	Challenges
Uncontrolled	Easy implementation EV owners have the freedom to make charging decisions Convenience for EV owners	Add burden to power grids Charging costs might be higher than smart charging Inflexible for demand-side management
Controlled	System operators have more freedom to make decisions	EV owners have to cede control to the system operators
Smart	Balance demand and supply Make charging decisions based on the real-time conditions Maximize profits for power system operators	Hard to encourage EV owners to participate in smart charging Benefits for EV owners are usually unclear
Indirectly controlled	Use monetary terms to encourage EV owners to participate in smart charging Incentive is clear	Electricity pricing signals need to be accurate to be effective

summarizes the benefits and challenges faced by each charging/discharging technique. In addition, the schema of each charging technique is illustrated in Figure 1. The uncontrolled charging-discharging approach allows EVs to charge or discharge at rated power as soon as it is plugged in until the battery’s storage level equals the maximum state of charge or unplugged [10,15]. Thus, this charging method is inflexible for demand-side management DSM [10]. The uncontrolled charging technique is convenient for EV owners to make charging decisions freely. However, uncontrolled EV charging might cause a negative impact on local distribution networks, such as power loss, demand-supply unbalance, shorter transformer lifespan, and harmonic distortion [31].

The controlled charging-discharging method, also known as unidirectional V2G, gives system operators more freedom to decide when EVs will be charged and discharged [10,16]. However, EV owners have to cede control to the system operators or aggregators immediately after the EV is plugged in under the controlled charging-discharging strategy [74].

The smart charging technique manages EVs’ charging and discharging based on real-time energy demand, grid requirements, and grid quality [10]. However, it is not easy to encourage EV owners with different preferences to participate in smart charging programs without incentives [75]. Smart charging allows EV owners to charge or discharge their EVs at a certain time and rate to achieve predefined goals such as minimizing charging costs or balancing demand and supply. However, smart charging strategies are usually designed by system operators to maximize their profit. In addition, EV owners do not clearly understand how smart charging can benefit them monetarily.

On the other hand, an indirectly controlled charging mechanism uses more straightforward price signals to incentivize EV owners to provide ancillary services to power grids. Wang and Wang [3] suggested using macro-incentive policies such as variable electricity tariffs to attract more EV owners to participate in the V2G system [3]. Dynamic pricing schemes [76], such as Time of Use (ToU) and Real-Time Pricing (RTP), have been commonly used as a special form of power load demand response, which encourages EV owners to choose charging or discharging time according to financial incentives [4,10,37,42]. Dutschke and Paetz [77] conducted two empirical studies in Germany to get the public’s opinions about the dynamic electricity pricing determined by the Time of Use (ToU) tariff and household load profile. The results indicate that consumers are open to dynamic pricing but like more straightforward programs with a smaller price fluctuation range [77,78]. Latinopoulos et al. [79] investigated EV drivers’ responses to the dynamic pricing of parking and charging services. The results suggest that younger individuals are more likely to exhibit forward-looking behaviors. Therefore, indirectly controlled charging with dynamic electricity pricing strategies will influence EV owners’ charging and discharging behaviors.

2.2. Vehicle to Grid (V2G) Concept

The V2G technology allows EVs to utilize onboard batteries as an energy source for driving and energy storage systems for power grids [8]. Therefore, utilizing EVs’ batteries with fast charging and discharging reaction time (as fast as tens of milliseconds [16]) as energy storage and power sources via V2G technology can avoid additional investment for a battery storage system. In addition, average cars are parked 95% of the time, equivalent to the total operating hours of baseload generators [8]. Therefore, V2G technology can benefit system operators by providing ancillary services and energy storage for renewable energy. It can also reduce EV owners’ charging costs by allowing them to sell extra stored energy back to power grids. Parsons et al. [33] investigated potential EV owners’ willingness to pay for an EV with V2G capability and contract terms. The survey results suggest that the V2G concept is most likely to attract more EV buyers if power aggregators provide either upfront cash payment or pay-as-you-go basis types of contracts.

Lund and Kempton [11] modeled a power system that integrates renewable energy into the transport and electricity sectors using V2G technology. The simulation results indicate that adding EVs with V2G capability can enhance renewable energy utilization (i.e., align EV charging pattern with renewable energy generation pattern) and reduce CO₂ emissions. Scott et al. [10] simulated a V2G model for a university campus under various charging scenarios. The simulation results show that using EVs’ batteries as energy storage over ten years can reduce electricity costs by 64.7% and 9.79%, respectively, compared to purchasing electricity from the power grid and using sole battery storage [10]. Al-Awami and Sortomme [12] have formulated a mixed-integer stochastic linear programming model to coordinate V2G services with energy trading. The simulation results show that using V2G services to coordinate short-term energy trading can increase the profit of the load-serving entity (LSE) by 2.4% and reduce emissions by 5.3% compared to the uncoordinated one. The advantages and disadvantages of the V2G concept are summarized in

Table 3. Summary of the advantages and disadvantages of V2G.

Advantages	Disadvantages
Avoid additional investment in a battery storage system	Battery degradation concerns Charging-discharging efficiency concerns Additional upfront investment
Enhance renewable energy utilization, thus reducing emissions
Mobile energy storage with a fast reaction time
Provide ancillary service for power grids
Reduce charging costs for EV owners

.

Although V2G technology can potentially provide many benefits to power systems and EV owners, the implementation of V2G still faces some challenges, such as high upfront investment [34], battery degradation [8], and charging-discharging efficiency concern [80]. In addition, the V2G concept is still relatively new and evolving. Many pilot V2G projects remain in the development stages [13]. By 2018, only 2 out of 486 (0.41%) utilities investigated by the Smart Electric Power Alliance (SEPA) had implemented the V2G pilot project [81]. By 2022, there were only around 100 V2G plot projects with different scales and testing phases worldwide [14,82]. Therefore, to accelerate V2G implementation, the charging and discharging efficiency need to be improved. On top of that, the V2G payment procedure and contract terms need to be simplified. Batteries’ degradation rate and upfront investment of V2G need to be quantified to allow EV owners to make informative decisions.

2.3. Battery Degradation and Charging Efficiency

The degradation of lithium-ion batteries occurs throughout their lives due to several chemicals and mechanical processes that reduce the cyclable lithium and other active materials [83]. Battery degradation depends on many factors, such as the charging and discharging rates, depth of discharge (DOD), temperature, voltage, cycle number, and storage stage of charge, which are complex to quantify [3,34,83]. The degradation of the battery can be classified into two types: calendar aging and cycle aging. Calendar aging occurs during storage, whereas cycle aging happens during charging and discharging. Battery temperature and state of charge (SOC) are the key factors that influence calendar aging, whereas cycle aging is affected by cycle number, charging rate, and DOD [84]. Therefore, additional cycle numbers due to the V2G service will accelerate battery degradation. It is important to quantify the degradation due to V2G service. Thus, adequate battery monitoring systems are needed to monitor the batteries’ SOC and state of health (SOH) during the charging and discharging process. Meng et al. [85] performed a Lithium-Ion battery monitoring and observability analysis with an extended equivalent circuit model. As a result, the necessary observability conditions for battery capacity are clearly indicated, which can be used to aid battery charging control design. Meng et al. [86] proposed a Kalman filter and Gaussian process regression-based battery end-of-life (EOL) prediction model. The simulation results show that the proposed model provides a better battery EOL prediction than the particle filter, a popular method for battery EOL prediction. The effectiveness of Gaussian process regression on battery SOH estimation is also shown in [87]. Chaoui and Ibe-Ekeocha [88] proposed a recurrent neural networks(RNNs)-based SOC and SOH estimation model which does not require battery modeling or knowledge of batter parameters. The simulation results indicate that the recurrent neural networks-based estimation model can make good SOC and SOH estimations based on measured voltage, current, and ambient temperature. The measured data’s accuracy is vital for empirical models’ performance.

More charging/discharging cycles occur in the V2G service than when there is no V2G service; thus, battery degradation due to V2G services might be more severe than without it [84]. Petit et al. [89] assessed the impact of V2G on two types of Lithium-ion batteries, nickel cobalt aluminium oxides (NCA) and lithium ferro-phosphate (LFP). The simulation results indicate that the effects of V2G on different batteries are different. For example, NCA is more sensitive to cycle aging compared to LEP cells. In addition, high SOC can increase battery capacity loss during storage. Pelletier et al. [90] also indicated that the calendar aging process occurs faster for the battery stored at high SOC. The authors in [90,91,92] found that battery overcharging and over-discharging degradation happen when the battery operates outside its specified voltage range. Although battery degradation is inevitable at the moment, it is possible to minimize the process by avoiding overcharging/over-discharging and encouraging charging/discharging batteries at an optimal rate, under the optimal temperature range, and storing the battery at an optimal SOC. Therefore, battery degradation estimation and modeling need to be more accurate to support battery charging and discharging control design.

Besides battery degradation concerns, charging and discharging efficiency is also one of the primary concerns of the V2G program. Apostolaki-Iosifidou et al. [80] conducted experimental measurements to determine power loss during EV charging and discharging. The measurement results indicate that most power losses occur in the power electronics used for AC-DC conversion [80]. Usually, the highest efficiency of the power electronics occurs at the top region of their rated power [80]. In addition, the efficiency of the power electronics is higher during charging than discharging [80]. This is due to the higher voltage during charging than discharging at a given power. The higher charging voltage results in a lower charging current, thus lowering internal resistance losses [93]. Therefore, the trade-off between inevitable battery degradation and power loss during battery discharging and the benefit of V2G needs to be farther investigated. More research on battery design and management is required to minimize battery degradation during charging/discharging.

3. Artificial Intelligence-Based Forecasting Model

The relationship between forecasting, scheduling, and dynamic pricing is shown in Figure 2. As can be seen, there are three critical components of EV charging/discharging that are interlinked. These components are EV charging/discharging-related forecasting, scheduling, and dynamic pricing. Accurate forecasting results provide insightful information on system conditions to scheduling models, which provide optimal charging/discharging control and pricing signals which are then used to update the forecasting model to enhance prediction accuracy. Thus, accurate forecasting results are crucial for making optimal EV charging/discharging strategies. Commonly used artificial intelligence-based forecasting models related to EV charging/discharging are discussed in this section.

3.1. Supervised Learning Methods

EV charging-related forecasting results are generally used as inputs to the optimization models to enhance EV charging scheduling performance. Many different types of machine learning methods can be used for both regression and classification problems. Commonly used machine learning methods for forecasting tasks are linear regression (LR), decision tree (DT) [94], random forest (RF) [95], support vector machine (SVM) [96], k-nearest neighbor (KNN) [97], artificial neural networks (ANNs) [98], convolutional neural network(CNN) [99], deep convolutional neural network (DCNN) [100], recurrent neural network (RNN) and its two gating mechanisms, gated recurrent units (GRUs) [101] and long short-term memory (LSTM) [102]. LR is suitable for linear models. Although LR is easy to implement, the performance of LR is largely affected by overfitting issues and outliers [53]. EV charging-related forecasting tasks are nonlinear. Therefore, LR has barely been used to handle EV charging-related problems. In contrast, DT is able to separate a complex decision into multiple straightforward decisions. However, a single DT cannot always guarantee a good prediction. Moreover, DT is often prone to overfitting issues [53]. RF is an ensemble of DTs; thus, it is more accurate than DT and can overcome overfitting by aggregating multiple DTs [95]. RF is proposed in [72] to forecast EV charging load. The results of the simulations indicate the effectiveness of RF on EV charging load prediction. Although SVM and KNN are suitable for solving regression problems, they are generally used for classification tasks. Erol-Kantarci and Mouftah [54] used KNN to predict electricity prices to reduce PHEVs’ charging costs and CO₂ emissions. The simulation results show that the prediction-based charging scheme reduces the PHEVs’ operating costs and CO₂ emissions. SVM is used in [103] to forecast EV charging demand with a 3.69% mean absolute percentage error (MAPE), which is lower than the 8.99% MAPE of the Monte Carlo technique.

3.2. Gated Recurrent Units (GRUs)

Deep neural networks, such as RNN, GRU, LSTM, and CNN, are very efficient and popular nowadays due to the availability of a large number of datasets and strong computational power. It has been proven that RNN with gating units (GRU and LSTM) performs better than without them [104]. The Gated recurrent units (GRUs) gating mechanism of recurrent neural networks was first introduced in [101]. GRUs is similar to LSTM, which can also be used to solve the vanishing and exploding gradient problems. In addition, the structure of GRU is simpler than LSTM. Moreover, GRU usually performs better than LSTM in solving problems with a low sample size [105]. Therefore, GRU usually requires less processing power and training time than LSTM [106]. As a result, GRU is also popular for solving problems with long sequence samples, such as spot electricity price prediction [57], load demand, and PV power production [107]. Lu et al. [57] proposed a GRU to forecast spot electricity prices used to plan travel routes and charging schemes. The results of simulations indicate that the route planning algorithm can effectively and accurately provide optimal charging routes for EVs users. The prediction of the future trend of load demand and PV power production is made by GRU in [107] to reduce the operating costs of EV charging stations. Boulakhbar et al. [73] compared the EV charging load forecasting performance of four popular deep learning models: ANN, GRU, LTSM, and RNN. The simulation results indicate that GRU has the best performance, followed by RNN, LTSM, and ANN. However, their results should only be used as guidance when selecting a model for forecasting tasks because the generalization of these models was not tested in [73].

3.3. Long Short-Term Memory (LSTM)

Long short-term memory (LSTM) is another popular gating mechanism of RNN. It was first introduced in [102]. GRU and LSTM can solve complex, long-sequence problems better than recurrent neural networks (RNNs). Additionally, LSTM usually performs better for solving high-complexity sequences than GRU [108]. Therefore, LSTM has been generally used to solve problems with long and high-complexity sequence samples. Kriekinge et al., proposed an LSTM model to predict load demand [64] and one day ahead of EV charging demand [67]. Schedulers then use the forecasted load demand to plan EVs’ charging and discharging. Wang et al. [55], Zhang et al. [22], and Wan et al. [20] used LSTM to predict future charging prices based on the historical charging price. LSTM networks and deep LSTM have been applied to predict V2G capacity by Shange et al. [59] and Li et al. [60], respectively. Nogay [61] compared the V2G capacity forecasting performance of LSTM with the Nonlinear Autoregressive Neural Network (NAR). The simulation results indicate that NAR performs slightly better than LSTM. However, both models can use historical knowledge to correct learned aggregated available capacity deviations for V2G service.

3.4. Hybrid and Ensemble

Different forecasting mechanisms have various advantages and disadvantages. The forecasting performances should be able to improve if the advantages of different mechanisms are combined to form hybrid models. Shipman et al. [62] proposed a hybrid of CNN and LSTM-based models to predict the next 24 h of aggregated available SOC of 48 vehicles for V2G services based on the previous 24 h of data. The performance of the model proposed in [62] is further enhanced by continually refining the prediction model based on the latest observed behavior [109]. Zhong and Xiong [66] proposed a load demand and renewable energy output prediction model that combines CNN with a deep belief network (DBN) to form the CNN-DBN prediction model. The forecasted results are then used to schedule EVs’ charging to minimize the operating cost of the distribution network. Sun et al. [68] proposed an artificial intelligence-based hybrid model that consists of K-means clustering, KNN classification, and LSTM prediction to predict EV driver charging behavior. Zhang et al. [65] proposed a CNN-based ensemble model to forecast traffic flow and EV load demand. The forecasted load demand can be used to make electricity trading decisions.

Artificial intelligence-based forecasting models are usually supervised learning-based models which use labeled data to train the models for predictions. The forecasting models are used to predict electricity price, EV load demand, driving pattern, and availability of state of charge, which can then provide EV charging and discharging schedules and pricing. However, the performance of these charging scheduling models is highly dependent on the accuracy of the prediction models. Uncertainty is an inherent property of forecasting models. The stochastic and unpredictable EV charging, discharging, and driving behaviors make forecasting harder. Therefore, forecasting results alone will not result in optimal charging scheduling strategies. Other than the suggested hybrid and ensemble techniques, online-based prediction models [110,111,112,113], that take the latest available information to update the prediction models might reduce the forecasting uncertainty.

Moreover, probabilistic learning algorithms, such as Gaussian processes (GP) [87,114], can provide prediction with an uncertainty interval that could also be applied to EV charging/discharging-related studies.

4. Artificial Intelligence-Based Scheduling

Conventional numerical optimization methods, such as convex optimization and linear programming, need to make some physically unknown assumptions [21]. The investigation results of [31] indicate that most researchers prefer artificial intelligence models to probabilistic models because artificial intelligence is suitable for complex nonlinear problems like forecasting EV charging demand and optimizing charging schedules. Moreover, artificial intelligence models such as neural networks and reinforcement learning approaches can obtain nonlinear relationships from historical data and learn via interacting with the environment, respectively. Therefore, they usually do not require expert knowledge of a complex system to build a model from system developers due to the ability of artificial intelligence to learn from the existing dataset. Furthermore, complex problems like demand forecasting and charging scheduling optimization are stochastic, requiring the models to be continuously updated. The online setup of artificial intelligence allows models to learn the latest available dataset to improve model performance. The following are some of the artificial intelligence-based scheduling algorithms.

4.1. Heuristic Algorithms

Heuristic algorithms such as Genetic Algorithm (GA) [115], Particle Swarm Optimization (PSO) [116], Differential Evaluation (DE) [117], Artificial Bee Colony (ABC) [118], to name only a few, are very popular for solving optimization problems due to their easy implementation and fast training speed. Among all the heuristic algorithms, GA and PSO are the most commonly used ones for EV charging and discharging scheduling problems. A GA was employed by [119] to optimize the battery charging and discharging schedule. The simulation results show that the proposed model can provide a demand response to adjust the main transformer load. Farahani [120] proposed a PSO-based EV charging/discharging scheduling to minimize the voltage unbalance factor of low-voltage distribution networks. The simulation results indicate that the proposed model is able to reduce the voltage unbalance factor. PSO-based EV charging scheduling model is also proposed in [121] to reduce power loss and peak valley difference. Although GA and PSO-based optimization models mentioned in [119,120,121] perform well, the performance comparison of these models is lacking in [119,120,121]. Dogan et al. [122] tested the performance of four different heuristic algorithms on EV charging/discharging scheduling. The simulation results show that GA can provide the lowest EV station operating cost and the most convenience for EV owners, followed by PSO, DE, and ABC. Although heuristic algorithms are easy to implement and can provide a fast solution, they do not always provide optimal solutions because they might be trapped in local optimal and provide unstable performances [123]. In addition, the parameters of these algorithms can largely impact their performance. Thus, the ranking of these algorithms should only be used as guidance.

4.2. Fuzzy Logic

A fuzzy logic-based controller [124] has been implemented in [69,70] to manage and schedule EVs’ charging/discharging intelligently. Authors in [69] used fuzzy logic-based controllers to maintain a balance between system operators and EV owners. The simulation results indicate that the fuzzy logic controller is able to improve the power system efficiency and increase profit for aggregators. The control action of [70] is based on EVs’ initial and target state of charge and available charging duration. The simulation results indicate that the controller can consider EV owners’ and power providers’ preferences and provide economic benefits for both EV owners and power providers.

4.3. Q-Learning and Deep Reinforcement Learning (DRL)

Many authors have applied reinforcement learning (RL) algorithms to schedule EVs’ charging and discharging. RL-based EV charging scheduling is usually present in the Markov decision process (MDP) [22,23,125]. A finite MDP can be solved by dynamic programming (DP), which assumes perfect knowledge of the environment and requires high computation expense [126]. Rotering and Ilic [5] implemented a DP-based model to optimally control PHEVs’ charging time and power rate based on the forecasted electricity prices in deregulated electricity markets. The simulation results indicate that the model can reduce daily electricity costs substantially. Iversen et al. [58] proposed an inhomogeneous Markov chain model to predict the probability of EVs’ driving patterns. Then a stochastic DP model is applied to the EVs’ driving pattern and electricity price to provide the optimal charging policy. The simulation results show that the optimal charging policy can reduce daily energy costs. However, the environment of EV charging scheduling tasks is not usually known. DP optimizes its policy based on a known environment, whereas Q-learning, a model-free RL, optimizes its policy via interacting with its environment. Watkins introduced Q-learning in 1989 [127], which is popular for scheduling problems with a limited number of state and action spaces.

Dang et al. [56] proposed artificial neural networks (ANN) forecasting model to predict the next day’s electricity price. The forecasted results are then used to train the Q-learning-based charging scheduling model. Lee and Choi [29] proposed a Q-learning-based energy management system that considers the ToU tariff, home appliances, and energy storage system with charging/discharging functionality. The simulation results show that the proposed model provides a 14% electricity bill reduction compared to MILP. The charging/discharging functionality of the energy storage systems is similar to that of EVs’ charging and discharging. Therefore, the results should be similar if the proposed model in [29] is applied for EV charging and discharging. Mhaisen et al. [128] proposed a Q-learning model to give EVs real-time charging and discharging schedules. The simulation results indicate that the proposed model can achieve a cost saving of approximately 68% compared to the uncontrolled one. However, one of the limitations of Q-learning algorithms is the curse of dimensionality. Thus, deep reinforcement learning [23,24,25,26,27,28,30,51,55,71,125,129], which combines deep neural networks with reinforcement learning, has been widely used to overcome the issue of the curse of dimensionality.

Deep-Q networks (DQN) combine deep neural networks with Q-learning to make RL more suitable for high-dimensional and complex environments [28,130]. DQN has been used in [20,21] to reduce charging costs and increase power grid reliability. Wang et al. [71] proposed a DQN-based battery-swapping scheduling strategy to provide fast frequency regulation services. The simulation results indicate that the proposed model can maximize the battery swapping station operators’ revenue by autonomously scheduling the hourly regulation capacity in real-time. Wang et al. [55] combined DQN and Dyna Q learning [126] to allow the model to learn from both real-time experiences and simulated experiences, which speeds up training. The proposed algorithm can reduce the long-term charging cost while avoiding battery depletion during trips. Double-DQN was introduced in [131] to reduce the overestimation of action values of DQN, and it is applied in [28] to control EVs’ charging/discharging based on the hourly electricity price. The simulation results indicate that the Double-DQN algorithm reduces the correlation of the action values with the target and provides better profit for EV owners than other state-of-the-art models such as RNN, CDNN, and LSTM. However, the training time of DQN for problems with complex and large state spaces is usually long due to its value iteration nature. Asynchronous Advantage Actor Critic (A3C), which requires less training time than DQN [132], was introduced in 2016 by Google’s DeepMind [133]. Although A3C has not been applied to EV charging and discharging-related studies in the reviewed literature, it is a good algorithm for a complex problem with many states like EV charging and discharging and should be used in the future.

In addition, both Q-learning and DQN are only suitable for discrete action spaces, significantly reducing the effectiveness of the EV charging control algorithm due to the limitation of the action space exploration. The deep deterministic policy gradient (DDPG) is suitable for tasks with continuous state and action spaces [134]. It has been proposed in [22,23,24,25,26] to maximize system operators’ financial benefit or reduce the charging cost of EV owners. The simulation results in [24] show that the higher pricing frequency can better reflect power system demand and supply situations and shift EV charging load. The simulation results in [23] suggest that a DDPG-based EV charging strategy can strictly guarantee voltage security by scheduling EV charging and discharging at suitable time periods. Sun and Qiu [129] proposed DDPG to solve the voltage control strategy problem to mitigate voltage fluctuation caused by stochastic EV and load demand. Qiu et al. [30] proposed a prioritized deep deterministic policy gradient (PDDPG), a combination of DDPG and prioritized experience replay, to optimize EV owners’ and aggregators’ profits. PDDPG can solve the problem in multi-dimensional continuous state and action spaces, which is preferable for practical application and requires lower computational resources than DDPG with uniform sampling. The simulation results of [30] show that PDDOG can provide 31%, 13%, and 5% higher profit than Q-learning, DQN, and DDPG, respectively. Another off-policy algorithm, soft-actor-critic (SAC) [135], is used by Yan et al. [125] to balance charging costs and drivers’ anxiety. The maximum entropy framework of SAC makes the proposed model more sample efficient and robust [125]. Lee and Choi [27] proposed a SAC model to provide dynamic EV charging and discharging pricing to maximize the profits of multiple EVCSs.

5. Dynamic Pricing and Peer-to-Peer for EV Charging/Discharging

The performance of these EV charging scheduling algorithms is highly dependent on the accuracy of the electricity pricing on reflection of the real-time power system condition. However, the predicted electricity discussed in Section 3, and the electricity used to schedule EV charging and discharging discussed in Section 4, are based on historical data or assumed figures. A large amount of electricity is already traded several months or years in advance via forwarding contracts and over-the-counter markets under the conventional electricity market [136], which cannot reflect the actual demand and supply of electricity in real-time [137]. Thus, dynamic pricing that can reflect real-time power system conditions is needed to enhance the performance of EV charging and discharging scheduling.

5.1. Time of Use (ToU)

Many utility companies in India, Sweden, the United Kingdom, Canada, and the United States, etc., have used the ToU electricity tariff scheme to encourage users to shift load demand from peak periods to off-peak periods [138]. ToU tariff provides different charging pricing for peak, standard, and off-peak hours [54]. In the study [78], the estimation results show that ToU is the most preferable pricing rule for consumers in Japan. The impact of the ToU electricity tariff scheme on a smart grid with plug-in hybrid electric vehicles (PHEVs) is investigated in [35]. The simulation results indicate that a properly designed ToU tariff scheme can reduce the peak demand for smart grids with PHEV penetration. Guo et al. [139] used the mixed-integer linear programming (MILP) model to determine the impact of ToU on different charging strategies. The simulation results indicate that smart charging can benefit from the ToU tariff compared with uncontrolled and constrained charging. Xu et al. [36] proposed a zonal and ToU EV charging pricing model based on demand price and power quality. The simulation results show that the proposed pricing model can smooth the load curve and alleviate the impact of load fluctuation. Wolbertus and Gerzon [140] investigated the impact of a time-based fee on the idle time of EVs at an EV charging station (EVCS). The idea is to charge an additional fee for the EVs that have completed the charging process and are still parked at the charging station. The simulation results show that a time-based fee can reduce EVs’ idle time at an EVCS and improve the efficiency of EVCSs.

Many EV charging strategies [17,18,19] are designed to adapt ToU tariffs to reduce charging costs. Cao et al. [17] proposed an intelligent method to control EV charging loads in response to the ToU tariff in a regulated market. The results of simulations show that the optimized charging pattern has excellent benefits in reducing cost and flatting the load curve. Lee et al. [18] proposed an EV charging pricing scheme to optimize the trade-off between inexpensive ToU pricing and peak power draw. Wang et al. [19] proposed a kernel-based estimator to estimate charging session parameters, such as stay duration and energy consumption, to schedule EV charging according to ToU tariff and renewable energy generation. The simulation results show that the average charging cost can be reduced by 29.42% with the proposed scheduling framework. Most of the existing ToU pricing strategies are determined based on the load information collected over many years [54], except Gautam et al. [63], who used a multilayer perceptron (MLP) model to predict load demand which is then used to determine ToU charging price in real-time. It is known that the traditional ToU electricity tariff is not dynamically adapted to changes in operating conditions [4]. Furthermore, authors in [35,141] suggest that the ToU tariff can economically entice PEVs to charge during off-peak periods to reduce the load peak. However, a new, more significant peak might be formed during the original off-peak hours due to the simultaneous charging of EVs during the off-peak period [44].

It can be argued that many researchers have focused on designing a suitable EV charging scheduling model that considers the ToU tariff. The scheduling models can shift EV load demand to the off-peak period to minimize charging costs. However, ToU tariffs are semi-dynamic because tariff, peak, and off-peak hours are determined based on historical data, and the values are set to last for a while. Moreover, EVs’ charging and discharging are more stochastic and challenging to predict than the load demand. Therefore, a more dynamic pricing model that can reflect the conditions of power systems is needed to schedule EVs’ charging more effectively, especially when the EV penetration is high.

5.2. Real-Time Pricing (RTP)

RTP with higher pricing updating frequency might overcome some of the challenges encountered by the ToU tariff [44]. The price updating frequency is one of the main distinguishing factors between RTP and ToU. RTP usually has high updating frequency than ToU. It has been shown that hourly-based RTP would have saved EV owners significantly over one without it [142]. However, the limitations of hourly-based RTP are that it cannot reflect the instantaneous power system conditions. Instantaneous RTP that has been proposed in [44] is more effective than hourly-based RTP in that it can more closely reflect the actual conditions of the power system and electricity market. Therefore, the price updating frequency of RTP is a crucial factor when designing the RTP mechanism.

5.2.1. Application of RTP

RTP can benefit both EV owners and power system operators if it can accurately reflect real-time power system conditions. On the one hand, EV owners can choose to charge their EVs when the electricity price is low and discharge their EVs when it is high. On the other hand, power system operators can use real-time power grid conditions, such as load demand, electricity price, and renewable energy generation, to generate price signals to change charging and discharging behaviors to benefit power grids. Although RTP has been effective for many applications, only a few research applied RTP for EV charging and discharging scheduling [27,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51].These models have been applied to maximize the welfare of the whole charging system in [37], satisfy customers’ quality of service and power-grid stability in [45], reduce charging waiting time in [46], maintain a stable demand-supply curve in [39,43,44], minimize EV owners’ charging costs in [41,47,50], and maximize EVCS operators’ revenue in [27,38,40,42,48,49,51]. However, in most studies [27,38,39,40,42,45,46,48,49,51], system operators determine the electricity price and control the EV charging to optimize the power system and maximize system operators’ profit, thus, giving the EV owners less freedom to trade stored electricity freely at their preferred price. The EV owners’ preferences and willingness to charge are considered in [41,51]. Kim et al. [41] have proposed an improved reinforcement learning-based real-time pricing algorithm that considers the customers’ load demand levels and the electricity cost. The simulation results show that reinforcement-based dynamic pricing achieves higher and long-term performance than the approach that only focuses on immediate system performance [41]. Maestre et al. [51] have used DQN techniques to provide dynamic pricing models to maximize revenue while considering fairness and equality.

5.2.2. RTP Classification

The majority of the existing EV charging scheduling RTP is based on either artificial intelligence [27,38,40,41,43,48,51] or optimization algorithms [37,39,42,44,45,46,47,49]. Reinforcement learning and GA are the most used to optimize EV charging pricing models. It has been shown in [40] that artificial intelligence such as ANN, GA, and Population-Based Incremental Learning (PBIL) can model and optimize dynamic pricing models better than well-established demand models such as linear, exponential, and multinomial logit models. Mao et al. [38] proposed a GA-based V2G pricing model that considers the system load condition, maximum power limit, and fair price rate to improve system operators’ profits and encourage more V2G program participants. RL-based optimization models are used in [41,43,48,51]. For example, Lou et al. [48] proposed DP based charging pricing optimization model, which uses the day-ahead wholesale electricity price, EV charging demand, and renewable energy generation to calculate dynamic charging price. The simulation results show that the DP model can increase profit by considering the entire profile. Moghaddam et al. [43] proposed an online RL for RTP for EV charging stations to reduce peak demand. The simulation results indicate that the online RL-based RTP can effectively reduce peak demand and increase the profits of charging stations. In addition, an improved reinforcement learning and DQN-based real-time pricing algorithms are proposed in [41,51], respectively, as discussed previously. Although reinforcement learning-based RTP models discussed in this section can achieve goals such as maximizing system operators’ profits and maintaining a stable demand-supply curve, many types of RL, as mentioned in Section 4.3, can also be used to solve RTP optimization problems.

In addition, the dynamic pricing models in most studies [37,45,46,47,48,51] only allow EV charging from power grids, which prevents EVs from feeding stored electricity back to the grid. Therefore, these pricing models are not suitable for EV discharging. Moreover, all the participants, such as EV owners and system operators, should co-decide on the charging and discharging price. Co-decided charging price can effectively improve the EVCS operator’s income and efficiency of the EV charging system while reducing the EV users’ charging costs [50]. A more transparent and open real-time pricing policy might attract more EV owners to participate in the V2G program.

5.3. Peer-to-Pear (P2P)

P2P electricity trading allows sellers and buyers to co-decide on the charging and discharging prices. Liu et al. [137] proposed a P2P EV power trading model based on blockchain and a reverse auction mechanism to optimize social welfare and market efficiency. Blockchain technology ensures safer, more efficient, and transparent transactions [137]. The reverse auction mechanism allows EV owners to adjust the bidding price to improve the matching success rate [137]. Sather et al. [143] also proposed a linear programming-based P2P electricity trading model for an industrial site. The results show that P2P electricity trading can reduce electricity costs, distributed energy resource curtailment rate, and grid power feed-in. Liu et al. [144] proposed a dynamic internal pricing model based on PV energy supply and demand ratio (SDR) to operate the energy sharing zone effectively. The results show that P2P trading with an internal pricing scheme allows the PV prosumers to earn more than directly trading with the utility grid under the feed-in tariff. The internal prices are co-decided by all the prosumers during the energy sharing. Co-decided internal prices have both advantages and disadvantages. Conversely, co-decided internal prices allow all the prosumers to decide the internal price. On the other hand, the flexibility of prosumers’ load profile makes it hard to converge on the internal price.

P2P trading enables individual EV owners to trade electricity at their preferred prices. However, human beings such as EV owners and EVCS operators are unpredictable and easily influenced by external factors such as prejudices, mood swings, and lack of capacity to process large data [145]. Therefore, artificial intelligence machines are expected to be used for decision-making in the fourth industrial revolution [145]. Using an intelligent machine to make decisions can minimize human behavioral characteristics such as heuristics, risk aversion, and diminishing sensitivity to losses and gains. Artificial intelligence-based pricing policy can provide an accurate and fair electricity market for EV owners because these models can process a large amount of real-time data and make decisions based on the data.

6. Discussion

The development of V2G can be classified into three phases [146]. The schema of V2G development phases and corresponding EV charging/discharging techniques is shown in Figure 1. First, the EV charging load only accounts for a small load demand proportion of power grids in the preliminary phase. The development of V2G is currently in the first phase. Therefore, the main charging control strategies presently used are uncontrolled charging and controlled charging. Many V2G pilot projects are still in the planning and testing phases. In addition, only a tiny portion of the existing commercial EVs is built with V2G capability. The existing ToU tariff that is applied to other load demands can also influence EV charging behaviors. However, the traditional ToU tariff will not be able to handle the high penetration of EVs if stochastic EV charging loads are not considered in the design of the ToU tariff.

In the second phase, the EV charging load will account for a higher proportion of power grids’ load due to the high penetration of EVs. A large number of uncoordinated and stochastic EV charging requests might burden power grids during peak demand periods, which could endanger power grid stability. Therefore, aggregators must play an important role in coordinating EVs to provide DSM services, such as enhancing renewable energy utilization rate, peak shaving, valley filling, and alleviating congestion of distribution networks [146]. The smart charging/discharging control strategies should be considered in this phase. The main focus of the existing EV-related literature is on EV smart charging/discharging. Table 4 summarizes the relevant literature on EV charging/discharging forecasting, scheduling, and pricing strategies.

Most prediction models are supervised learning-based models that many researchers apply to predict future electricity prices, EV load demand, renewable energy generation, and the availability of battery SOC for V2G services. Optimization models then optimize the forecasted information to provide EV charging/discharging decisions. A small forecasting error might have a significant impact on decision-making. Therefore, the forecasting models’ accuracy is crucial for optimal EV charging scheduling models. Although there are many different types of artificial intelligence methods are very popular due to the availability of datasets and strong computational power, among all the deep learning methods, long short-term memory (LSTM) and gate recurrent units (GRUs) are the most popular methods for forecasting EV charging/discharging related tasks due to their ability to handle nonlinear and long-term dependency. Although supervised learning models can provide adequate forecasting accuracy, uncertainty is one of the inherent properties of forecasting models. EVs’ charging and discharging patterns are stochastic and unpredictable, making forecasting even harder. Hybrid and ensemble techniques can combine the advantages of different forecasting methods or model configurations to enhance forecasting performance. Other than hybrid and ensemble techniques, online-based forecasting [110,111,112,113], and prediction confidence interval techniques [114,147] are two potential solutions to minimize the impact of the inherent forecasting uncertainty. Online-based forecasting models can use the latest available data to update the model, and prediction confidence interval techniques allow decision-makers to plan by considering uncertainty ranges. Moreover, individual forecasted information alone is insufficient for optimal EV charging/discharging decisions, such as when to charge EVs and discharge power from EVs back to power grids. For example, it is difficult to make optimal charging/discharging decisions based on the forecasted EV charging demand alone. The charging/discharging decision should be made based on all the important power system conditions, such as renewable generation, system constraints, supply and demand curves, and charging/discharging dynamic pricing signals. Therefore, optimization models are required to process the essential forecasted information to make optimal decisions.

In recent years, reinforcement learning-based models have been commonly used to schedule EVs’ charging/discharging. However, the conventional Q-learning requires a lookup table which is infeasible for a problem like EV charging scheduling with many states and actions [29,128]. Deep-Q networks (DQN), which combine deep neural networks with Q-learning, have been implemented by many researchers to overcome the curse of dimensionality faced by conventional Q-learning-based approaches. Therefore, DQN-based models are more suitable for high-dimensional and complex environments [20,21,71]. Although DQN can handle high-dimensional and complex environments, the discrete action spaces limit its action space exploration. In addition, overestimation of action values of DQN for some applications has been encountered [131]. Double DQN can be used to address the overestimation issue of DQN [28,131]. In addition, DDPG and SAC have been used to provide continuous state and action spaces to further improve the performance of DQN [22,23,24,25,26,27,125,129]. Although DQN, DDPG, and SAC can handle continuous complex problems, they usually require a long time to train. It has been shown in [132,133] that A3C requires less training time than DQN. Based on the literature that was reviewed, it is found that A3C has not yet been used for EV charging/discharging-related tasks. However, A3C, with multiple agents training in parallel, has many advantages, such as fast convergence towards optimal policies and learning process stabilization. Therefore, it should be considered in future studies.

Additionally, episodic memory and meta-reinforcement learning techniques have the potential to improve training speed and meta-reinforcement learning. More information about episodic memory and meta-reinforcement learning can be found in [148,149]. Furthermore, reward functions and many hyperparameters of reinforcement learning models need to be tuned. Adjustable reward function [150] and Bayesian Optimization [151,152] can be used for reward function and tuning hyperparameters, respectively. Moreover, RL methods are evolving and improving regularly. Therefore, the latest advanced RL methods should be applied to EV charging/discharging scheduling and dynamic pricing to see if performance can be further enhanced. Different parameters of RL-based models, such as reward functions, learning rate, exploration and exploitation trade-off, and model structures, also need to be tested and compared to find optimal combinations to suit the tasks.

Although simulation results of most EV charging/discharging scheduling show good performance, most of them were tested on historical or artificial data. The historical or artificial electricity price cannot reflect real-time operation conditions such as EV charging load demand, V2G discharging requests, and renewable energy generation. Therefore, a suitable dynamic pricing model that accurately reflects power system operating conditions is essential.

Finally, in the third phase, adequate technology and management will allow many EVs to provide ancillary services to power grids via bidirectional power flow to achieve an ideal state of smart grids [146]. By then, substantial communication resources will be required for EV owners and power system operators to interact with each other. The event-triggered communication mechanism can save bandwidth and signal-processing resources compared with the periodic communication mechanism [153]. In addition, battery degradation and charging/discharging losses should be minimized to accelerate the adoption of V2G technology. Therefore, artificial intelligence-based battery design and management are also important research fields. Although V2G has many benefits, it is challenging to encourage EV owners with different driving and charging preferences to participate in the same V2G program. Moreover, designing different V2G programs to satisfy all the participants is not feasible. Thus, dynamic pricing can be used as a special form of demand response to influence EV owners charging and discharging behaviors.

Dynamic pricing has been applied in many fields, such as airline ticket pricing, e-commerce product pricing, and advertising bid pricing. The strategy is to increase the selling price when demand is higher than supply and decrease the selling price when supply is high than demand. More factors, such as power system constraints, renewable energy generation, and EV charging/discharging preferences, need to be considered when designing a dynamic pricing model for power systems with high EV penetration. Therefore, a properly designed dynamic pricing strategy that can accurately reflect the conditions of the power grid is needed to correctly guide EV charging and discharging to optimize the energy use of power grids. However, there is a limited literature focusing on the design of EV charging/discharging dynamic pricing schemes. Most researchers focus on designing scheduling models that consider dynamic pricing to minimize charging costs [27,38,40,42,48,49,51]. In addition, most of the existing dynamic pricing schemes provided by power utility companies might not consider the effect of high EV penetration and EV owners’ preferences [27,38,39,40,42,45,46,48,49,51]. Although the P2P electricity transaction mechanism allows sellers and buyers to co-decide a tariff, the auction-based mechanism does not always converge. Moreover, it is difficult for EV owners to make pricing decisions without having the whole picture of power system conditions and the electricity market. For example, EV owners might undervalue their stored battery power during power system constraints and overvalue their power when renewable energy production is high. Therefore, more research should be done on dynamic pricing schedule design that can solve the above-mentioned issues.

Table 4. Summary of the relevant literature on EV charging and discharging forecasting, scheduling, and pricing strategies.

Algorithm	Applications	Advantages	Disadvantages	Possible enhancement
Supervised Learning [20,22,54,55,56,57,59,60,61,62,63,64,65,66,67,68,72,73,103,107,109]	Load demand prediction. Battery capacity availability prediction. Renewable energy generation prediction. Electricity price prediction.	To learn complex relationships from historical data. Able to handle nonlinear and long-term dependency.	Uncertainty is one of the inherent properties of forecasting models. Small forecasting errors might have a significant impact on decision-making.	Hybrid and ensemble techniques. Online-based learning uses the latest available data to update the model frequently [110,111,112,113]. Use a confidence interval [147], i.e., a 95% confidence interval.
Reinforcement Learning [5,20,21,22,23,24,25,26,27,28,29,30,51,55,56,58,71,125,128,129]	Schedule charging and discharging. Reduce charging cost. Maximize the profits.	Able to learn from experiences. Maximize performance. Able to solve complex and continuous problems with function approximators.	Many hyperparameters to tune. Require expert knowledge when designing reward functions. Require large computation power to train models for complex problems.	Adjustable reward function during training [150]. Use Bayesian Optimization to optimize hyperparameters [151,152]. Use episodic memory [148,149], and meta-reinforcement learning to make training faster and more efficient.
Dynamic Pricing [27,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51]	Balance supply and demand. Reduce charging waiting time. Increase the profit of system operators. Reduce charging costs.	Able to reflect power system conditions in real-time. Able to encourage EV owners to shift charging time or to discharge power back to the grid when needed.	ToU tariffs are usually determined based on historical data, which cannot reflect power systems’ actual conditions. ToU might cause simultaneous charging at low tariff periods, which results in a new peak. RTP is mainly designed by system operators to achieve their goals without considering EV owners’ preferences	Update tariffs more frequently to reflect the actual demand and supply situation. Use forecasting and scheduling model to provide real-time pricing that reflects power system conditions better.

Table 4. Summary of the relevant literature on EV charging and discharging forecasting, scheduling, and pricing strategies.

Algorithm	Applications	Advantages	Disadvantages	Possible enhancement
Supervised Learning [20,22,54,55,56,57,59,60,61,62,63,64,65,66,67,68,72,73,103,107,109]	Load demand prediction. Battery capacity availability prediction. Renewable energy generation prediction. Electricity price prediction.	To learn complex relationships from historical data. Able to handle nonlinear and long-term dependency.	Uncertainty is one of the inherent properties of forecasting models. Small forecasting errors might have a significant impact on decision-making.	Hybrid and ensemble techniques. Online-based learning uses the latest available data to update the model frequently [110,111,112,113]. Use a confidence interval [147], i.e., a 95% confidence interval.
Reinforcement Learning [5,20,21,22,23,24,25,26,27,28,29,30,51,55,56,58,71,125,128,129]	Schedule charging and discharging. Reduce charging cost. Maximize the profits.	Able to learn from experiences. Maximize performance. Able to solve complex and continuous problems with function approximators.	Many hyperparameters to tune. Require expert knowledge when designing reward functions. Require large computation power to train models for complex problems.	Adjustable reward function during training [150]. Use Bayesian Optimization to optimize hyperparameters [151,152]. Use episodic memory [148,149], and meta-reinforcement learning to make training faster and more efficient.
Dynamic Pricing [27,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51]	Balance supply and demand. Reduce charging waiting time. Increase the profit of system operators. Reduce charging costs.	Able to reflect power system conditions in real-time. Able to encourage EV owners to shift charging time or to discharge power back to the grid when needed.	ToU tariffs are usually determined based on historical data, which cannot reflect power systems’ actual conditions. ToU might cause simultaneous charging at low tariff periods, which results in a new peak. RTP is mainly designed by system operators to achieve their goals without considering EV owners’ preferences	Update tariffs more frequently to reflect the actual demand and supply situation. Use forecasting and scheduling model to provide real-time pricing that reflects power system conditions better.

7. Conclusions

This paper reviews three crucial aspects of EV charging and discharging: forecasting, scheduling, and dynamic pricing. The interconnected relationship between forecasting, scheduling, and dynamic pricing is identified. Scheduling models’ performance mainly depends on the accuracy of forecasting results and pricing strategies. On the other hand, forecasting accuracy and scheduling performance largely influence the effectiveness of dynamic pricing strategies in reflecting real-time power system conditions.

Most forecasting models mentioned in this paper are supervised, learning-based models. Among them, LSTM and GRU are the most popular methods due to their ability to handle nonlinear and long-term dependency. However, uncertainty is one of the inherent properties of forecasting models. Therefore, the performance of forecasting models needs to continue improving. Besides hybrid and ensemble techniques, using the latest available data to update forecasting models and adding uncertainty intervals are other options to assist decision-making.

Reinforcement learning-based optimization models that can take many variables as state spaces have been applied by many researchers to make optimal EV charging and discharging decisions based on the forecasted results, including charging and discharging prices. DQN, DDPG, and SAC are some of the most popular reinforcement learning models. Each of them has its advantages and disadvantages. DQN can overcome the curse of dimensionality faced by conventional Q-learning. However, overestimation of action values and long training time requirements are common issues faced by these methods. Double-DQN and A3C can solve action value overestimation and reduce training time, respectively. Improving reinforcement learning performance is also a key field of research. Scheduling models cannot make effective charging/discharging decisions to optimize power grids if the information they use to make decisions cannot accurately reflect the power grid’s real-time conditions. Therefore, both forecasting results and dynamic pricing signals that can reflect the real-time conditions of the power grid are important.

Many studies have explored the forecasting and scheduling aspects of EV charging. However, only a limited literature focused on EV discharging and dynamic pricing design. Most of the existing dynamic pricing is designed by system operators without considering EV owners’ preferences. In addition, not all the key factors related to EV charging/discharging are used to design dynamic pricing models. Dynamic pricing strategies are very important for indirectly controlled charging/discharging. Moreover, dynamic pricing can incentivize more EV owners to participate in V2G programs. Therefore, researchers should pay more attention to designing dynamic pricing schemes that can reflect real-time power system conditions and provide a balance between system operators and EV owners.

In addition to the technical aspect of EV charging/discharge scheduling and dynamic pricing discussed in this paper, research on the social and economic aspects of EV charging/discharging and dynamic pricing is required. On the social aspect, the EV owners’ and system operators’ response to dynamic pricing needs to be surveyed and analyzed. The opinions of all the involved parties can be used to enhance the dynamic pricing policy design. On economic aspects, the feasibility and profitability of the dynamic pricing model for the overall system, including individual EV owners, system operators, and power systems, need to be investigated.