1. Introduction
Zawadzki et al. [
1] recently conducted a survey with 503 respondents in Poland highlighting the importance of users’ perceptions and attitudes towards electric vehicle (EV) charging infrastructure and support. The respondents, however, also expressed concerns about the environmental impact of EV use. To reduce greenhouse gas emissions, many countries have implemented incentives to promote the use of renewable-energy-powered transportation. In this context, the European Commission announced in October 2022 that by 2035, all new vehicles registered in Europe must be zero-emission [
2], a regulation that servers as a major incentive for EV production. According to a review by P. Bryla, S. Chatterjee, and B. Cibiada-Bryla [
3], the number of research studies on EV adoption is highest in Asia, followed by Europe, the United States, and Canada. This suggests that countries with advanced technologies and higher investment are seeking effective strategies to increase EV adoption. China aims to become carbon-neutral by 2060 and has implemented measures to promote the use of EVs and other renewable energy sources, according to W. Wei et al. [
4]. Such incentives aim to reduce carbon emissions to acceptable levels within a short time frame and allow the environment to recover.
The increasing use of EVs in many countries has increased the need for efficient management of EV energy consumption as charging stations interact with the same grid that supplies homes, businesses, and industry. EV charging also represents a significant, regular, and highly intense amount of energy demand, which can substantially affect the grid, particularly if the system is unprepared. Integrating EV charging with renewable energy generation to support the reduction of greenhouse gas emissions is necessary. W. Lewicki, H. Coban, and J. Wróbel [
5] analyzed the impact of EVs on power supply systems and found that EV charging planning should be integrated with renewable energy generation to avoid potential negative impacts.
As electric energy must be used at the same time it is generated, managing energy in EVs is complex. This characteristic requires a system capable of forecasting to ensure proper electricity generation. Failure to do so may result in system instability. as real systems cannot be used for experimentation, simulations are required, as demonstrated by B. Canizes et al. [
6]. Their work examined the effectiveness of various strategies for simulating energy management of EVs. The use of big data, particularly historical data, is essential for such simulations, but this data is often sensitive difficult to obtain. Consequently, this study also explores methods for acquiring historical data for use in Machine Learning (ML) models and for creating new datasets based on existing data.
For this survey, a simulation structure is presented that models a physical system in which each component contributes to the energy management of EVs. The key components of the electrical system for EV charging are shown in
Figure 1, where they are interconnected and allow for either bidirectional or unidirectional energy flow. This study focuses on the distribution network where EVs are charged and considers typical daily travel activities, such as commuting, running errands, and short-distance trips close to home.
The transmission system in
Figure 1 represents the remainder of the electrical system. It connects to and interacts with the distribution network by supplying energy or absorbing any surplus energy generated, thus enabling bidirectional energy flow. Distributed generation connected to the distribution network can be either renewable or non-renewable. However, the energy flow is unidirectional, meaning the energy generated is supplied to the distribution network. The battery energy storage system functions as a regulatory component in the system, capable of absorbing excess energy generation and supplying it when required. In this case, the energy flow is bidirectional. Other entities, such as industrial facilities, stores, or homes, may also integrate renewable generation and enable bidirectional flow of electricity. The final category is the EV, which connects to the system via charging stations. Unlike other components, it does not maintain a fixed connection to a specific point in the distribution network. An EV can operate in three modes: charging its battery from the grid to the vehicle (G2V), supplying energy from the vehicle to the grid (V2G), or transferring energy from one vehicle to another (V2V).
The electrical power distribution network consists of numerous components distributed throughout its service area, many of which perform similar functions in different locations [
7]. A clear example is the network of public EV charging stations, which are geographically dispersed but serve the same purpose. Several researchers have proposed an aggregator agent to consolidate such services in a single platform, thereby simplifying the scheduling or charging of the EV [
8]. The aggregator agent establishes a direct connection between the customers and energy suppliers by providing information on available charging points, managing reservations, or managing charging sites. The key quality of the aggregator agent is its objectivity. It does not interfere in negotiations or share information among competitors. The aggregator can represent and coordinate multiple user groups, facilitating communication between buyers and sellers.
The adoption of optimization methods is imperative for the effective management of energy across the entire system. These procedures require prior knowledge of power generation and consumption to schedule optimal times and locations for EV charging and discharging [
9] and to monitor the battery’s state of charge (SoC). This optimization is made possible through the application of Artificial Intelligence (AI) or optimization algorithms used in day-ahead planning to forecast energy demand based on historical data. This survey focuses on the use of Multi-Agent Systems (MASs), including Multi-Agent Reinforcement Learning (MARL) and AI. Each agent represent a component of the distribution network structure used in EV energy management [
8]. The distributed computational structure of the MAS enables the processing of a significant amount of data.
Consumer habits play an essential role. According to C. Silva et al. [
10], variations in electricity pricing under the Demand Response (DR) program represent one mechanism for promoting energy savings. Accordingly, the Demand Response program is analyzed alongside the other components in this study.
To identify relevant data sources for developing an energy management system model, this review analyzes the data referenced in the selected studies. Due to its sensitivity, the data must be handled carefully to comply with data protection regulations. The analysis sought to identify potential open data sources for this study. This information is crucial, particularly for ML algorithms, which require data that is difficult to collect due to its unique characteristics.
This study is motivated by the challenges experienced by EV owners in managing the charging of their vehicles, which involves numerous variables. Unlike the widespread availability of gas stations, there is a limited number of charging stations, which creates a significant challenge to EV owners. In addition, electricity prices fluctuate more significantly throughout the day than those of conventional fuels, and charging an electric vehicle is often more time-consuming. Therefore, autonomous learning and real-time adaptation are imperative [
11].
This study considers multiple factors that influence the energy management of electric vehicles, addressing the balance between time, energy consumption, and cost. These include the integrated energy supply system, EVs themselves, and charging stations located in residential areas, places of work, and public places. MASs are used to represent these components within the energy management framework. This review highlights current models and approaches for MAS implementation as a significant trend, particularly for simulating and managing real-world scenarios.
The main contribution of this study is the identification of a management model that supports individual users, particularly those with light passenger vehicles, by considering their weekly work and leisure schedules in close proximity to their homes. Other EVs may interact with the system when using the modeled charging stations. However, this study does not address long-distance EV travel, as this would require charging stations beyond the scope of the MAS model. The ultimate objective of this study is to provide a response to the question of whether it is feasible to assist individual users in the management of EV charging through the utilization of smart grid information and AI-assisted technology.
The main contributions of this study to EV electrical energy management are as follows:
A focus on individual EV users during their routine work week;
Identification of the key participants in EV energy management;
Characterization of the connection between participants;
A survey of AI techniques and tools that can automate management actions;
An analysis of the current state of research in the field;
A new emphasis on individual-user energy management.
This study also examines the use of aggregator agents in the power system, which centralize information generated in a dispersed manner and provide customers with a single point of access to relevant data. Through this approach, the customer can obtain information about services offered at multiple points in the distribution network. By integrating EV charging stations under one agent, it optimizes the need to establish multiple connections for decision-making.
The remainder of this paper is structured as follows.
Section 2 presents the methodology used for the state-of-the-art survey and explains the literature review process.
Section 3 describes the main components of EV energy management.
Section 4 summarizes the techniques applied in this field.
Section 5 outlines the key challenges to energy management, and
Section 6 concludes this paper.
2. Methodology
To identify methodologies employed in EV energy management, a comprehensive literature review was conducted, considering users’ daily tasks and the corresponding electrical energy requirements. The survey addressed important issues, such as how to effectively represent the intelligent electrical distribution network; the fundamental nature of EVs and the qualities they possess; how they interact with the power grid; how EVs are integrated into the electrical system; and the planning strategies that optimize charging. This study also considers the factors that influence the behavior of EV users, how realistic elements are portrayed within multi-agent systems, and methods and algorithms that support the decision-making process. To this end, we surveyed six major research databases to identify the most pertinent publications from the last seven years.
The Web-of-Science, Association for Computing Machinery (ACM), Institute of Electrical and Electronics Engineers (IEEE) Xplore, Springer, SciELO, and ScienceDirect databases were used as research sources. Articles were selected according to the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) [
12] methodology, which provides a structured framework that clearly defines the criteria for paper selection based on the study’s objectives, ensuring a thorough research process.
The next step involved specifying the criteria for including and excluding papers in the systematic review, as listed below:
Inclusion:
- (a)
Studies that describe the components and characteristics involved in managing EV electrical energy, emphasizing the battery and energy transfer between the EV and the power grid, as well as the roles of the power grid, charging stations, vehicle, and EV owner/user;
- (b)
Research on the application of distributed management based on multi-agent systems to control EV charging;
- (c)
Papers containing keywords used to identify publications on EV energy management, as shown in
Table 1.
Exclusion:
- (a)
Duplicate papers;
- (b)
Papers without full access;
- (c)
Papers not written in English;
- (d)
Research involving vehicles unintended for residential use;
- (e)
Studies on EV rental and sharing systems.
This review focuses on personal EVs, such as five-seater cars, and excludes electric public transport vehicles, trucks, bicycles, scooters, and shared or leased EVs.
Table 1 lists the keywords used and briefly describes how each combination relates to the survey topics.
As each database uses a different query syntax, it was necessary to adjust the queries to ensure consistency across the sources and apply a filter. The objective was to obtain at least one relevant citation for each keyword in
Table 1.
Given the growing relevance of AI in energy management and the vast amount of data available within power systems, there are numerous opportunities to apply AI technique data for efficient data use. Several studies have addressed AI-based approaches to optimize EV charging and energy management. An initial survey from 2018 to 2024 identified 481 articles that addressed challenges related to EV management, integrating battery charge control with optimization techniques and ML algorithms. The survey also observed the role of MASs in EV energy management, emphasizing their ability to individualize agents, segregate information domains, and promote distributed processing. Additionally, studies using ML methods for day-ahead forecasting were identified as a significant area of interest.
Figure 2 presents the results of the sequence of selection stages carried out based on the results of the search of published articles in the sources. After the third selection, 146 publications were selected based on a title and abstract analysis, and from these, 81 articles were considered relevant for inclusion in the systematic review.
Several characteristics of the 81 selected papers merit attention. Vehicle-to-grid (V2G) is mentioned in 20 of the studies, which reflects the current absence of established regulations in distribution networks. This highlights the need for preparation measures and further research to develop a regulation framework that ensures grid reliability and integration readiness. Another key aspect is the use of techniques applied for decision-making. The studies were divided into three categories—Machine Learning, optimization, and both—as shown in
Table 2. Finally, twenty additional papers were review articles.
Although many selected articles adopt approaches that differ from the main objective of this research, publications that evaluate EV charging represent an essential source of knowledge for advancing the study of methods and strategies for residential EV energy management. The remaining sections of this document will provide a more detailed discussion of the topics covered in the selected studies, their potential applications, and any existing gaps that need to be bridged to enable EV owners or users to effectively manage their charging for everyday use.
3. Components Related to Electric Vehicle Energy Management
The literature identifies key components that constitute the system required to control EV charging and manage energy through the use of a MAS. The EV, the power system, and other entities are referred to as agents, and various contexts are explored, including cooperative EV charging scheduling [
17,
18], coordinated EV management systems in residential networks [
19], competitive EV charging at public stations [
8,
11,
20,
21], and impact analyses of electric vehicle penetration [
22]. These agents interact according to the management strategy. Six main elements involved in EV energy management were identified: electric vehicles; the electric power system; public charging stations; charging stations in public parking lots; workplace charging stations; and residential charging stations. These can be represented as agents in a MAS framework and are detailed below.
3.1. Electric Vehicles
The EV is considered a component of the electrical system and is characterized by high-power charging capabilities and the potential for bidirectional energy flow. EVs can connect to the grid at multiple points. Depending on the representation adopted within the MAS model, the EV may be viewed as either an individual entity or as part of a group. When physical and behavioral traits are unique, they are considered individual agents that interact with the system, aiming to optimize financial benefits while reducing energy consumption. On the other hand, when the EVs share universal characteristics, they are grouped into collective agents that cooperate with each other. Rewards are assigned to the group rather than to individual vehicles.
According to the review by L. Calearo, M. Marinelli, and C. Ziras [
23], the key data required for studying EVs are battery capacity, charging power, plug-in SoC, plug-in/out time, and charged energy. These variables have a direct impact on charging scheduling, autonomy, and overall energy management.
Table 3 summarizes 55 studies that used basic charging information and 26 that used additional variables—such as battery life (in pricing), distance to destination, and EV location—to estimate consumption. For example, Z. Zhai et al. [
11] used CRUISE (
https://www.avl.com) (accessed on 15 May 2023) software to simulate an EV with detailed construction characteristics. Notably, the UCLA Smart Grid Energy Research Center (UCLA SMERC) (
https://smartgrid.ucla.edu/) (accessed on 23 March 2023) laboratory was cited by Yingqi Xiong et al. [
24] for contributions to EV research.
3.2. Electric Power System
The essential components of the electrical power system are generation, transmission, and distribution. For this study, the power system is represented by the distribution network, which connects homes, charging stations, and other loads. The reviewed studies show that distribution network limitations are critical to charging management.
Most studies did not apply electrical power network modeling through IEEE-bus models. Instead, they used techniques such as load curve smoothing or determining optimal EV charging times to reduce costs or improve energy use, often based on historical consumption and pricing data. However, some studies use bus models of different sizes, such as 5-bus [
69], 33-bus [
65,
92], 33-bus and 69-bus [
62], 69-bus and 94-bus [
13], 33-bus and 118-bus [
17], 22-bus [
19], 3-bus [
20], 2-bus [
8], 6-bus and 24-bus [
28], 15-bus [
44], 34-bus [
73], 30-bus [
76], and 37-bus [
84,
85]. These models determine the magnitudes involved, which allows the results to reflect real operating conditions more closely. Some studies still use nodes to represent the network points and to establish operational limits for each node [
11,
75].
The electric power system is represented as an agent within a MAS to prevent load restrictions, power flow, and voltage limits, as stated by C. B. Saner, A. Trivedi, and D. Srinivasan [
17], to maintain the system’s security and liaise with local load groups and EV agent aggregators. Similarly, M. S. H. Nizami, M. J. Hossain, and K. Mahmud [
19] modeled the distribution network as an agent connected to an EV aggregator agent, which in turn interacts directly with EV agents rather than with load locations. A. Marinescu, I. Dusparic, and S. Clarke [
30] considered the distribution network as the environment, while the EV and other network components are the agents that function within it. Several studies [
18,
27,
35,
36,
44,
70,
71,
83] used reinforcement learning (RL) in conjunction with MASs, and [
40,
75,
77] discuss the use of agent MASs in electricity management. The electricity system may be modeled as a single agent or as multiple agents in different environments, with aggregators frequently serving to group related components of the system.
Some papers did not include data from the distribution network, focusing instead on price behavior. W. Zhang et al. [
21], for example, used price behavior to predict future electricity prices, while other studies analyzed historical EV usage patterns to forecast future behavior [
33,
37,
67,
80,
86]. Similarly, Zachary J. Lee, Tongxin Li, and Steven H. Low [
25] analyzed open EV datasets from ACN-Data (
https://ev.caltech.edu/dataset) (accessed on 12 July 2023) but did not consider the electrical power system in their analysis.
3.3. Public Electric Vehicle Charging Station
Distributed public infrastructure at EV displacement locations is essential to support EV charging. The core feature of this infrastructure is the availability of public charging stations where any EV user can charge their vehicle, whether at fast-charging points or standard charging stations. Several studies adopted a cooperative approach to managing public charging resources. The work of C. B. Saner, A. Trivedi, and D. Srinivasan [
17] presented a MAS framework that optimizes the allocation and use these stations through cooperative strategies. Alternatively, EVs may compete with each other for charging locations and time slots while incorporating financial incentives such as Demand Response (DR).
The public charging infrastructure for EVs consists of stations equipped with multiple charging ports, most of which support rapid charging and are typically located near conventional gasoline stations, although concentrated in urban areas.
Multiple studies analyzed three primary aspects, economic factor, load curve, and station availability as well as a combination of these factors. The articles were classified according to the three aspects, and these are listed in
Table 4.
Economic studies focus on reducing costs by analyzing users’ ability to select optimal locations and times for charging and evaluate both competitive and cooperative charging strategies. Studies that address the electricity system load curve consider controlled charging as a means to smoothing power demand using mechanisms such as power availability or financial incentives. Finally, given the current limited number of charging stations in use today, researchers are developing techniques to predict future charging station locations by planning and managing lines and waiting times.
3.4. Electric Vehicle Charging Stations in Public Parking Lots
At public parking lot charging stations, EVs can remain parked for extended periods, which facilitates charging schedule optimization and enables V2G connections. EVs may also remain idle and benefit from renewable energy generation during charging [
39,
46,
49,
69].
O. Sadeghian’s review [
14] on smart charging of electric vehicles emphasized planning in parking lots and other locations. It highlighted vehicle departure time and the timing of next trip to ensure minimum charging. L. Calearo, M. Marinelli, and C. Ziras [
23] analyzed data sources used in recent research and their significance, citing public parking lot data as an example.
A. Almaghrebi [
80] used historical data from public parking lots to predict EV idle times, optimizing charging of another vehicle. In their research work, J. Huber, D. Dannb, and C. Weinhardt [
82] proposed methods for predicting EV status in parking lots and the distance of the next trip to improve parking lot scheduling and energy management. The state-of-the-art review by A. Comi and E. Elnour [
93] explored the integration of V2G technologies with parking lot location and nearby loads as a potential solution for EV charging in public parking areas.
3.5. Workplace Electric Vehicle Charging Station
At workplace parking lots, EVs typically parked for extended periods, allowing for a multi-approach charging schedule to be implemented. As workplaces often face power constraints and must manage multiple charging stations, one common planning strategy is the definition of charging priority [
26,
38,
72]. EVs with higher priority are charged first to ensure they reach the minimum charge for the next trip before leaving the parking lot. Studies that determine loading priority rely on historical data to forecast upcoming trips.
The use of V2G connections in workplace parking facilities represents a significant opportunity for managing overall site. It can generate financial benefits for both EV owners and workplaces by balancing building energy demands and vehicle charging requirements. It also accounts for factors such as battery life and consumption forecasts, as well as ensuring the minimum load required for the upcoming journey. V. Lakshminarayanan et al. [
75] conducted a study on using MASs to model the interaction among the key elements involved in the management of workplace charging stations. Similarly, S. A. Jawale [
34] used V2G connections in combination with photovoltaic generation and parking lot charging to flatten the load curve while minimizing charging intensity for each EV. O. Sadeghian et al. [
14] and Q. Hoaraua and Y. Perez [
48] carried out systematic reviews examining the role of V2G in workplace and public parking spaces. The authors highlighted how coordinated charging and unloading of EVs can support energy management at the location and ensure minimum charging levels at departure. L. Calearo, M. Marinelli, and C. Ziras [
23] conducted a second systematic review that focused on EV data sources and analytical methods used in recent studies.
Amara-Ouali et al. [
37] highlighted the importance of using open datasets in research to facilitate comparisons of EV energy management techniques across databases.
Other studies on workplace EV charging focused on examining user behavior [
47,
81], charging costs [
39], and the need to achieve adequate charge levels before the next journey [
82,
84].
3.6. Electric Vehicle Charging Station at Home
In most households, EVs are charged by using a slow, low-power station, which results in prolonged charging times. In most studies, and in the review by O. Sadeghian [
14], residential charging stations are modeled alongside the household’s total energy consumption. Charging control strategies together with demand–response incentives were applied to reduce peak consumption or shift consumption to off-peak periods. Other researchers [
18,
43,
48,
83,
88] integrated EV charging at home with photovoltaic (PV) generation to lower charging costs and promote the use of renewable energy.
According to several studies [
47,
82,
84], some researchers integrated home charging stations with other charging points and used charging forecasts for upcoming trips. Other researchers [
20,
81] emphasized the importance of analyzing EV driver behavior to determine optimal charging management strategies that encompass residential, public, and parking lot charging stations.
4. Survey of Energy Management Techniques
The study of the current state of the art in energy management for EV charging revealed that a variety of ML and optimization techniques are used, as shown in
Table 5. These methods can be combined to improve solution development or adapted and refined to increase efficiency in specific applications.
Several studies employed optimization tools and techniques for decision-making in the energy management of EV loads. However, these approaches often face scalability challenges and required substantial computational resources when applied to models containing a large number of elements.
Some researchers have adopted the term aggregator, which groups multiple entities into a single unit to simplify energy management. Another approach is to use MASs, which enable the distribution of processes across multiple agents operating in different locations. This reduces the number of elements in a single management process.
To clearly illustrate the association between methods, tools, and models, these were identified in the selected studies and manually counted in each study. The collected data was organized into an auxiliary table and then cross-referenced and visualized through a Sankey diagram generated from the Plotly library to support Named Entity Recognition (NER), as shown in
Figure 3. The studies associated with the methods in
Figure 3 are shown in
Table 6. Review papers were excluded from
Figure 3 if they were systematic reviews that covered multiple methods but did not include experiments.
4.1. Optimization Techniques
In electrical energy management systems, optimization algorithms are typically designed to minimize energy consumption or maximize economic return. These algorithms are formulated as mathematical equations that model the system and incorporate the relevant constraints. In a power system, there are typically multiple actors with ambiguous and conflicting interests, leading to multi-objective [
39,
43,
62,
65,
66,
73], multi-criteria [
38], or even multiple-stage optimization problems [
22,
24,
31,
42,
73,
85]. In some cases, these problems involve integer or binary variables and are solved by mixed-integer programming techniques [
9,
19,
25,
31,
40,
44,
46,
47,
76,
83,
84]. Furthermore, optimization challenges may be non-linear, requiring significantly greater computational resources, and, in some cases, are extremely challenging to solve.
Optimization tools and solvers, such as GAMS, CPLEX (
https://www.ibm.com/products/ilog-cplex-optimization-studio?lnk=flatitem) (accessed on 10 May 2023), GUROBI (
https://www.gurobi.com/) (accessed on 24 April 2023), and MILP, among others, are used to address the complexities in optimization problems. However, as these models are typically centralized to identify the global optimal solution, scalability can be significantly limited, finding the solution within a reasonable time frame can become unfeasible, or the computational requirements can become prohibitive. The challenge lies in the need to obtain sensitive data, as all relevant information must be available to the centralized system.
4.2. Artificial Intelligence
Several studies have explored the application of ML methods in EV energy management. E. Xydas, C. Marmaras, and L. M. Cipcigan [
20] implemented an SVM model within a MAS to optimize EV charging schedules through aggregating agents that respond to variations in electricity prices across the distribution network. P. Rezaei and M. A. Golkar [
78] employed an SVM to manage energy exchanges between buildings and EVs, demonstrating how load curves in EV-equipped buildings can be smoothed while ensuring sufficient charge for subsequent trips. To improve efficient use of parking spaces, A. Almaghrebi et al. [
80] applied multiple ML models—SVM, XGBoost, RF, and DT—to estimate EV idle time during charging. In a comparative study, S. Shahriar et al. [
29] analyzed the SVM, RF, XGBoost, and Deep Neural Networks (DNNs) to predict EV charging demand using historical charging data, weather conditions, and traffic information.
Researchers frequently employed ANNs as deep learning techniques because they produce encouraging outcomes in predictions based on historical data. Z. Zhang et al. [
35] integrated an ANN with a Recurrent Neural Network (RNN) and an LSTM to estimate energy prices. The LSTM component enhances performance by retaining information from previous time steps, thereby improving the modeling of time-series data. M. J. Scott and M. Hu [
83] developed an intelligent EV charging unit that uses an ANN to estimate state–action within a Markov Decision Process (MDP).
In addition to the SVM, A. Almaghrebi et al. [
80] and S. Shahriar et al. [
29] used the RF technique to predict EV idle time and charging behavior. The DT method and the XGBoost algorithm, as discussed in the previously mentioned studies [
29,
80], were also applied in conjunction with other methods.
Clustering is frequently employed to reduce the volume of data required for analysis by aggregating similar EV loading behaviors and applying ML techniques with lower computational effort. M. K. Daryabari, R. Keypour, and H. Golmohamadi [
94] have published papers on clustering techniques such as K-Means for grouping EV load patterns. Other grouping methods have been applied: C. Fang et al. [
9] used the Mean Shift method, Y. Xiong et al. [
24] implemented the CLSA technique, and Q. Liu et al. [
95] used binary operations to cluster user behavior.
RL in EV charging management has been widely covered in many studies. M. Alqahtani, M. J. Scott, and M. Hu [
83], for example, applied a Decentralized Markov Decision Process (DEC-MDP) framework in which an EV agent is represented by the tuple (S, A, P, R). For each instant or period considered, the S represents finite information including variables such as EV position, SoC, solar radiation, and energy load. P denotes the transition function from one state to the next, while A denotes the set of actions available to the agent. R corresponds to the reward function for actions taken in a given state and influences subsequent decisions. Within this MAS, each EV agent modeled in this framework can interact with other agents in the network.
In addition to the ML methods mentioned previously, several other methods have been applied to develop algorithms for EV energy management. P. Huang et al. [
96] applied a GA to optimize charging processes, while Z. J. Lee, T Li, and S. H.Low [
25] used the Gaussian Mixture Models method to analyze the ACN-Data platform. Z. Zhai et al. [
11] implemented an Agent–Cellular Automata model to simulate the dynamic behavior of EV transportation and used Monte Carlo simulations to determine regional load demand.
4.3. Emphasis on the Application of Optimization and Artificial Intelligence Strategies
Two methods are often used to organize the application of optimization and AI techniques within decision-making structures. The first is the use of aggregators, which group multiple entities in the power system to simplify its representation. The second uses a MAS to distribute the necessary processing volume. The majority of the reviewed studies incorporate one or both of these approaches, as described below.
4.3.1. Aggregators
Many elements in the electric power system share similar or identical characteristics and functions, enabling them to be aggregated for more efficient management. The term aggregator is used by several researchers to describe an entity which groups system components that provide the same service. The aggregator represents the entire group and acts as an intermediary between service providers and consumers. To ensure fairness, the agent must remain neutral, without favoring any individual participant. The aggregator is also responsible for managing communication, transactions, and negotiations and maintaining the stability of the relationship between consumers and suppliers.
The use of aggregators in EV energy management, either cooperatively [
17,
44,
46], competitively [
9,
13,
19,
22,
45,
47] or for financial optimization [
31,
85,
87], enables more efficient control of charging processes and a smoother influence on the grid load. This represents one of the main advantages of aggregator use. In the case of EVs, aggregators facilitate coordinated charging [
62,
64] to avoid grid overloads, reduce peak load demand, and fill load valleys. When V2G connections are available, the aggregator can modify the behavior of the EV group to lower other load peaks. E. Xydas, C. Marmaras, and L. M. Cipcigan [
20] applied an aggregator to integrate distributed generation with EV charging, which enhanced the effectiveness of carbon-reduction strategies. The aggregator is not only limited to EVs or generation units. It also encompasses residential loads, charging stations, and other resources, mostly within MASs.
4.3.2. Multi-Agent Systems
MASs model the behavior of the EV charging environment through entities known as agents. This approach allows each agent to act independently while distributing the computational effort and maintaining improved control over its own data. Another feature is its ability to secure interactions between agents and their environment. Several studies have explored MAS-based EV charging management. For instance, C. B. Saner, A. Trivedi, and D. S. Srinivasan [
17] proposed a cooperative scheduling framework, and M. S. H. Nizami, M. J. Hossain and K Mahmud [
19] developed a coordination model using local aggregators. MAS approaches are also suitable for hybrid environments where competition and cooperation occur in different groups of agents [
35]. H. S. V. S. K. Nunna et al. [
97] implemented an auction-based mechanism, specifically a second-price auction within a MAS, to define electricity prices for aggregators in EV charging. An example of a MAS applying aggregators is shown in
Figure 4, where the power agent aggregates the elements of the electricity network, the EV charging station agent aggregates the charging stations, and the home and EV agents represent the users-related elements.
The key feature of a MAS is that each agent operates in an isolated system. This separation ensures that the information managed by one agent remains independent from that of others, thereby ensuring confidentiality. Agent communication typically occurs through Extensible Messaging and Presence Protocol (XMPP) (
https://xmpp.org) (accessed on 16 April 2022), which is managed by an XMPP server responsible for message exchanges and agent authentication within the MAS. This communication feature is addressed by E. Xydas, C. Marmaras, and L. M. Cipcigan [
20]; D. Qiu et al. [
44]; and H. Lin et al. [
22], where each EV agent negotiates with the power system through an aggregation agent, allowing all data to be treated anonymously within the system.
In MAS decision-making, one of the main learning techniques used is MARL, which uses RL principles to determine optimal decisions. MARL is examined in studies [
21,
27,
30,
35,
44,
83] as a mechanism through which agents interact to identify and select the most effective decisions within their environment based on the rewards received from their actions. Other studies [
8,
11,
18,
22,
36,
40,
75,
77] applied MAS frameworks for energy management using other decision-making techniques grounded in optimization and ML.
5. Implementation Challenges and Requirements for Energy Management
The relation between energy management in electric vehicles and the magnitudes involved is extremely complex due to the high interconnectivity of the system. The transmission of energy from the point of generation to the point of consumption is constrained by multiple factors, including active and reactive power flow, current, voltage, and frequency limits, all of which can being modified by ancillary services.
T. U. Solanke [
13] and M. S. H. Nizami, M. J. Hossain, and K. Mahmud [
19] evaluated the use of EVs in V2G connections to provide ancillary services within the distribution network. However, the adoption of this technology remains under analysis due to the significant investment requirements and legislation challenges associated with its implementation. According to the review by J. Oladifbolu, A. Mujeeb, and L. Li [
91], the potential of V2G as a solution for smart grids continues to expand, particularly with the growing integration of renewable energies and the increase in charging stations.
The researchers identified several common factors that hinder the implementation of EV energy management systems, as detailed below.
5.1. Data Acquisition
This data can be divided into historical records and real-time data. The ML models learn system behavior from historical data and apply this knowledge to real-time inputs to predict future behavior or, in the case of optimization, to identify optimal solutions. Therefore, accurate, comprehensive, and consistent data are essential for appropriate decision-making and must be collected systematically.
Some studies, such as those by H. Khan1, M. J. Khan, and A. Qayyum [
98], generated data on EV arrivals and departures at charging stations using a Poisson distribution to compensate for limited information. In contrast, the work of Z. J. Lee, T. Li, and S. H. Low [
25] led to the release of ACN-Data, a publicly accessible dataset for EV charging research that provides access to tens of thousands of real EV charging sessions performed in California and managed by PowerFlex. These two extremes highlight the ongoing challenges of collecting reliable EV data, underscoring the substantial progress still required in this area.
A systematic review by L. Calearo, M. Marinelli, and C. Ziras [
23] analyzed several data sources employed in EV energy management. The authors identified a key challenge for future studies: ensuring that data sources are publicly accessible. This would allow researchers to test new approaches on common databases and directly compare different techniques and their performance. The review also highlighted the need for data synchronization across diverse EV-related sources. While some sources contain charging information from public stations, others focus on residential or workplace parking sites. Complementary information on vehicle traffic patterns or weather conditions is often lacking, which places a significant burden on researchers in their attempt to develop and validate energy management models effectively.
5.2. Sensitive Information
Obtaining the necessary data for EV energy management presents significant challenges, as much of it is primarily private information subject to varying data protection regulations across different countries. EV adoption remains relatively recent in most countries, and current EV owners typically represent higher socioeconomic groups rather than the full spectrum of potential future users.
As previously mentioned, some researchers have proposed the use of aggregators to address data confidentiality concerns. Other researchers proposed methods for EV user anonymity, such as EV grouping [
94]. However, these strategies do not guarantee user anonymity or protection of sensitive data.
The reviewed studies did not offer a conclusive solution for ensuring data confidentiality, and a considerable amount of work is required to enable access to valuable data while protecting personal information. The systematic review conducted by L. Calearo, M. Marinelli, and C. Ziras [
23] provided an important foundation for overcoming this challenge.
5.3. Big Data Analysis
Data from EVs, EV charging stations, residential loads, power systems, and meteorology must be collected at short time intervals. These continuous measurements generate vast amounts of information for both real-time analysis and historical storage. All of this data is stored in appropriate locations, collectively forming what is known as “big data”.
For efficient storage and management of such data, an appropriate infrastructure is required. F. Soldan’s study [
33] examined the use of Apache Spark as a tool for managing big data in this context. One approach involves the use of computer clusters. However, managing such an infrastructure is expensive and requires specialized personnel.
One approach is to reduce the volume of information through the use of cluster techniques, as demonstrated by M. K. Daryabari, R. Keypour, and H. Golmohamadi [
94]. Their study grouped EVs into clusters and used a representative from each cluster to obtain and process information. Another strategy is to extend the recording intervals to lower the volume of stored data. For example, data originally collected at one-minute intervals can be grouped into hourly records, effectively reducing the dataset size to 1/60th of its original volume.
The dataset storage structure was excluded from the keywords for this survey because only a very low number of publications were found.
5.4. Real-Time Analysis
Effective management of an electric power distribution network must operate in real time to ensure system stability and to meet the requirement that generated energy be consumed as it is produced. Although ancillary services can provide short-term flexibility, these should be reserved for emergency situations, such as variations in generated energy or component failure, rather than be used to compensate for inadequate load planning.
V. Lakshminarayanan et al. [
75] described a scenario in which the electrical energy of a workplace is managed in real time through the use of EVs parked on-site to regulate loads in real time. In this model, an RF algorithm was used to predict both EV energy consumption and load forecast. Several Decision Tree (DT) classifications were trained on various data samples, and their outputs were averaged to improve prediction accuracy. The resulting forecasts were then used as inputs to an optimization algorithm to determine the optimal load balancing strategy for the workplace microgrid.
Some studies used real-time data. W. Zhang et al. [
27], identified EV charging stations near the vehicle, and the study by H. Lin et al. [
22] analyzed the influence of EV penetration on power management centers. Both studies employed MASs to enable real-time solutions in which processing was distributed and required less data, thereby reducing the time needed to solve the problem. The paper by N.K. Sifakis and F. D. Kanellos [
50] proposed the use of an aggregator to represent a group of similar elements and reduce the amount of data. This allowed for real-time optimizations.
Operating in real time requires a balance between quality, result, and response time. For example, even highly accurate outcomes lose their value if they are produced after the execution deadline. For effective decision-making with large datasets, the use of tools such as Apache Spark, MASs, and data reduction techniques is essential.
5.5. Ensuring Interconnection of System Elements
In a real-world system, such as electric power communication systems, connections between elements are not maintained continuously, which makes it extremely challenging to ensure system-wide connectivity. Several studies, including that by Q. Liu et al. [
95], used the Internet of Electric Vehicles (IoEV) to connect EVs to the network and facilitate communication among them. This approach, however, relies on Road-Side Units to connect EVs and Road Administration to the network with the rest of the devices. In addition to this mode of communication, connection can also occur through an extensive cellular network.
The communication system poses a significant challenge for energy management, as EVs must continue to make decisions even in the event of a communication failure. Furthermore, security issues can result in unauthorized access, data theft, and identity cloning, among other risks. Although this topic was beyond the scope of this review, few studies addressed it specifically. Nevertheless, researchers whose methods involved public communication networks were advised to pay particular attention to it.
5.6. Discussion of the Identified Challenges and Future Investigations
Several studies examined the application of AI to the energy management of EVs, including those analyzed in this review and the systematic reviews conducted by T. U. Solanke et al. [
13], O. Sadeghian et al. [
14], Y. Wu et al. [
45], and Q. Hoarau and Y. Peres [
48]. These studies focused primarily on cost reduction, energy efficiency, and energy balance of the power system. Another common approach in use was optimization, which, despite its potential, presented several implementation challenges. Optimization models required expensive and complex infrastructure, operated in a centralized manner, and had prohibitive computational requirements. In addition, they faced difficulties in acquiring data.
The existing studies were found to be more applicable to EV charging station operators and managers than to individual EV users. Consequently, there was a lack of approaches specifically targeted at EV owners to manage the electric energy of their vehicles’ coordination with charging stations. Few studies addressed tools or actions that would allow users to obtain optimized charging options or schedules for their daily use.
The reviewed studies addressed EV groups rather than individual users, which revealed an important gap in the existing literature. This can be attributed to models that require large volumes of data to identify optimal patterns. To address this gap, the present study proposed the use of a MAS combined with DRL to personalize both users and system components. DRL uses initial model-based learning, which is subsequently reinforced through user behavior. Aggregators were introduced to represent similar elements and simplify user access.
In addition to the lack of approaches addressing the issue of time management of EV users, this systematic review highlighted another essential issue: researchers rarely discussed methods for storing and transferring information between process elements. The researchers did not specify how the data was processed or managed within their research frameworks. Frequently, the datasets were made available in structured text files without adequate access security. In their review, L. Calearo, M. Marinelli, and C. Ziras [
23] demonstrated the need for open and standardized data to facilitate cross-study comparisons. An urgent need exists for secure storage and transfer that allow for access traceability, particularly in light of the increasing number of security breaches.
Distributed processing increases computational power, distribution, and segmentation of information to improve to data privacy, which is an aspect that is increasingly observed as a trend and facilitated by MASs. However, an urgency to identify communication patterns was identified between the elements of an electrical distribution network, EV charging stations, and EVs. In the models proposed by several researchers, information on communication between agents was not specifically described. One approach to narrowing this gap involved the use of communication protocols, which define access control mechanisms and govern how network elements exchange information. The structures presented in the reviewed studies typically consisted of an agent representing the electricity system or distribution network, which was connected to an aggregator of EV charging stations and, subsequently, to the EVs themselves. The distribution network’s loads could also be represented by aggregators, thereby completing an interconnected system that encompassed both electrical energy flow and communication among its elements. Different energy management strategy models could be implemented within this structure provided that they complied with external communication standards and that the participating entities were clearly defined.
5.7. Results of the MAS with DRL Model
Earlier in this article, we answered the question of whether it is possible to help individual users manage EV charging using MASs and AI. In this section, we demonstrate an application that implements the solution presented in this article. While a MAS enables interaction between system agents and users, AI uses DRL to generate suggestions for optimizing EV charging. Initial learning takes place in a model or through knowledge sharing between users with similar profiles, and this is refined during use. The proposed communication protocol between agents involved in EV charging management is founded on the standard MAS communication principle, as outlined in the paper by Veiga et al. [
99].
To verify whether this study met the needs of individual EV users, a learning model was developed using a MAS integrated with DRL. The resulting time savings were then analyzed. The data was obtained from the source Pecan Street, as published by Parson et al. [
100], which provides building-level energy data designed for non-intrusive load monitoring. The processing of these data was conducted in accordance with the methodology delineated in the publication by Veiga et al. [
101]. The model was based on a traffic simulation in SUMO (Simulation of Urban Mobility), as detailed by Lopez et al. [
102]. The map used in the simulation was generated by OpenStreetMap
® [
103].
The test was conducted using data from 2018, collected in a small area of Austin, Texas, USA. The preliminary results, obtained over a 28-day period, analyzed battery SoCs of 0.2, 0.4, 0.6, and 0.8. The finding indicated an increase in the number of charging events when AI was applied, as shown in
Figure 5. However, the average battery charging time was significantly reduced (
Figure 6), and the total charging time for the period was shorter except in the case where the initial battery charging state was 0.4 (
Figure 7).
The proposed model enabled the control of EV charging without the need for long charging periods. It efficiently managed the charging periods to ensure each journey could be completed without battery depletion. The underlying concept is that the system informs the user when and for how long to charge based on the individual’s daily schedule.
6. Conclusions
Society currently uses AI, and several researchers have demonstrated its benefits for managing large volumes of data. In the case of electric vehicle energy management, a large amount of data is involved, and quick decisions are sometimes necessary. One solution is to implement a MAS with DRL, combined with an aggregator and a communication standard between agents to facilitate user access. To this end, the following strategies are suggested:
Aggregator: We recommend utilizing a representative for a group of elements with similar characteristics or goals, as in the case of electric vehicles and EV recharge;
MASs: Multi-agent systems are appropriate for managing elements capable of conducting negotiations and transactions via communication and actions;
Smart grids: Electricity distribution networks may be modernized to enable intelligent duties and automatic negotiations with loads and generation, increasing the efficiency of energy generation and distribution;
Communication in the electricity system: The standardization of communication will play an essential role in allowing other systems to interact with the electricity system, thereby facilitating the shared management of resources;
DRL: This is necessary for developing the intelligence of MASs based on models to initiate learning and evolution through habit reinforcement learning.
There is still a long way to go before using AI in power systems, but using it makes the smartest grid. However, it must wait until the concepts are fully developed and their implementation is safe.
Our research group is currently engaged in developing a multi-agent system that will enable the individual user to implement EV energy management. This system will contain a machine learning model that will assist the individual user in managing the energy of their EV, taking into account their daily activities and optimizing their time within the commitments of their schedule.
Subsequent research endeavors should encompass the evaluation of the efficacy of this contribution in the management of charging other types of electric vehicles, in conjunction with the assessment of its impact on the electrical load of the smart grid. A further critical domain of inquiry pertains to residential charging, which is incorporated into the proposed model. This aspect bears significant implications for both the grid and charging stations utilized by electric vehicles. Concerns persist regarding the privacy and security of sensitive information given the necessity of this data for ML.