Multi-Criteria, Co-Evolutionary Charging Behavior: An Agent-Based Simulation of Urban Electromobility

In order to electrify the transport sector, scores of charging stations are needed to incentivize people to buy electric vehicles. In urban areas with a high charging demand and little space, decision-makers are in need of planning tools that enable them to efficiently allocate financial and organizational resources to the promotion of electromobility. As with many other city planning tasks, simulations foster successful decision-making. This article presents a novel agent-based simulation framework for urban electromobility aimed at the analysis of charging station utilization and user behavior. The approach presented here employs a novel co-evolutionary learning model for adaptive charging behavior. The simulation framework is tested and verified by means of a case study conducted in the city of Munich. The case study shows that the presented approach realistically reproduces charging behavior and spatio-temporal charger utilization.


Introduction
In order to decelerate climate change, greenhouse gas emissions have to be reduced substantially and as fast as possible, a fact that is long established in and outside the scientific community and that has gained broad acknowledgment and binding worldwide commitment with the ratification of the Paris agreement [1] in 2015. Among other measures, decarbonization of the transport sector helps us to remain within target emission thresholds. Comprehensive policies are in place that regulate vehicle fleet emissions in all major automobile markets of the world. The European Union, for instance, has set an emission threshold of 95 g CO 2 km −1 for vehicles licensed in 2020 and 2021 [2]. Conventional vehicle fleets with internal combustion engines are hardly able to comply with these ambitious targets. Thus, to avoid financial penalties, car manufacturers are adding more and more energy efficient battery electric vehicles (BEV) to their product portfolios. Furthermore, governmental incentives attempt to convince the general public to buy BEV by offering both financial and organizational benefits to BEV drivers [3]. However, despite all efforts, current adoption rates of BEV continue to remain behind expectations. One of the key drivers of BEV adoption is the availability of adequate charging facilities, close to homes and work places [4,5]. A large body of literature [6,7] now deals with the placement and roll-out of charging infrastructures, comprising private, public, and work chargers, as well as single-purpose charging infrastructures for taxis and buses. At the same time, city councils, commercial businesses, and home owners are already creating charging facilities. Consequently, the general public now has access to a heterogeneous universe of charging facilities with various operators, hardware systems, access restrictions, price structures, and levels of service.
Charge point utilization is determined by the interplay of various sub-infrastructures. For instance, users might choose to either use public charging facilities or to charge at home and they are known to prioritize one charging point over another based on factors like individual mobility pattern, price, convenience, occupancy, and personal preference [4,5,8]. As BEV adoption rates increase along with the numbers of charging points, the overall system complexity increases. This means that an analysis of future charging infrastructure and design cannot be decoupled for different infrastructure types. Moreover, it is subject to users' charging behaviors in a dynamic and competitive environment. This applies especially to urban areas, where the majority of the population cannot install private charging points, owing to space limitations and low rates of home ownership [9]. Consequently, cities are reliant on a combination of infrastructure types and are exposed to mutual influences between all stakeholders and resources. Hence, decision-making regarding electrification policies and charging station setups requires comprehensive electromobility models that provide a suitable framework for scenario analysis. Owing to the dynamic and interdependent nature of the system's behavior, decision models need to consider all influencing factors of relevance to urban electromobility. These include current and future infrastructure types, charging behaviors, and vehicle types as well as the interdependencies between users, vehicles, and charging points. Such models enable informed choices regarding the allocation of financial and organizational resources. It is this line of thought that has led scholars to employ electromobility simulations in an attempt to guide decision-making. These simulations serve as sandboxes with which to assess the interplay of charging infrastructure layouts and BEV adoption rates [10], potential charge point operator revenues [11], and user behaviors [12][13][14][15]. Furthermore, they can be used to estimate spatial charging demand and to optimize charging station layouts, if the simulations are embedded in an optimization scheme [9,[16][17][18][19].
While a multitude of simulation techniques is potentially able to model aspects of urban electromobility, agent-based simulations are particularly suited for the analysis of complex and dynamic systems [20]. Instead of modeling global phenomena explicitly, they are bottom-up approaches in which system properties emerge from the local interaction and decision-making of the simulated entities (agents). Thereby, global impacts of agent properties and behavior can be analyzed and predicted, even if the system behavior eludes analytical considerations or contains unforeseen properties [20,21]. In transport engineering, agent-based traffic simulations allow for a detailed analysis of the impact of individual mobility patterns, vehicle types, route choices, and driving behavior on systemwide variables like street capacities and congestion [18]. This is why such simulations are often used to plan policy, infrastructure, and business decisions [21]. Electromobility simulations as sub-types of transport simulations also benefit from agent-based simulation methods: they provide the possibility to analyze microscopic aspects like individual SOC or charging patterns as well as global circumstances like charger utilization under consideration of heterogeneous vehicle fleets, infrastructure types, charging behaviors, and charger access of the simulated population [22]. Thus, many electromobility simulations are agent-based. In them, driver-vehicle pairings are frequently represented as a single agent, commonly known as a driver-vehicle-unit (DVU) [13]. Figure 1 shows the basic structure and components of agent-based electromobility simulations. Each DVU can be modeled independently of all others in terms of its system states and attributes, as each DVU has its own behavioral and consumption model. The interplay of all DVU in an agentbased electromobility simulation captures system-inherent rivalries among BEV drivers for public charging points and can be analyzed regarding many possible observation variables depending on the model detail.
Section 2 of this paper presents a review of key publications that deal with the subject of agent-based electromobility simulations and sets out the need for a novel simulation approach that is specifically aimed at the analysis and design of urban charging infrastructures with a particular focus on non-rule-based user behavior. Section 3 introduces UrbanEv-Contrib, a simulation framework that closes the gaps identified in the literature review. The proposed framework is based on the well-established Multi-Agent Transport Simulation (MATSim) framework [23,24] and the associated electric vehicle contribution (EV-Contrib) [25]. In contrast to existing approaches, the majority of which mostly model user behavior based on SOC thresholds, it makes use of a multi-criteria co-evolutionary algorithm to model charging behavior. In Section 4, the resulting model is applied to a case study in the city of Munich to test and assess the presented approach. A detailed analysis of the simulation results reveals that the proposed framework realistically models charge point utilization and charging behavior (Section 5). Section 6 discusses the results and draws conclusions taking into account the case study, the existing literature, and the practical applicability of the framework to real-world use cases.

Related Literature and State of the Art
This section presents a review of key publications relating to the subject of agent-based electromobility simulations. Existing simulation frameworks and user behavior models are presented, research gaps identified, and the need for a new agent-based simulation approach for urban electromobility is set out.
Unlike other forms, microscopic, agent-based transport simulations enable highresolution tracking of individual DVU within a road network over time. The modeling of updates to DVU states is not restricted to points of origins and destinations, as is the case with existing discrete event-based simulations, but can be done at any timestep of the simulation. Thus, DVU generate time-series of vehicle and agent states. Microscopic transport simulations therefore enable the integration of detailed consumption models, dynamic user behaviors, and flexible mobility plans of the agents. It is on the basis of these strengths that they have emerged as standard tools for infrastructure and policy planning [18,30,34]. Because of its prominence, well-established validity, and usability in transport and infrastructure planning, the MATSim framework and EV-Contrib form the basis of the development of the proposed UrbanEV-Contrib. The following section contains a short introduction to these frameworks.

MATSim and EV-Contrib
MATSim is an agent-based transport simulation framework implemented in JAVA. It has been validated and used in several scientific contributions. For further information regarding the framework itself and existing scientific applications of the framework, the interested reader is referred to [34]. Based on [34], this section briefly introduces the general operating principles of MATSim and the associated EV-Contrib to facilitate understanding of the development and functioning of the UrbanEV-Contrib, as presented in this article. Figure 2 shows the MATSim simulation loop. A synthetic population of agents is simulated in each iteration of the simulation cycle (mobsim) along with their mobility behavior. Agents express their mobility demand in the form of mobility plans, consisting of origins, destinations, purposes, times, route choices, and transportation modes for each trip. During mobsim, all pre-defined mobility plans are executed within the common environment. By repeating this process, MATSim iterations are used to train agent behavior based on a co-evolutionary learning approach. Co-evolutionary learning involves scoring each mobility plan of a person based on the utility of the executed behavior. Depending on the score obtained, replanning of agents' plans can be used to improve their performance. As a result of this repeated scoring and replanning, plans are optimized heuristically over several generations. The approach is thus characteristic of an evolutionary method. It can also be regarded as a co-evolutionary approach, due to the interdependencies of behaviors and plan scores within the mutual simulation environment of agents. Score consolidation resulting from population-wide phenomena emerging through the interaction of agents in the virtual environment can be observed during training. This consolidation-a steady system-state-exists when no single agent can adapt its behavior independently to improve its own performance. According to the mathematical theory, this state is known as a Nash equilibrium which has been shown to reasonably reflect real traffic behavior. Once a Nash equilibrium exists within the simulated system, all system and agent states can be analyzed to derive information about the real system. MATSim's simulation logic was originally designed to model traffic flows and congestion. Nonetheless, it has also been applied to and extended for several other mobility use cases. EV-Contrib, for instance, adds basic electric vehicle support to MATSim. This is done by implementing a special vehicle type that consumes electric energy according to an energy consumption model. Furthermore, charging stations are introduced, and the application programming interface (API) contains endpoints for the implementation of linear and non-linear charging processes. Charging events are introduced into the MATSim simulation as separate activities in agents' mobility plans. Charging activities can either be invoked by the agent based on a specified charging behavior, or by the module's framework whenever trips would result in a zero or negative SOC if carried out without charging. Either way, the EV-Contrib models charging as a stand-alone activity and expects users to disconnect from the charging point as soon as charging concludes. While this modeling approach works well for long-distance journeys, in which stopping for the sole purpose of charging can be realistically expected [19,30], as well as for special-purpose vehicle fleets like taxis [28], it is in direct opposition to typical urban charging scenarios and behaviors. This will be elaborated upon in the following sections.

User Behavior
Besides technical considerations, users' behavioral patterns, mental models, price expectations, and personal preferences have been shown to have a great influence on policy effectiveness and charge point utilization [35][36][37]. For this reason, many authors explicitly integrate user behavior modeling into their simulation approaches. In this respect, we distinguish between two aspects of user behavior in electromobility simulations:

1.
Charging behavior-users' decision-making regarding their general wish to charge.

2.
Location choice-users' decision-making regarding the choice of a specific charging point.

Charging Behavior
In agent-based electromobility simulations, DVU consume energy when traveling through a virtual street network. Independently of the consumption model, agents have to charge their vehicles at some point of the simulation to avoid becoming stuck with an empty battery. The point in time at which an agent decides to initiate charging or to begin searching for a charging facility is a question of charging behavior. In contrast to real-life charging behavior, charging behavior models are often based on fixed sets of rules, which usually initiate charging when certain SOC thresholds are reached. Hidalgo and Trippe [17], for instance, assume charging to take place whenever an agent parks at a destination with an SOC of below 30% or when their SOC falls below 60% and the parking duration is longer than 2 h. Similarly, Jäger et al. [26,27], Bischoff and Maciejeweski [28], and Zhang et al. [31] stipulate fixed charging thresholds. Threshold-based charging behavior models users who avoid range anxiety and possible car break-downs long before they appear. However, they are not suited to model realistic opportunity charging and do not take into consideration that real users are aware of their mobility behavior as a whole and are capable of estimating the amount of energy they will need until the next charging opportunity.
As well as SOC thresholds, trip feasibility is often used as a trigger for charging procedures in agent-based simulations. In corresponding contributions, agents charge en-route if they would be unable to complete their ongoing trip due to an empty battery. Marquez-Fernandez et al. [19] and Bischoff et al. [30] apply this concept to a simulation of long-distance journeys on a motorway network. Zhuge and Shao [29] use the same approach on an inner-city road network. They also assume that vehicles are always charged while parking. In contrast to real-life charging behavior, in which users might decide to charge in anticipation of their impending energy consumption, threshold-and feasibilitybased behaviors assume that users blindly use their vehicles until charging becomes unavoidable.
Few approaches expect agents to consider charging opportunities within the context of their overall mobility behavior. A notable exception is presented by Bi et al. [13], who compare three charging behavior models with different estimation horizons. The authors show that more realistic charging behavior can be simulated when future demand is taken into consideration. In another publication, Bi et al. [14] expand this approach by adding opportunity charging to their behavior model. Consequently, agents will accept small detours and walking distances if there is an unoccupied charger close to their trip destination, even if the SOC does not necessarily require charging. In a similar method to Bi et al., Chaudhari et al. [15] implement plan-aware agents who choose to charge whenever their SOC is below their estimated SOC requirements for future trips plus a safety margin. Unfortunately, none of the aforementioned models considers influencing variables other than the vehicle's SOC.
Pagani et al. [11] employ a stochastic charging behavior model and consider comfort and price in addition to plan feasibility. Whenever an agent's battery falls below 50% SOC, the probability of charging at nearby chargers is evaluated. The probability of using a charger is determined on the basis of a price-driven behavior and a comfort-driven behavior. The price-driven behavior model assumes the charging probability to be inversely proportional to the charging price at a candidate charging point. The comfort-driven model assigns fixed charging probabilities based on the charger and destination type.
These probabilities are taken from current studies on real-life user behavior. Despite the potential of their approach, the authors state that "the use of stochastic models [. . . ] can result in uncertainties in the outcomes; sometimes this can be a challenge to interpret". Furthermore, model accuracy is largely dependent on sufficient and accurate user data for the target region.
A promising approach toward modeling realistic and interpretable charging behavior was introduced by Jäger et al. [12]. The authors simulate a fleet of electric taxis in Munich, in which, instead of rule-based charging behavior, each taxi agent attempts to optimize its charging schedule by means of an evolutionary optimization. To this end, charging plans are generated, tested for feasibility and, if they prove infeasible, modified. The simulation is then reiterated, resulting in a successive enhancement of the charging behavior. Hence, the simulated agents optimize their charging behavior under consideration of their mobility plans and the expected charging behavior of other agents. Thus, agents are able to anticipate charge point occupancy and expected energy demand. Despite the contribution's advantages, Jäger et al. do not integrate the behavioral model in a transport simulation and do not offer any indication as to whether the resulting charging plans are optimal. Instead, their evolutionary algorithm converges as soon as all charging schedules are feasible and does not consider the overall quality of the plans. Furthermore, no additional influencing factors other than the SOC are considered. Nonetheless, the general approach to using evolutionary algorithms is appealing because of the potential ability to extend it to multivariate charging behavior.

Location Choice
Besides charging behavior, location choice is another important aspect in the modeling of agent behavior in electromobility simulations. Once an agent has decided to initiate charging, it has to choose a specific charge point to charge the car. In many cases, both decisions are directly coupled by model assumptions or an integrated decision-making process that takes into account both charging behavior and location choice. The simplest forms of integrated location choice assume that users always charge at home [10] or-even more bindingly-at any destination that offers charging facilities [29]. In the vast majority of studies with explicit location choice, charging is modeled to take place at the nearest charger as soon as the respective charging behavior calls for recharging [12][13][14]19,28,30,31]. In simulations in which trips cannot be interrupted or in which chargers are available at only a few, dedicated facilities, agents charge at their trip destinations [15,17] or at the nearest charging station after completing their trip [26,27]. The aforementioned studies only consider proximity in the context of choosing a specific charger, and any other potential influences are left out. To close this gap, Pagani et al. [11] include cost in their location choice model by reducing the charging probability at charging points depending on the charging price. Although this is a large step towards realistic user behavior, it still lacks other relevant aspects. Specifically, technical considerations like plug types and charging powers are not considered in the reviewed models. Moreover, agents are unaware of the charge point occupancy when they make their decision. Instead, they queue at charging points until a spot becomes available. This is in obvious contrast to real-life charging behavior which is aided by a multitude of smartphone apps and websites indicating the current occupancy of charging points, especially in big cities. Table 1 summarizes the approaches reviewed in terms of the implemented charging behavior and location choice as presented in this section.

Review Conclusion and Research Gap
Agent-based simulations have emerged as standard tools for policy and infrastructure planning in both general transportation and electromobility. Of all agent-based approaches, microscopic transport simulations are the most prominent owing to their strengths in terms of behavioral modeling, consumption models, and the analysis of mutual interdependencies between agents and the environment. Furthermore, well-documented simulation frameworks are available. The most established framework for agent-based transport simulations is the MATSim framework, which has been employed successfully in many publications on the subject. However, despite the quantity and breadth of publications on the subject of electromobility simulations, existing contributions fall short in their ability to model realistic user behavior and often introduce assumptions that are not up to date or do not apply to wide study areas. Section 3.1 elaborates on these shortcomings in detail. In general, the modeling of urban electromobility is especially challenging. In city scenarios, with dense traffic, little private (and thus generally heterogeneous) charging infrastructure, and a high degree of rivalry for charging points, strong interdependencies between individual charging schedules are to be expected. Hence, a realistic simulation of urban electromobility relies on the accurate representation of user behavior, traffic, and consumption models. Unfortunately, existing approaches reduce charging behavior to a fixed and often overly simplistic set of rules, which trigger the charging process either at a fixed SOC threshold, at any opportunity, or on the basis of rudimentary energy forecasting. Contrasting current models, multiple studies have shown that real charging behavior is influenced by a multitude of factors, including SOC, price, opportunity, activity type, occupancy, time of day, comfort, range anxiety and information. It is for that reason that we propose a novel, multivariate user behavior model within an agent-based simulation based on the most commonly used transport simulator, MATSim. In contrast to existing approaches, this model employs a co-evolutionary algorithm similar to the one presented in [12] to simulate user behavior. Unlike [12], we design the behavioral model for the general public and extend the basic approach to include consideration of multiple influencing factors in addition to the feasibility of mobility plans. The following sections introduce the novel simulation framework and test it in a case study for the city of Munich.

Agent-Based Simulation of Urban Electromobility
This section describes a novel electromobility simulation approach for urban areas. The simulation framework used to implement the approach is based on MATSim and EV-Contrib, hence its name UrbanEV-Contrib. Further information, including the source code, can be retrieved from https://github.com/TUMFTM/urbanev. In concordance with the MATSim base contribution [24] and EV-Contrib [25], all code is open-source and licensed under the GNU Public License (GPL) [38], providing full reproducibility and transparency.

Assumptions and Requirements
Based on the review of literature that addresses agent-based electromobility simulations, behavioral surveys, up-to-date vehicle specifications, and considerations of urban environments and their specific characteristics, we establish the following assumptions and requirements regarding key aspects of charging behavior and location choice for the development of the UrbanEV-Contrib: • The mere feasibility of charging plans does not constitute a sufficient basis on which to model charging behavior, since current BEV typically offer driving ranges that are considerably longer than the average daily mileage of an urban BEV driver. This means that charging activities can be postponed and do not have to take place on a daily basis. Hence, soft criteria like comfort, price and personal preference have to be considered in addition to SOC-driven charging behavior. • Owing to the relative flexibility afforded by charging in city environments and long driving ranges, multi-day charging behavior has to be considered. Furthermore, real BEV drivers can be expected to take historic experience and future mobility demand into account and do not act solely on the basis of short-term considerations. • For the same reason, queueing is also not a characteristic user behavior for most charging points within cities. Therefore, location choice has to be explicitly modeled and the level of information an agent has when picking a charging site needs to be aligned with real drivers who use apps and websites to determine the current occupancy of charging stations before choosing a location. • Users do not interrupt ongoing trips if charging is not strictly essential owing to a very low SOC. Instead, they aim to integrate charging into their daily activities, i.e., agents should be modeled such that they charge during other, primary activities. • Because charging needs to be modeled as a concurrent activity, charging and parking durations are not necessarily equally long. Specifically, charging points are occupied until primary activities have been completed even if the charging process ends earlier.

•
It is not safe to assume that every agent has an opportunity to charge at home in city environments. Instead, private charging points are part of a larger, more heterogeneous charging infrastructure. Therefore, not all vehicles can be expected to be fully charged at the start of a day, and private chargers have to be supplemented by public and work chargers.

User Behavior
To enable modeling of realistic, urban charging behavior, agents' mobility plans are augmented by concurrent charging activities. A charging plan P with n activities is represented by a binary sequence of charging decisions a i , indicating whether or not charging should be carried out during an activity: On the basis of these charging plans, agents charge their BEV during mobsim. The SOC history is tracked throughout the simulation based on the consumption and charging models introduced by EV-Contrib. MATSim's scoring and replanning steps are used to optimize agents' charging plans. Each charging plan is scored on the basis of several factors that are known to influence the charging behavior of BEV drivers. Depending on the scores obtained, charging plans are replanned in order to simulate the reasoning of a BEV driver. Because not all agents are equipped with home chargers, public chargers, being a limited, shared resource, induce mutual dependencies between agents. Thus, the optimization of charging plans-like the optimization of mobility plans-takes a co-evolutionary approach.

Charging Behavior
The score of a charging plan S plan is calculated as the sum of the scoring of each charging decision on the mobility plan S a .
Plan scores are evaluated at the start of each activity on an agent's mobility plan, irrespectively of whether or not charging actually takes place during the activity. Noncharging activities are included so as to consider the effects of not charging. The score of a charging decision can be understood as agent utility. Each activity's score is evaluated with respect to factors that influence utility. As known from the related literature, these factors have several aspects, such as comfort, price, and personal preference. Therefore, the total score of an activity is a sum of separate score components S if derived from these influencing factors. Each basic score component S if is weighted by a factor β if to model the strength of influence each factor has on the total score of a charging plan. From a utilitarian perspective, this factor can be interpreted as the marginal utility of an influencing factor.
Depending on the direction of an effect, β if may be either positive (adding utility) or negative (adding disutility).
While the proposed framework is open to further extensions, the current implementation includes the aspects of comfort, feasibility, and user preference. Comfort is incorporated into the model by scoring the walking distance between the charger and the activity destination (S walk,a ) and by range anxiety (S rangeAnxiety,a ) which adds to the discomfort. Feasibility is factored in by adding a punishment term whenever the battery of an agent runs empty (S emptyBattery,a ). Users are known to prefer home charging over other forms of charging. Hence a reward term is added, when an agent charges at home (S homeCharging,a ). Equation (4) constitutes the implemented scoring model. Equation (5) defines the punishment term for an empty battery.
S a = S walk,a + S rangeAnxiety,a + S emptyBattery,a + S homeCharging,a S emptyBattery,a = β emptyBattery i f SOC a = 0 0 else Range anxiety has a strong influence on charging behavior. Range anxiety refers to the phenomenon of users trying to avoid a low SOC to prevent the psychological stress associated with the possibility of an empty battery [35]. It is a driver of opportunity charging, because it makes users charge before they technically need to. Range anxiety is modeled using Equation (6). Here, thres SOC denotes the SOC threshold under which agents feel anxious. The strength of their discomfort is inversely proportional to their SOC. Figure 3 shows the unweighted range anxiety component (S rangeAnxiety,a ) for different SOC thresholds.  When an agent decides to charge at a charging facility that is not located at the trip destination, walking adds to the discomfort of the agent. Clearly, an agent experiences more discomfort for longer distances between charger and trip destination and no additional discomfort from walking when the charger is located directly at the trip destination. Hence, the (dis-)utility of walking is closely related to the degree of accessibility of a charger from the trip destination. Geurs et al. [39] provide an overview of the great variety of accessibility measures both from a conceptual and a mathematical point of view. For the scope of this work, a distance-based accessibility metric is chosen to reflect the discomfort induced by the length of walking. To this end, the following, negative exponential cost function, which is commonly used in location-based accessibility assessments [39], is employed: where A is the accessibility, c ij is the cost of traveling between an origin i and a destination j and β sens is the cost sensitivity. To transfer this measure to the accessibility of a charger, we define the travel cost c ij as the walking distance between the charger and the trip destination d walk . In order to arrive at a scoring component in the form of S walk = β walk S walk with a negative β walk to indicate disutility, we convert the utility formulation for the walking component to the following form: Equation (8) employs the accessibility term from Equation (7) and substitutes the cost sensitivity of walking β sens for the term −ln(A r )/d walk,max . With this substitution, a residual accessibility A r is assumed for the maximum accepted walking distance d walk,max . Depending on the choice of A r , different slopes of the score function can be obtained that model different user attitudes to walking. Figure 4 shows the resulting accessibility measures and scores.  Since there is no quantity associated with home charging, agents are rewarded if they charge using their private chargers at home and otherwise are not given any reward or punishment (Equation (9)).
S homeCharging,a = β homeCharging i f the agent charges at home 0 else After scoring, replanning is the next component in the charging behavior model in the UrbanEv-Contrib. Pseudocodes for the replanning module are given in Figure 5. First, the agent population is divided into two sub-groups: a critical group, containing DVU whose batteries ran empty given their current charging plans and a non-critical group, whose executed charging plans are feasible but not necessarily optimal. All charging plans in the critical group are replanned, whereas only an adjustable proportion of plans p replan,nonCrit in the non-critical group is scheduled for replanning. All other non-critical DVU select the best plan in their plan memory for the next simulation iteration. Since the score of charging plans is subject to other agents' charging decisions, the best charging plan in an agent's plan memory may alter without any alterations being made to the plan itself, possibly inducing a change in the order of memorized charging plans. Therefore, the replanning selection in Figure 5a promotes the population-wide optimization of charging plans by exploration of the solution space depending on the p replan,nonCrit parameter, targeted replanning of non-feasible plans, and an evolutionary survival-of-the-fittest mechanism performed by selecting the best performing plans for future iterations. The replanning procedure for the replan group is given in Figure 5b: first, a random number of changes to the charging plan is determined. Each plan experiences between one and N maxChanges simultaneous changes. Each change can either be an adjustment of the arrival time at an activity, the removal of an existing charging activity, the addition of a new charging activity, or the shifting of a charging activity to another primary activity. Alterations of the arrival time are introduced into plans that contain a failed charging attempt owing to chargers being fully occupied with a possibility p timeAdjustment . The maximum arrival time flexibility is set to t flexibility . Changes in the arrival time enable agents to arrive at a contested charger earlier than other drivers, to obtain a charging spot that would otherwise be occupied already. Additions, movements or removals of charging activities from charging plans are introduced based on the number of charging and non-charging activities on the charging plan. Charging plans with a small proportion of charging activities in relation to non-charging activities have a higher probability of adding a new charging activity. The inverse logic applies for charging plans with a disproportionally large amount of charging activities. The shifting of existing charging activities from one primary activity to another is another mechanism for exploring the solution space and possibly discovering charging patterns that are more efficient than existing ones.

Location Choice
Whenever an agent's charging plan requires a recharging activity, a suitable charging location has to be determined. Possible options are shown as chargers available in the vicinity of the agent's primary activity. Thus, the set of alternatives comprises public chargers, work chargers, and any available home charger. UrbanEv agents select a charging location at the time of their arrival at the primary activity. The location choice process implemented considers airline distance, occupancy, and technical suitability. Figure 6 illustrates the corresponding filtering process. Occupied chargers and chargers with incompatible plug types are not included in the set of simulated chargers. Private and work chargers that do not belong to the agent in question are also not considered. The remaining chargers are further reduced to those that are within the maximum acceptable walking distance d walk,max , and the closest of these is then chosen for the scheduled recharging activity. Since the current implementation uses airline distances to measure walking distances to improve the simulation performance, the reachable area is given by a perfect circle. This distance metric can easily be substituted by a network-based distance, because the MATSim framework already offers a routing module. Depending on the underlying road network and city characteristics, this may help to avoid a possible underestimation of walking distances especially for large values of d walk,max . The right hand side of the figure provides an indication of the walk score of the selected charging station. If no charger can be found because the remaining charger set is empty, the charging activity is skipped and marked as failed in the agent's charging plan. The implemented location choice assumes that users are aware of the occupancy of candidate stations, which is a reasonable assumption in today's charging infrastructures, which include smartphone apps and navigation software to inform the users.

Activity Location Street
Usability

Munich Case Study
To assess the usability and accuracy of our modeling approach, we conducted a simulated case study for the city of Munich. This section presents an overview of the simulated population and environment and the choice of simulation parameters in the case study. Section 5 then goes on to present the results of the simulation and compares them with usage statistics from existing charging infrastructures installed in urban areas.

Environment
The case study environment comprises the simulated population, their mobility plans and vehicles, the street network, and the charging infrastructure. By early 2020, 5695 BEV and 5588 plug-in hybrid electric vehicles were registered in the city of Munich [40]. Thus, the case study is set-up to represent a scenario in which there are 10,000 electrified DVU on the streets of Munich. Each simulated agent has a BEV. Four different vehicle types are selected to represent the different vehicle classes available on the BEV market: a small car (Renault Zoe), a compact car (Nissan Leaf), a large size model (Tesla Model 3) and a sport utility vehicle (Audi e-tron). Table 2 lists the vehicle types along with the properties that are relevant to the simulation. The vehicles' maximum charging rates are approximated from the charging times indicated by the respective manufacturer. Vehicles are assigned at random and with equal probability, resulting in an almost uniform number of agents per vehicle type. The positions of public chargers and the street network were extracted from Open Street Map data, which was retrieved from geofabrik [41]. The power of public chargers is set to 22 kW which is in line with the power predominantly available at public charging points in the city of Munich. Furthermore, each public charging station provides two connectors, which is equivalent to their real-life counterparts. Private and work chargers are assigned to the agent's homes and work places at random, with a probability of 80% for home chargers and 20% for work chargers. Because the number of private and business-owned charging stations is not monitored by a central agency in Germany, these ratios are estimations based on the observed strong self-selection among BEV drivers, which depends on their ability to access private charging points at work or at home [42][43][44]. Table 3 summarizes the simulated charging infrastructure. The spatial distribution of the simulated public, home, and work charging points are shown in Figures A1-A3 in the Appendix A. A strong focus of public charging points on the inner city can be observed. Since home chargers are assigned at random, their spatial distribution is proportional to the spatial distribution of the simulated user homes. Analogously the position of work chargers correlates to business areas.  To simulate the population as described, a mobility plan is assigned to each virtual agent. Correct modeling of these plans is paramount to ensure the accuracy of an agent-based electromobility simulation. Because of the complexity of mobility demand generation, the use case scenario is based on existing mobility plans obtained from the Professorship of Modeling Spatial Mobility at the Technical University of Munich. These plans stem from a population and workplace model called SILO [49,50], which is based on land use, household, and transportation data in the target region. Agents' mobility plans are generated by the Microscopic Transport Orchestrator (MiTO) [51,52] based on the SILO population and workplaces. Both MiTO and SILO were developed at the Technical University of Munich. While the mobility plan generation is validated [53] and thus provides a reliable basis for further simulations, MiTO mobility plans are trip-based and do not contain activity sequences that are essential for consistent tracking of DVU and their SOC. We therefore select round-trips from the original set of mobility plans. We augment these by adding a home activity to the end of trips starting with a home activity and ending with a different activity. This ensures, closed activity sequences for each mobility plan, which enables the generation of multi-day mobility plans derived from single day plans by repeating daily activity sequences of each agent. The resulting set of mobility plans mostly comprises home-work commutes and a few trips of other activity types. The consequences of this design decision will be discussed in Sections 5 and 6. Table 4 lists the configuration parameters of the simulation. A simulation period of 10 days was chosen, to account for multi-day charging activities and decision-making. The size of the agents' plan memory is set to five to provide some resilience against chaotic plan changes by others. The maximum walking distance is set to 500 m, because the vast majority of drivers do not favor walking distances longer than 500 m for frequent activities like grocery shopping, work, and social activities [54]. Agents' range anxiety is parametrized to start at 20%, on the basis of studies showing that a majority of users starts to feel uncomfortable at a 25% SOC or less [42]. 20% was chosen instead of 25%, to account for increased median battery sizes. Replanning is configured to occur for all of the critical charging plans and for 30% of the non-critical population to explore the solution space and avoid local optima. The maximum number of simultaneous plan changes within an individual charging plan is set to two, as a compromise between convergence speed and stability. Agents can adjust their arrival times by a maximum of 10 min and the probability of them doing so is 10%. Lastly, score weights are chosen relative to one another: the strongest negative weighting being set to −10 for an empty battery, which is twice as punishing as the feeling of range anxiety and ten times as influential as the reward for home charging, which is set to +1. This design choice was made to ensure that infeasible charging plans are quickly removed from the plan pool. Home charging is set to a small rewarding factor to incentivize agents to charge at their private home chargers and favor them over public chargers and other charging opportunities, while still making it impossible for agents to compensate for range anxiety and empty batteries with frequent home charging. While the punishment for walking is low compared to the influences of range anxiety and empty batteries, a slight negative reward for walking is chosen to optimize agents plans in terms of comfort, once empty batteries and range anxiety have been fully avoided. In order to observe the behavioral training process of the agents, all charging plans are initialized to include only non-charging activities. Each agent thus has to learn to charge, and many agents will run empty in early simulation iterations. Section 5 shows that the implemented charging optimization converges towards a state in which the majority of agents can drive their cars without range anxiety or battery depletion throughout the entire simulation period of 10 days.

Results
This section presents the results from the Munich case study. It assesses the coevolutionary behavioral training process, analyzes the resulting user behavior in terms of both charger occupancy in time and space and SOC over time, and compares it to literature sources and expectations. Figure 7 presents an overview of the training process, with the upper part showing the scores of the agents' charging plans during the simulation iterations. The four lines represent the average values of the agents' executed, best, worst, and average plans. As hypothesized, all scores converge over the course of the training, which indicates both stability and the emergence of a steady-system state that can be analyzed for charger utilization and SOC profiles. Furthermore, it can be seen that the distance between all plan score averages decreases over the iterations, which means that most plans perform similarly well.  During the first five iterations of the simulation, the average of the worst plans is equal to the performance of the initially executed charging plans, because it takes at least six iterations for the agents to memorize at least five charging plans that are better than the initial charging plan. The overall scale of the scoring functions is arbitrary, because it is directly related to the weighting factors chosen for the simulation scenario. Nevertheless, charging plans do not, on average, score higher than zero, indicating that the majority of charging plans are feasible and range anxiety can be avoided. The lower part of the figure depicts the occupancy ratio of home, work, and public chargers at iterations five, ten, twenty, and fifty. The total charger occupancy line clearly shows that agents quickly learn to add charging activities to their charging plans in order to avoid range anxiety and empty batteries, which directly translates to improved plan scores. It can be seen that the occupancy ratio of home and work chargers increases with higher numbers of iterations, while the utilization of public chargers decreases, meaning that agents' first priority is to avoid the high punishments for range anxiety and empty batteries rather than choosing charging facilities that enable optimum walking distances and parking times. During later stages of the training, a smaller graph slope coincides with a successive redistribution of charging activities from public chargers to more convenient home and work chargers. At the end of the simulation, roughly 40% of home chargers are occupied during the night and 10% during daytime, while work and public chargers are used mostly during work hours, with an occupancy ratio of 15% for work chargers and about 30% for public chargers. Since most agents with a home charging opportunity exclusively charge at home and over night, a nightly occupancy of 40% translates to a charging frequency of 2.8 charging activities per week, which falls in the range of other sources which find between 2.8 and 3.1 charging activities per week [35]. Figure 8 sets these numbers in relation to observations from real-life charging infrastructures. The upper graph compares the occupancy of all chargers in the Munich case study with the occupancy rates of a Dutch urban public charging infrastructure [55] over the course of a typical day. Due to the large share of home chargers in the simulated scenario, the total occupancy rate in the case study is dominated by home charging activities. Since Dutch public charging infrastructures are known to be used for close to home charging activities more than in other countries [56], a large similarity of the graphs can be expected if the simulated agents show a realistic home charging behavior. The graphs show that the occupancy ratios are qualitatively and quantitatively similar. The absolute difference between occupancy ratios is of a magnitude of low, single-digit percentage figures during nighttime. A maximum deviation of less than 10% can be observed during daytime. Both graphs display transitional phases between high and low occupancy ratios at the start and end of typical workdays (6 a.m. and 4 p.m.). The bottom part of the figure compares German public charge point data taken from urban and industrial areas [57] with the simulated occupancy of public chargers from the Munich case study. Since both occupancy ratios are scaled differently owing to differences in charge point surpluses in the study area, direct comparison of the raw data is insignificant. Therefore, occupancy ratios were normalized, putting all graphs on a scale of 0 to 1. Prior to normalization, value ranges for the Munich case study are 14-37%, 10-16% for industrial areas in the reference infrastructure, and 15-23% for urban areas in the reference infrastructure. There are strong qualitative similarities between the three data sets. The level of conformity of the simulated chargers with charging points observed in industrial areas is especially significant. This indicates that the majority of the simulated charging transactions at public chargers are work-related. The urban reference, however, exhibits a second occupancy peak during after-work hours at around 8 p.m. This peak is not reflected in the simulated infrastructure, which can be explained by the lack of after-work activities in the mobility plans. Another striking difference between simulated occupancy ratios and the observed behavior is the temporal misalignment of the occupancy peak and the transition phases in the mornings and evenings. The simulated occupancy runs approximately one hour ahead of observed charger use. Without any more information relating to the data presented in [57], it can only be assumed that there is a temporal difference due to the observed population or study area. It is striking, however, that the data presented in [57] also lags behind the data taken from [58].   Figure 9 adds a spatial dimension to the public charging behavior of the simulated agents, by averaging charger occupancy and charging utilization in the districts of the city of Munich. The left-hand side of the figure maps the occupancy ratio for three significant time steps. As mentioned above, the simulated public charging infrastructure mainly serves the purpose of work-related recharging. For this reason, the occupancy ratio at night (3 a.m.) is generally below 30%. At this time, a low occupancy can be observed in the city center because of a large number of unused opportunity chargers in the downtown areas. As there are only two public chargers in the northernmost district of Munich, the occupancy for that district can be regarded as an outlier. As expected, charge point occupancy increases dramatically during working hours and reaches its peak at around 9 a.m. The highest occupancy ratios of up to 50% simultaneously occupied charging points occur in areas with a large number of workplaces, such as the city center and the commercial zones in the city's outskirts. At 7 p.m., after typical working hours, the spatial occupancy already resembles the distribution at night. The right hand side of the figure contrasts actual charger utilization with the state of occupancy, i.e., it shows the ratio of chargers that are currently not only occupied but also used for charging. Late at night, utilization of public chargers is virtually non-existent, as the vast majority of residential charging activities have already concluded by 3 a.m. In contrast to the low night-time utilization, many cars are being charged at 9 a.m. because most drivers have arrived at their workplaces by then. Another interesting point in charging utilization is reached shortly after the majority of drivers return home at around 7 p.m., when another, smaller peak in charging utilization occurs, this time mainly in residential districts. The pattern of charging utilization points towards a highly volatile electricity demand in both temporal and spatial terms. The concentration of plug-in hours shows that high demand peaks can be expected in the mornings in particular, when people arrive at their workplaces. In addition to charger occupancy and utilization, developments and distributions in relation to SOC convey important information about charging behavior as a result of co-evolutionary optimization. Figure 10 shows the cumulative share of SOC values at which agents plug in their car for charging, with, for instance, more than 80% of charging activities starting with an SOC of 30% or more. Data from three different studies is compared, whereby it can be seen that Franke and Krems [35], who conducted a field study of charging behavior in 2013, found that charging events start with much lower SOC values than those in the Munich case study simulation. A more recent analysis of real-world BEV driver behavior published by Weldon et al. [58] in 2016 shows that three years after the study by Franke and Krems, drivers commenced charging at significantly higher SOC values, lessening the gap between the values in the literature and those outputted by the simulation. This may be due to the increase in average battery sizes, which leads to smaller SOC drops before recharging, assuming that the average charging frequency, vehicle utilization, and distance-specific consumption are stable. The fact that the Munich case study environment comprises the latest generation of vehicles, with an average battery capacity of 49 kWh, explains the differences in start SOC distributions. Further evidence of this is provided by the bottom part of the figure, which shows that the absolute deviations between the most recent observation study from 2016, its predecessor, and our simulation are equally large. Nevertheless, future simulations with more detailed modeling of current vehicle stock may reveal a behavior closer to [58] owing to the decrease in average battery sizes compared to those in this simulation.  Another insightful distribution of SOC is provided by Figure 11, which presents the number of agents whose vehicle SOC are in different SOC groups over the course of the simulation. The first 48 to 72 h of the simulation period are greatly influenced by the initial SOC of agent vehicles. Nevertheless, it is evident that most agents always have an SOC of 60% or more, which effectively precludes range anxiety. Only a small proportion of agents fail to avoid range anxiety and a few stranded vehicles occur during the simulation (black proportion in the figure). While it is not realistic behavior for agents to run their batteries down entirely, the occurrence of a few stranded cars is almost inevitable if home and work chargers are allocated entirely at random. In realistic scenarios, users who cannot charge their car at home or work will most probably not acquire a BEV in the first place, unless they can reasonably expect to be able to charge their cars at public chargers located close to their regular venues. Since this constraint has not been modeled, it is reasonable to assume the existence of stranded cars. Although there are few agents in low SOC groups, the overall SOC values are stable, periodic on a daily scale, and generally high. It should be noted that less than 20% of the population fall below 50% SOC at any given time, although only 80% of the agents are equipped with home chargers. This means that public charging and work charging can successfully compensate for a lack of home chargers, as long as the overall number of public chargers is high compared to the number of agents without home charging facilities, resulting in low charge point rivalry. In the simulated scenario with a ratio of approximately 0.07 public chargers per BEV, 0.39 public chargers per BEV without home charging facility, and additional work chargers for 20% of the population, the level of charge point rivalry can be considered low. In summary, the simulated use case-while realistic-clearly points to an oversupply of (public) charging infrastructure due to the high ratio of drivers today who have the opportunity to charge at home. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Discussion and Conclusions
The simulation framework presented in this article produced realistic results in the case study for the city of Munich. Comparisons with infrastructure analyses from real-world observations exhibit qualitative and quantitative similarities. Furthermore, co-evolutionary behavioral modeling opens up new possibilities and offers greater flexibility regarding influencing factors than other contributions on the subject. The simulation framework and behavior model presented here, are suitable for performing realistic analyses of urban electromobility and can also be used for change analysis, scenario analysis, and general planning activities relating to policy and infrastructure design on a city-level. Used in combination with a static demand estimation, UrbanEV-Contrib can serve as a sand box validating charging infrastructure designs in a dynamic simulation environment [16]. Despite the advantages and novel features of the approach, some aspects of the simulation require further attention before a large-scale application of the simulation framework can be considered viable.
In terms of modeling, some additional features need to be implemented to enable a holistic representation of real-world electromobility. First, commuters from surrounding areas have to be factored in, along with their energy demands. The addition of commuters constitutes a boundary value problem because the study area can not be increased at will. Data availability and modeling complexity limit the size and accuracy of the model. Hence, commuters' energy demands cannot be simulated and, while the number of commuters can usually be determined for a city, reasonable assumptions regarding commuters' charging activities outside the observation area have to be introduced. Second, commercial vehicle fleets may also use public charging infrastructures, increasing charge point rivalry and charger utilization. Unfortunately, commercial vehicle fleets are diverse and hard to model within an electromobility simulation. While some commercial vehicles, like taxis fleets may use separate, dedicated charging infrastructures, others, like carsharing fleets, are known to cause high charger utilization in public charging infrastructures [9]. Third, the price of charging should be added to the score functions because of it being an important factor influencing the decision to charge. This requires a city-wide database of charging prices. With the integration of cost into the behavioral model, design decisions regarding the relative weightings of walking distance, and price need to be parametrized according to real-world user behavior. In contrast to the weighting of range anxiety, battery depletion, walking discomfort, and home charging, which we determined by prioritization, price and walking sensitivity, are strictly quantitative. Fourth, Coffee&charge activities should be integrated into the framework with a special focus on fast-charging infrastructures, especially when study areas offer a large quantity of fast-chargers. When integrating fastchargers, a simultaneous addition of charging prices to the behavioral model is necessary. Without a consideration of prices, an unrealistic proportion of agents will choose fastchargers, because without any cost-based punishment, fast-chargers appear to be more efficient in any case. Lastly, studies have shown that BEV adaptation and user behavior are dependent on socio-demographic factors such as age, education, income and dwelling types [44]. In future applications, vehicle and charger allocations should thus be modeled in greater detail based on the respective socio-demographic city structure than was possible in the present study. Furthermore, to fully validate modeling and simulation concepts, the framework should be applied to a city in which detailed data on charger utilization is publicly available. With such data at hand, decision-makers and city-planners can model the status-quo of an urban electromobility system and then derive scenario analyses after validating the status-quo scenario.
Apart from the aforementioned aspects of modeling, the presented simulation framework may serve as a powerful tool for city planners to design or extend urban charging infrastructures. They can use the framework to analyze the charging demand at different candidate sites, forecast occupancy times and impacts on the power grid and evaluate whether or not a planned charging infrastructure supplies enough charging opportunities to avoid that users who are dependent on public charging reach low SOC values or experience range anxiety. Nevertheless, the necessary modeling steps require a great amount of data to be available. Especially the availability of mobility plans poses a challenge when applying the simulation framework to worldwide study areas. While origin/destination matrices are usually available, activity-based mobility sequences are not. Adding to this challenge, activity sequences are not only needed on a single-day basis, but for work days, weekends, and holidays alike. The mobility plans used in the present case study generally do not include a realistic number of after-work activities and are representative of working days only. Hence, the transferability of the simulation results is limited to weekdays. A future comparison of results based on varying accuracies in mobility plan modelings may shed light on the depth of modeling needed for electromobility analysis.
Other than mobility plans, the accuracy of all results is largely dependent on an accurate modeling of the city's characteristics. The presented assumptions and requirements from Section 3.1 may be more or less valid depending on the study region, since international electromobility environments have shown to be diverse regarding many influential circumstances like housing types, charging speeds, user behavior, socio-demographics, and market maturity [56]. This is why for each target environment, city planners have to model score weightings, vehicle types, walking preferences, available charging powers, and charger locations individually. Thus, using the proposed framework is an expert's task requiring a lot of domain specific knowledge, data collection, data pre-processing, data analysis, data visualization, and programming. A graphical user interface and further research aimed at the derivation of best practices and guidelines can alleviate the challenges potential users are faced with when applying the proposed framework to increase its potential impact and foster practical applicability. modeling of simulation agents. We are equally grateful for the continuous support and exchange of ideas provided by Nico Kühnel (N.K.) from the Professorship of Modeling Spatial Mobility at the Technical University of Munich. N.K. provided us with valuable insights into mobility demand generation, artificial mobility plans, MiTO, and SILO. Last but not least, N.K. supplied us with a dataset of mobility plans for use in the simulation conducted in the case study. The author L.A. would like to thank his colleagues of the in-tech GmbH, especially Albert Habermann who supported the project with his great experience regarding practical considerations of urban charging infrastructures and user behavior.

Conflicts of Interest:
The authors declare no conflict of interest.

Abbreviations
The following abbreviations are used in this manuscript: