Measurement of Regional Electric Vehicle Adoption Using Multiagent Deep Reinforcement Learning

Abstract

This study explores the socioeconomic disparities observed in the early adoption of Electric Vehicles (EVs) in the United States. A multiagent deep reinforcement learning-based policy simulator was developed to address the disparities. The model, tested using data from Austin, Texas, indicates that neighborhoods with higher incomes and a predominantly White demographic are leading in EV adoption. To help low-income communities keep pace, we introduced tiered subsidies and incrementally increased their amounts. In our environment, with the reward and policy design implemented, the adoption gap began to narrow when the incentive was equivalent to an increase in promotion from 20% to 30%. Our study’s framework provides a new means for testing policy scenarios to promote equitable EV adoption. We encourage future studies to extend our foundational study by adding specifications.

1. Introduction

The adoption of Electric Vehicles (EVs) is experiencing rapid growth, propelled by governmental policies that position them as the future of transportation. According to the U.S. Department of Energy [1], there was a substantial increase in EV sales from 2020 to 2021, when overall sales of light-duty vehicles also rose. On a global scale, EV sales saw a 55% and 60% increase in 2021 and 2022, respectively, compared to the preceding year’s sales [2]. Major investment firms, including Morgan Stanley [2], project that by 2050, EVs will constitute 90% of global auto sales.

The promotion and growth of EVs primarily stem from their environmental benefits and cost efficiency. Transportation is one of the largest contributors to Greenhouse Gas (GHG) emissions [3], which disturb the Earth’s energy balance and significantly increase the risk of global warming and climate change. Battery Electric Vehicles (BEVs), which are fully electrified, produce zero tailpipe emissions. Hence, replacing conventional Internal Combustion Engine Vehicles (ICEVs) with EVs is viewed as an effective strategy to reduce GHG emissions, aligning to achieve net zero emissions. EVs can convert two to six times more energy from the grid to power at the wheels than gas-fueled vehicles (Li et al., 2012). This efficiency is further enhanced when the electricity used to charge EVs is derived solely from sustainable renewable energy sources [4,5]. Factors contributing to the rise in EV sales include decreased battery prices, increased charging options, a broader range of EV models, growing consumer trust [6], and heightened environmental awareness among consumers [7,8].

However, early-stage analyses of transportation equity in EV adoption have revealed disparities across different races and income levels. In particular, deploying and accessing EV chargers present persistent inequalities [9,10,11]. Hsu and Fingerman [11] analyzed public EV charger access across racial and income groups in California, the leading state in the U.S. for the EV market. Their findings indicated that communities of color, specifically African-American and Hispanic majority communities, had less access to EV chargers compared to predominantly White communities. This disparity is even more pronounced in low-income communities within the communities of color.

Consequently, EVs are still considered luxury vehicles, with their primary owners being high-income, highly educated [10,12], and predominantly male homeowners [10]. A survey by Farkas et al. [12] of 1257 EV owners found that they are generally older and socioeconomically more stable than non-EV buyers. Incentives for EVs often need an equity discourse [10,13]. For instance, when census tracts are grouped by income, the top 12.5% receive 25% of the total rebate amount, while the bottom 75% only receive 38% [13]. Moreover, residents living in and owning Single-family (SF) housing units have a greater capacity to install personal chargers than those living in Multi-family (MF) housing units [14]. Given that the ability to purchase an EV involves not only the upfront cost but also the associated costs of maintaining the vehicle [14], ensuring equitable distribution of incentives for EV purchase and access to the related infrastructure is critical.

The promotion of EVs fundamentally hinges on policy implications. Researchers have primarily analyzed subsidy policies and their impacts on the EV market, informed by governmental incentives and tax rebate programs. Subsidies can influence the market segmentation of automobile vehicles, but their effects vary between high- and low-income individuals [15]. High-income individuals are more likely to purchase high-quality EVs, while low-income individuals opt for lower-tier EVs [15]. This outcome seems plausible, as price-sensitive consumers are 1.6 times more susceptible to subsidy policies than price-insensitive consumers. However, without subsidies, the overall preference for purchasing EVs could decrease by 35% [16]. Generally, each $1000 subsidy has a 12–15% effect [17]. Other subsidy types, such as industrial subsidies [16] or subsidies for deploying chargers, also improve EV sales [18]. However, recent studies have begun to challenge the general assumption regarding subsidy benefits for EV consumers [13,19]. For instance, Nunes et al. [19] recently argued that current policies structurally fail to achieve emission savings because disadvantaged households often undertake long-distance vehicle travel and are not adequately ‘rewarded’ by economic savings.

Currently, most states in the U.S. favor a uniform distribution of subsidies. However, a few have initiated tiered subsidy programs to offer price buy-down opportunities to low-income communities. For instance, California’s Clean Vehicle Assistance (CVA) program provides up to $7500 in subsidies alongside affordable financing opportunities with interest rates below 8% [20]. A city like Riverside offers rebates ranging from $1000 to $2500 for used BEVs or Plug-in Hybrid Electric Vehicles (PHEVs) purchased or leased after 1 January 2023 [21]. The necessity to financially assist disadvantaged communities is evident, given California accounts for 40% of U.S. state EV sales [22]. However, consumers with low or moderate incomes make up only 0.6% of new PHEVs and BEV buyers and 1.8% of new hybrid electric vehicle (HEV) buyers [22]. It remains uncertain how different levels of financing would affect EV adoption among low or moderate-income households, even after conducting policy simulations [22]. While the upfront financial benefits of EV ownership are apparent [23] and the adoption of tiered incentive programs is being discussed, it is still unclear how to establish the appropriate parameters.

This research presents a framework based on a reinforcement learning approach to simulate a regional EV adoption scenario. The primary goal is to gauge the impacts of offering additional incentives to low-income communities. Low-income residents often live further from their workplaces and have longer commutes [24]. Transitioning them to driving EVs makes achieving the targeted net zero emission goals easier. The importance of addressing their reluctance to adopt EVs should be recognized. Our study incorporates the peer effect among consumers [24,25] and the interaction effect of climate change on EV purchasing decisions [26], using multiagent systems that represent high- and low-income groups.

Our case study focused on Austin, Texas (TX), where a uniform subsidy distribution is implemented. Like many U.S. cities, Austin has experienced significant racial divisions stemming from interstate highway programs in the 1950s and 60s. Despite these challenges, as of 9 May 2023, Austin boasts the highest percentage of registered EVs in TX, at 1.95% of total registered vehicles [27]. The City of Austin (COA) has set the ambitious goal of replacing 40% of all vehicles in Austin with electric counterparts by 2030, and it has been actively promoting this transition [28]. Additionally, Tesla’s engineering headquarters is located in Austin. Beyond the focal state leading the EV adoption, another case study that illustrates the early stages of EV adoption can help us devise a better transportation electrification plan. The insights from Austin’s experience could be applicable and beneficial to other cities.

Hence, EV equity studies show a possible bias towards the West Coast, as observed by Hopkins et al. [29], who reviewed seven North American literature pieces on this subject. Five of these studies were focused on the West Coast, including California, Seattle, and Los Angeles, while only two originated from the East, specifically Chicago and NYC. States like Florida and Texas, which follow California as leaders in EV adoption in North America [30], are overlooked. Furthermore, the equity literature rarely includes studies that ‘explicitly’ mention equity or justice with normative frameworks [31]. Among 37 peer-reviewed articles in mainstream journals, a mere 29.7% directly address equity in EVs. A case study in Austin could further enrich the discourse on disparities in ongoing EV adoption by broadening the scope of the investigation.

The remainder of this paper is organized as follows: First, we present a literature review on the current subsidy program, its impact on EV adoption, and other factors related to EV adoption. Next, we describe the environment designed for running our reinforcement learning simulation. In this phase of our study, we manually classified EVs from various manufacturers into either PHEVs or BEVs before incorporating the count of each type of EV into the simulation environment. We utilized the Geographic Information System (GIS) to interpolate the data spatially and facilitate the sampling process. After optimizing the simulator with fixed hyperparameters, we compared the mode choice and rewards between high- and low-income agents. This process was repeated multiple times, each time adjusting the impact of the tiered subsidy on the low-income agents. We then discuss the results and end the paper with a conclusion summarizing the findings of our study.

2. Related Work

2.1. Current EV Inccentive and Policy Program

Incentives for customers to purchase EVs come in the form of either federal or state-specific programs. Federal funding for EV subsidies was first allocated during the Obama Administration and has been heavily promoted under the Biden Administration. Initial incentives aimed at promoting the purchase of Plug-in Electric Vehicles (PEVs) were introduced in the American Clean Energy and Security Act of 2009. However, these were diversified a year later as PHEVs and BEVs entered the market [32].

The tax credit for new purchases could be as high as $7500 [32]. As of now, the amount of tax credit remains the same. According to the U.S. Department of Energy [33], all EVs, whether PHEVs or BEVs, are eligible for a tax credit of up to $7500 if purchased in or after 2023. Pre-owned vehicles purchased in or after 2023 also qualify for a tax credit of up to $4000, provided the purchase price is less than $25,000. These credits may be increased through regional policy portfolio programs. The federal tax credit has significantly impacted consumers’ decisions to switch from ICEVs to EVs [34]. However, the effect is not uniformly distributed. Interestingly, the incentive might not significantly influence the purchasing decision of buyers interested in luxury EVs [34].

State-specific programs vary in incentives, ranging between $2500 and $7500. While the premise of these programs is generally robust, specific criteria must be met for applicants to receive the total amount. Nunes et al. [19] pointed out that current incentive programs are solely targeted at first adopters and need to take into account the existence of second-hand EV owners. Factors such as income levels and household characteristics, including family type and number of children, can pose potential barriers preventing applicants from receiving the maximum incentive [35]. A study by Liu et al. [35] performed an equity analysis on state-level Income Tax Credit (ITC) programs. They found that high-income households with fewer children were typically more likely to receive the total amount. However, this trend can vary based on the state’s tax treatments, tax rate structures, and thresholds. Overall, incentives have a significant impact on increasing EV sales [17,36]. Jenn et al. [36] examined both the impacts of monetary and non-monetary incentives on EV sales across all states in the U.S., compiling data on 198 different incentives. In addition to ITCs, other factors such as fleet credits, High-occupancy Vehicle (HOV) lane access, inspection exemptions, registration fee reductions, Time of Use (TOU) rates, and charging subsidies also contribute to the increase in EV sales.

Most evaluations of EV incentives in the U.S. focus primarily on the state level [36,37] or leading states [20,22]. Subsidy and rebate programs are already in place, and there is a need for them. However, measuring their actual effect on adoption is often challenging [38]. While new guidelines limiting the $7500 tax credit under the Inflation Reduction Act have been announced, it will take years to finalize, and the outcome remains uncertain [39]. Meanwhile, policies ought to take into account the distinct characteristics of each city, underscoring localized strategies and solutions. We need a more diverse set of case studies to increase the probability of substantial EV adoption and overcome potential barriers [40] By simplifying real-world environments using reinforcement learning, our presented research framework will offer a nuanced understanding of EV adoption, enabling more effective incentive implementation.

2.2. Factors Associated with EV Adoption

EV adoption involves complex factors. Electricity prices, range anxiety, brand utility of vehicles, consumer awareness, access to public electric vehicle charging stations (EVCSs), sociodemographic factors, and the built environment are found to affect EV adoption. Bushnell et al. [41] compared the asymmetry of pricing information between electricity and gasoline prices, identifying a correlation between the pricing of electricity and gasoline and EV purchases. Tesla holds dominant brand utility [42], influencing EV adopters’ awareness of the costs and benefits of purchasing EVs [43]. Environmental awareness about reducing climate mitigation costs plays a pivotal role; however, it alone is not deterministic [44]. Rauh et al. [45] conducted an experimental field survey among experienced EV drivers and those with no experience, concluding that range anxiety affects the purchase of BEVs.

Neubauer and Wood [46] argue that public EVCSs contribute to mitigating range anxiety issues. Ensuring fair access to EVCSs, especially for those unable to install personal chargers, is critical. The cost of purchasing an EV extends beyond the capital cost and includes associated costs such as obtaining permits, installing chargers, and calling technicians for assessments [14]. Residents of MF housing units face challenges in charging EVs unless their landlords provide facilities in their garages, while residents of SF homes have greater capabilities to install chargers in their garages [14].

Nevertheless, the current deployment of EVCSs has recently been criticized in transportation and charging equity studies, based on distributional philosophies [31]. There is still a lack of studies explicitly mentioning equity or justice [31]. Currently, disparities in charger access between races and income levels are being recognized [11,47,48]. Assumptions about the possible cause of the growing disparities are twofold. First, the private–public partnership nature of the EV market means that public chargers, operated by private vendors, may prioritize benefit over cost. Suitability analyses for locating EVCSs heavily rely on proxies for demand, such as the population density, traffic volume, and parking demand [49,50]. Therefore, the optimization of charger installation functionally fails to consider the equity layer. Second, structural issues with rebate programs suggest that, despite purchasing EVs, low-income buyers might not fully benefit from these programs due to their income levels not meeting the threshold required to claim tax rebates. Guo and Kontou [13] highlighted that rebates were favorably given to affluent neighborhoods. ITC programs inherently give fewer credits to low-income families, especially those with more children [35].

3. Materials and Methods

3.1. Study Area and Current EV Adoption

Our study focuses on Austin, TX, including three counties: Hays, Travis, and Williamson. The geographic unit of analysis used to establish the state in the reinforcement learning environment is based on census block groups. Figure 1 presents the sociodemographic characteristics in Austin, derived from the American Community Survey (ACS) 5-year estimates (2015–2020). The I-35 highway visibly divides the city. West Austin is primarily populated by wealthier, White residents, while low-income communities and communities of color, including Non-White Hispanic and African-American populations, mainly reside in East Austin.

We acquired EV registration data from the Dallas–Fort Worth Clean Cities [27]. The dataset provides the count of registered EVs from various manufacturers, categorized by different Texas zip codes and updated monthly. As of April 2023, it has collected records from 141 distinct companies. We organized this data into two primary categories, PHEV and BEV, utilizing twenty months of records from September 2021. The organized data were spatially interpolated into census block groups based on population density. Figure 2 visualizes the spatial distribution of EV registration records as of April 2023, supplemented by a histogram. The “total count” signifies the combined sum of PHEV and BEV registrations. Notably, all distributions lean towards the left. Downtown Austin in Travis County and West Austin generally exhibit higher numbers of registered EVs than East Austin. This suggests that wealthier, predominantly White communities may own a more significant proportion of EVs.

Figure 3 demonstrates that the growth rate of BEVs in Austin has outpaced that of PHEVs. The average monthly increase in registration records is approximately 3.6%. PHEVs show a monthly growth rate of 2.3%, while BEVs display a higher rate of 4.1%. By April 2023, Austin had 183,076 registered EVs, accounting for nearly 7.7% of the total vehicles. Should the current average growth rate persist, it is forecasted that the COA will accomplish its goal of electrifying 40% of its vehicles by September 2026. This timeframe is four years earlier than the original target, indicating a faster adoption rate of EVs than initially expected.

3.2. Reinforcement Learning

Our study applied reinforcement learning to approximate the parameters of providing tiered incentives to low-income communities. Reinforcement learning offers unique features, particularly well suited for simulating real environments with simplicity [51], as their performance depends on characterizing policy and addressing problem solving [52]. After a comprehensive review of EVs and artificial intelligence technology literature, Abdullah et al. [53] concluded that transportation engineers commonly utilize reinforcement learning to develop EV management systems.

However, akin to studies that utilize supervised learning, there are not many that apply reinforcement learning within the planning sector. This is attributable to the unique nature of reinforcement learning. In contrast to using pre-defined modules such as scikit-learn, reinforcement learning necessitates that researchers construct the environment manually. We employ packages from Keras/Tensorflow or Pytorch to run a Deep Q-Network (DQN), a widely preferred reinforcement learning model. Nevertheless, prior to modeling the DQN, designing an environment to simulate reinforcement learning requires a specialized skill set. Therefore, the causal interpretation of reinforcement learning continues to be a subject of debate among computer scientists [54]. Doing something well and ensuring things go well are different concepts, and the goal of a reinforcement learning model is to ensure things go well. We hope that this research will contribute to bridging this knowledge gap.

Our research involved modeling a multi-agent DQN. One agent represented the state of high-income communities, and the other represented the state of low-income communities, both chosen from census block groups. DQN was introduced by Mnih et al. [55] and integrates the Q-learning algorithm with neural networks. Q-learning involves training an agent to associate a specific action with a certain state [56]. The agent is trained to identify the maximum total reward within a constant number of steps. The pairings of state and action are stored in a Q-table, with the total reward referred to as the Q-value. The approximation of this Q-value follows a particular equation and adheres to the Bellman equation, as follows:

$$Q\left(s,a\right)=r\left(s,a\right)+\mathit{\gamma } \underset{a}{\max }Q\left(s^{\prime },a\right)$$

(1)

where s: state; a: action, r: reward when performing state s and action a; γ: discount factor in controlling the value of reward in the future; and s′: next state with the possible highest Q-value.

The process is mathematically represented by a Markov Decision Process (MDP), which operates on the assumption that the future state depends only on the present state, not on the sequence of states that preceded it. Based on the explanation of Abdullah et al. [40], an MDP problem is defined by the tuple (S, A, γ, P, r). Here the set of possible states is S, set of possible actions at s_t at time step t is A, and R is cumulative and discounted reward in the range $\gamma \in$ [0, 1]. Next s_t+1 is calculated based on the transition probability P(s_t+1|s_t, a_t), indicating the likelihood of moving to state s_t+1 after taking action at state s_t for every episode.

A policy refers to the probability distribution over actions ($a\in$ A) for each state ($s\in$ S). If an agent follows policy π, it observes a state at given time step and selects an action based on the policy function π(a|s), with the aim of maximizing the expected reward. An updated reward r_t+1 is provided as feedback at each time step. The agent repeats these steps, striving to maximize its cumulative reward over time.

A solution to achieve maximization of cumulative reward is creating and updating Q-table. Q-table contains rows of potential states and columns of actions. It stores the expected rewards (Q-values) for every state–action pair, guiding the agent on which action to take in a given state for optimal long-term rewards. However, as the state and action spaces grow, manually filling and updating the Q-table becomes impractical, labor intensive, and costly. As a response, Google’s DeepMind developed the DQN [56]. DQN substitutes the manual computation of the Bellman equation with Deep Neural Networks (DNN), using the initial state as the input layer and returning the Q-value as the output layer with a decaying epsilon. Epsilon refers to the hyperparameters that determine whether to train the agent to explore and choose random actions or use the existing memory stored in the experience replay buffer. Epsilon values range from 0 to 1, with higher epsilon values implying a greater probability of the agent exploring random actions. Mnih et al. [55] astonished the world by integrating DNN with Q-learning, enabling AI to play Atari games.

3.3. Environment in Reinforcement Learning

The reinforcement learning environment necessitates defining its environment and components, including (1) the agent, (2) the state, (3) the action, and (4) the reward. Our study utilizes a multi-agent DQN. The two agents in our model represent either high or low-income communities, which were sampled based on the results of a hot spot analysis using median household income. Our sampling approach utilizes GIS to effectively represent spatial features [57,58]. The state considers the areal interpolated PHEV and BEV counts, the number of vehicles, and the estimated total GHG emissions. The latter was calculated based on the state average in TX from the Alternative Fuel Data Center, U.S. Department of Energy [59]. It is estimated that 7635 and 9574 pounds of CO₂ are reduced annually when using PHEVs and BEVs, respectively, as opposed to ICEVs. ICEVs emit 12,594 pounds of CO₂ per vehicle annually [59].

The actions in our model are defined in three stages. The first is promoting PHEVs, the second is promoting BEVs, and the final is maintaining the current mode choice without any promotional activity. Based on the mean growth rate, promoting PHEVs will increase its state by 3%. Similarly, promoting BEVs results in a 5% increase in its state. The reward is based on CO₂ reduction estimates obtained by multiplying the updated constant by CO₂ reduction estimates. The rationale for designing the agents to seek the highest reward involving CO₂ reduction is derived from findings that environmental awareness is a significant factor impacting the purchase of EVs [6,7,8,26]. Discretization was implemented to save on the cost of training. For instance, a reward is divided by 1000.

One adjustment we make is that we assume the low-income agent is influenced by their neighboring high-income peers. EV adoption is impacted by peer influence [23,25], and it is more widely adopted by White and wealthier individuals [10,11,12]. Thus, it is reasonable to hypothesize that low-income communities, who possess fewer EVs, could be influenced by their wealthier counterparts. Utilizing the epsilon decay function, we assumed that the low-income agent has a 10% probability of mirroring the action of the high-income agent. We hypothesize that the neighborhood peer effect in EV adoption exists similar to the diffusion of solar photovoltaics [60,61].

Tiered subsidy scenarios were tested four times. The default scenario is to maintain the current uniform subsidy policy. The others assume the impact of a subsidy intended to boost EV sales among low-income agents by 10% incrementally. The total reward per episode was recorded and compared between agents. The agents’ mode choice in the most optimal episode was also calculated. The agents are identifying the optimal policy to achieve maximum CO₂ reduction in this environment. The research question we pose is as follows: To what extent should we distribute tiered incentives to low-income communities to mitigate the existing disparities in EV adoption? The results would help us measure the magnitude of allocating tiered incentives.

Other hyperparameters include a batch size of 32, a discount factor set at 0.98, and a learning rate of 0.01. Our multi-agent DQN has an input layer of 4, two hidden layers of 32, and an output layer of 3, which returns a set of three actions. Figure 4 illustrates the structure of the neural network utilized. The activation functions in the hidden layers follow a linear activation, while the output layers employ SoftMax activation. This hyperparameter setting is based on optimization to prevent catastrophic forgetting. We trained the model with over 100 episodes, with each episode consisting of 100 steps. Epsilon was designed to decrease linearly by 0.01 as it approached 70 steps. The experience replay buffer records 30 episodes. After that point, the agent would either use its memory or explore its action. Keras was used during the modeling process. The descriptive statistics are listed in

Table 1. Descriptive statistics.

Variable	Unit	Mean	Std	Min	Median	Max
Registered PHEV [27]	count	5	6	0	4	168
Registered BEV [27]	count	20	19	0	15	360
Total Number of Vehicles [62]	count	1577	890	0	1429	6674
Median Household Income [62]	$	93,843	42,579	2499	90,641	250,001

.

3.4. Methods

Figure 5 summarizes the methods used in our study. First, after preprocessing the data, a hot spot analysis was conducted using GIS on median household income to categorize the agent’s state into high- and low-income groups. Once the environment was defined to simulate reinforcement learning, we modeled a multi-agent DQN. The promotion of EVs by the high-income agent was assumed to have a 10% likelihood of influencing the choice of the low-income agent. Again, there are three sets of actions: the agents could either (1) promote PHEVs, (2) promote BEVs, or (3) maintain their current position. After optimizing the hyperparameters to prevent catastrophic forgetting, multiple scenarios were computed. Behind the scenes, the model initiates training from the initial state, which is the default value of the current mode choice. The two agents start exploring and updating their states and actions. The agents continually explore and select actions until they have completed all remaining episodes.

4. Results

4.1. Sampling Result Using Hot Spot Analysis

Figure 6 displays the result of a hot spot analysis on median household income. A hot spot analysis normalizes the observations to Z-scores. Hot spots refer to clusters of high values, while cold spots denote clusters of low values. Consistent with demarcated sociodemographic conditions (see Figure 1), statistically significant hot spots are primarily found in West Austin, while East Austin exhibits clusters of lower values. There are 580 statistically significant hot spots and 544 cold spots at a 90% confidence level. The record of hot spots was used to define the high-income agent, hereafter referred to as Agent H. Cold spots were organized to define the state of the low-income agent, henceforth named Agent L. The set of states for the two agents is derived based on the identified hot and cold spots.

4.2. Reward Plot of Two Agents

The rewards per episode across different policy scenarios are displayed in Figure 7, with a 90% confidence interval. The environment was designed to be fair. The increase in total rewards is not significant until after 30 episodes, when both agents begin to utilize their memory to make better choices in subsequent episodes. Agent H eventually reaches a point where his total reward can be consistently maximized. The standard deviation for Agent L was relatively greater than that of Agent H. In the default setting, there is a noticeable gap in the total reward between Agent H and Agent L. Generally, agent L rarely outperforms agent H. However, distributing and incrementally increasing the tiered subsidy to Agent L does help this agent catch up with its peer. Increasing the effect to 20–30% gives Agent L a better chance of equaling or outperforming its counterpart.

Table 2. Mode choice in best episode.

Scenario	Agent	Best Episode	Promoted PHEV	Promoted BEV
Default	H	56	800	4800
	L	69	1000	3700
Subsidy Effect of 10%	H	66	800	4800
	L	32	1000	4200
Subsidy Effect of 20%	H	67	800	4800
	L	93	1200	3700
Subsidy Effect of 30%	H	63	800	4800
	L	96	1300	3700

summarizes the mode choices made in the best episode of the two agents. Both agents consistently favored BEVs over PHEVs. Increasing the subsidy effect does not guarantee that Agent L will promote more BEVs. Instead, from time to time, it may continue to prioritize the promotion of PHEVs.

5. Discussion and Conclusions

Our study introduces a reinforcement learning framework and employs a multi-agent DQN. Two agents are used, each based on different income levels. With the peer effect of EV adoption and the influence of climate awareness regarding purchasing EVs in mind, our reinforcement learning environment measures regional EV adoption scenarios. We assessed the extent to which tiered incentives should be given to low-income communities to catch up with their peers regarding EV adoption.

Our findings can be summarized as follows: First, EV adoption is increasing rapidly in Austin, TX, with the adoption rate of BEVs approximately twice as high as that of PHEVs. The city’s EV adoption rate is projected to surpass the forecasts made by government agencies sooner than expected. However, EVs are primarily registered in predominantly high-income and White neighborhoods. Currently, communities of color have a relatively lower rate of EV adoption. While Austin currently primarily operates a universal ITC program, there is a need to provide tiered incentives similar to the CVA program. One notes that rebate programs should target not only the first EV adopters but also consider second-hand EV owners [19]. A new program launched by the City of Riverside in California might be replicable.

Second, reinforcement learning provides interpretable simulation results. Our model suggested that low-income communities are likely to stay within the EV adoption rates of high-income groups. Even with the opportunity for low-income communities to mimic the actions of high-income communities, a structural gap exists in the default setting. Indeed, offering tiered incentives increases their likelihood of bridging this gap. If providing a $1000 subsidy equates to a 12–15% impact on EV purchases [17], offering an amount of $1500–2000 might be a feasible way to ensure their adoption rates. This amount aligns with the actual amount given in the City of Riverside. However, tiered incentives should be carefully addressed to prevent moral hazards. It is worth noting that setting the boundaries of recipients and interest rates [20] are significant factors to consider. We should consider other policy initiatives that can be implemented and have effects equivalent to the identified subsidy amount.

Nevertheless, we would like to acknowledge the simplified environment for the two agents. EV adoption is influenced by multiple factors and exhibits complexity, such as electricity price [41], brand utility [42,43], consumer awareness [44], range anxiety [45], public EVCS [11,14,47,48], and the built environment [49,50]. Reinforcement learning is designed to solve complex real-world problems, but perfectly reflecting every detail is practically impossible. Therefore, simplification to a certain level was necessary depending on the aim and scope of the research, which enhances the model’s feasibility for learning. Factors associated with EV adoption are mainly associated with non-linearity [14,44,48], and addressing these factors requires the specification of distinct layers. As we have demonstrated by testing reinforcement learning to address EV adoption disparity issues, future studies could extend by adding specifications focusing on particular aspects. Then, we will be able to improve practices gradually.

Third, offering tiered incentives does not guarantee that the recipient will purchase higher-tiered types of EVs, such as BEVs. Instead, it may result in increasing the sales of PHEVs. This result can be attributed to the reward structure of our designed environment. Our setup encouraged agents to maximize CO₂ emission reductions through the purchase of EVs. Although subsidies were offered, the low-income agent appeared to be still assessing potential actions, leading to decisions that may seem impulsive.

Purchasing an EV involves complexity. Björnsson and Karlsson [63] suggest that purchasing PHEVs/BEVs depends on their efficiency, and often, PHEVs are preferred from a fuel replacement perspective. Their savings depend on charging availability, battery range, and consumer willingness [64]. However, some may prefer purchasing BEVs such as Tesla, due to their strong brand utility [42,43]. The capital costs and additional expenses related to charging and storing EVs complicate the generally positive perception of EV adoption. Perhaps discussions on EV adoption should focus more on what ought to be rather than what it is. Adopting more EVs is still beneficial in the long term; however, if government agencies want to ensure equal adoption rates of high-tiered EVs, they should review the current charging and purchasing availability. Indeed, the cost of purchasing an EV goes beyond the capital cost and includes the capability to absorb the capital cost [14]. Before offering tiered incentives, the implicit cause of the structural gap should be addressed. Implementing a ‘society-centric’ approach to pursue transportation justice and communicating with local grassroots organizations are viable options [65]. This will help gauge the genuine sentiment of communities and hear their voices regarding current and future policy initiatives.

Our study concludes an emerging disparity issue in Austin, aligning with recent transportation equity analyses concerning transportation electrification [9,10,11,12,13]. We demonstrated the application of reinforcement learning to explore tiered subsidy scenarios for disadvantaged communities. It is important to clarify that our research framework, introducing new methodologies, should not be seen as perfect knowledge. Our conclusions represent a fallibilistic truth from a neo-positivist perspective, and as such, should be subject to an ongoing process of refinement from a pragmatist standpoint [66]. Urban planners usually cannot conduct experiments within communities, presenting an inherent vulnerability in gathering evidence-based observations compared to natural scientists. Conducting experiments in an artificial setting emerges as a pragmatic solution to address pressing issues. We hope our new methodological framework will resonate with practitioners, contributing to the development of better theories and practices in the future.

Our designed environment with agents verified the necessity for implementing tiered subsidies. However, subsidies should be approached from a distribution standpoint and a pre/post-monitoring perspective. The pre/post-monitoring allows for tracking recipients’ qualifications and using subsidies to prevent moral hazards.

Overall, the reinforcement learning framework allowed us to simplify the real-world environment effectively. Compared to regression analysis, reinforcement learning stands out for its simplicity. Our initial attempt to apply reinforcement learning to gauge regional EV adoption scenarios within the planning sector will hopefully encourage future studies to incorporate more complex functions and expand upon our work. Exploring policy simulations in cities to develop AI toolkits holds promise and warrants further study. One point to consider is the importance of contextualizing studies rather than proposing institutional solutions. Rather than solely adhering to instrumental rationality from a modernist perspective, research should also lean towards postmodernity, reflecting on the history of planning [67]. Sandercock [68] highlights ‘diversity’ and ‘difference,’ arguing for planning that supports a more plural and diverse society. While AI may be viewed as an instrument of rationality, it is crucial not to ignore its ‘darker side’. The improvement of practice and the development of alternatives to engage communities are tasks we leave to future research.

Nonetheless, it is important to note a limitation in our study: our agents are defined based on income level. Incorporating race as a factor alongside income might yield more nuanced findings. Low-income agents showed higher variance than their counterparts, which may indicate that they are possibly exploring the environment more broadly. It would be more accurate if we measure, then include the peer effect of EV adoption across communities. Therefore, incorporating a specification related to EV adoption should more accurately represent the urban environment. Unobserved factors like personal preference [41,42,43,44,45], vehicle durability [69], and life-cycle costs [70] could significantly influence EV adoption decisions. We hope that future research will be able to tune and create a complex but robust environment for agents to warrant real-world application.