Key Performance Indicators of Charging infrastructure

The Netherlands are one of the frontrunners in stimulating electric mobility in Europe when it comes to the charging infrastructure density and electric vehicle adoption. Municipalities play an instrumental role in the rollout of public charging infrastructure while they have little insight in the relevant key performance indicators of the charging infrastructure as a means to support effective decision making. This paper aims to contribute to providing a more thorough understanding of relevant key performance indicators for public charging infrastructure. An approach is presented that explores result and performance indicators to support policy makers optimizing the roll out of and improvement of the business case for charging infrastructure.


Introduction
In recent years electric mobility has gained a great deal of attention, leading to electric vehicles on the market and development of necessary charging infrastructure. Charging infrastructure is mostly enabled through subsidies by local or national governments to overcome the chicken and egg problem, while the business case for charging stations in this early stage of development is not yet sufficient [1] [2].
In this phase of adoption of electric mobility, municipalities and service providers struggle how to optimize the roll out of further charging points and how to optimize the use of the charging points [3], [4]. This is due to the fact that the stakeholders (such as municipalities, charging point operators, utilities) of the Charging infrastructure have (1) little insight in the detailed performance of the charging infrastructure and (2) limited insight in the possible measures (as well as and their effect) to manage the effectiveness of the charging infrastructure [5].
Where several cities such as Amsterdam, London and Oslo in recent years have developed a more extensive charging infrastructure for electric vehicles the question arises how to evaluate the performance of charging infrastructure [6], [7], [8]. Typically, performance indicators such as amount of kWhs charged, amount of charge sessions and amount of unique users are mentioned as relevant [9]. However to what extent they are exhaustive or whether other indicators may be more instructive remains unclear.
Municipalities have the ambition to stimulate clean air through placing charging infrastructure, but need to do so in a cost effective way to justify their investments within the municipality. Similarly the municipality needs to balance placing charging infrastructure at the expense of parking spaces, and limiting citizen complaints about under-or over-used charging points. Meanwhile the intention of municipalities is to facilitate a positive business case for charging infrastructure, safeguarding that the electric grid can manage added demand and balance sufficient access to charging points without severely increasing parking pressure.
Different stakeholders in the chain of charging infrastructure pose a number of concerns for municipalities that need to be managed to some extent. In this paper it is argued that these stakeholder interests provide powerful direction to important performance indicators that municipalities must manage, where the performance metrics can also be used to identify possible interventions municipalities can take to manage the different interests set by relevant stakeholders.
This paper provides an exploration of most relevant key result indicators (KRI) and key performance indicators (KPI) that policy makers can use to monitor and optimize the effectiveness of public charging infrastructure in view of these stakeholder interests. Next it provides suggestions which interventions policy could take to increase the performance of their public charging infrastructure.

Result and performance indicators
Performance indicators have been a subject of research for many years to study the measurement of performance of companies as well as evaluate possible instruments to increase their performance. According to Parmenter [10] two types of indicators are distinguished: Result Indicators (RI) tell stakeholders how a company has performed from a specific perspective, while Performance Indicators (PI) provide guidance what to do to increase the results. The common used balanced scorecard is an example of how indicators can be categorized, such as financial, innovation, growth. The field of business administration has resulted in a large amount of KPI short lists such as [10], [11].
In literature methodologies on performing KPI-related research can be found [12] [10]. Most methodologies relate to a top-down development of KRIs and KPIs starting with mission and vision of a company leading to either department or scorecard specific indicators. This method is often related to strategy mapping and aims to create alignment of indicators for all stakeholders on all levels of the organization [12].
Literature on performance measurement of local government in relation to policy programs is less found than its business counterpart [13], [14]. According to Boyle [15], public sector productivity is difficult to measure and assessing the productivity of policy oriented organizations has proved particularly challenging. Kloot et al [13] explain that in local government the politic accountability plays an important role next to the traditional performance measurement aspects. While companies are to report on KRIs for their internal and external stakeholders such as employees, customers and shareholders, governments deal with a political system of responsibilities towards types of stakeholders. For example, citizens demand health policies from government while businesses demand open or at least healthy markets, which can be of opposite interest for citizens. In this research the political accountability is related to sustainability goals in the framework.
A typical categorization such as a balanced scorecard is of less value due to limited usefulness of KPIs on growth and innovation. Therefore, in this research it was concluded that strategy mapping and the use of literature based indicators should not be the starting point, particularly in a new and complex context such as electric mobility. Rather, given the large amount of stakeholders with their specific interests in the development and utilization of charging infrastructure it was chosen to take a stakeholder perspective as a starting point for identifying relevant result indicators.
The first step in this research was to collect responsibilities of policy makers from local governments that are responsible for the rollout of charging infrastructure. Based on discussions with policy makers of 4 major cities in the Netherlands (Amsterdam, Den Haag, Rotterdam, Utrecht) a first set of basic indicators was developed in a real world data based monitoring tool that enabled policy makers to (1) give insight in performance of charging infrastructure YTD (year to date) and (2) allow policy makers to communicate on governmental responsibilities to relevant stakeholders. An effect of the use of this monitoring tool by stakeholders was that policy makers started to request new insights as their acquaintance with the data grew. In the next step, these requests new indicators for this research were developed. The last step in this research was to collect KRIs, KPIs and potential interventions and project them on a literature based framework.

Relevant stakeholders for rollout of charging infrastructure
Developing public charging infrastructure has effect on multiple stakeholders for municipalities. Municipal programs with the goal to stimulate the development of charging infrastructure have to manage these different stakeholders and play a role in how policy makers related to this program evaluate the performance of the charging infrastructure. This goes beyond assessing whether sufficient charging points are provided, or whether a particular amount of charging sessions have been achieved for a certain month. Policy makers also have to manage how residents evaluate the development of charging infrastructure at the expense of parking spaces, as well as how EV users evaluate the availability of charging infrastructure in their neighbourhood. As such stakeholders' interests and their importance for policy makers provides an important starting point for assessing the major stakes they have to manage when it comes to the rollout of new charging infrastructure. Table 1 provides an overview of the five most prominent stakeholders which influence how policy makers evaluate the performance of charging infrastructure. The selection was based on interviews with policy makers involved in or responsible for the electric mobility program in a number of cities in the Netherlands. It includes concerns these stakeholders have and how these translate to desired results for policy makers that would support them in managing these stakeholder concerns.
First stakeholder is the municipality itself. Its sustainability goals have led to the investments in public charging infrastructure. Political accountability of municipalities includes the extent to which the public charging infrastructure contributes to sustainability goals (climate, air quality) but also at which costs this occurs (costs per reduced amount of emissions). In order to legitimize the investment program in developing charging infrastructure policy makers will typically require measurable results in terms of emissions reduced due to facilitated electric driving as well as the height of the investments relative to the amount of emissions reduced.
A second type of stakeholder are EV user in general, both non-residents as residents; by developing charging infrastructure municipalities aim to play a stimulating role of electric mobility on a larger scale than for its residents and commuters only. The desired outcome is then to attract a large number of users for the charging infrastructure, not only residents but also visitors, commuters and entrepreneurs. Providing accessibility of available charging infrastructure is then a major concern.
A third type of stakeholders are residents of the city. Municipal stimulation programmes for EVs tend to receive complaints from residents, for instance regarding charging points that are seldom used. Given that parking spaces are limited in most cities, loss of parking facilities due to the dedicated charging infrastructure leads to frustration among residents that policy makers aim to reduce by reducing the amount of under-utilized charging points in the city. Similarly policy makers have to consider the interests of EV user-residents in providing charging infrastructure that is available when required -which forms a balancing act to prevent both under-utilization as well as over-utilization. Relevant factors to consider and to utilize is different connection profiles of EV users [16], and for EV users the charging speed to be provided (also in view of growing battery size of new generation EVs on the market).
A fourth line of stakeholders to consider are (semi-)commercial parties responsible for the rollout and exploitation of charging infrastructure. Typically this includes two types of companies: (i) EVSE providers (electric vehicle supply equipment) providers (the providers of the charging station and ancillary components) and (ii) CPOs (charge point operators -they provide the charge point access e.g. through service cards). One of the underlying assumptions in EV programmes is that the role of the municipality is to overcome the initial chicken-egg problem for EVs by commissioning the first rollout of infrastructure, but in parallel the long term perspective of commercial parties taking over after this initial phase. In practice municipalities aim to support the charge infrastructure market but in parallel facilitate the transition to a positive business case of charging infrastructure to be taken up by the developers of charging infrastructure (EVSE providers and CPOs). Major concerns for the municipality then include managing costs and benefits of placement of public charging infrastructure. A last typical semi-commercial stakeholder that municipalities need to deal with are the grid operators (or DSO: distribution system operator). Grid operators have the responsibility to guarantee the functioning and stability of the grid. Given that EVs in larger amounts may have a significant impact on local capacity requirements, grid congestion and network stability, the municipality has to manage its ambitions regarding rollout of charging infrastructure to include concerns of network operators regarding the grid. However, apart from a possible threat to the grid stability, EVs can also provide a solution for grid operators through smart charging, for instance if vehicle2grid functionality is applied to use EVs as buffer stations enabling to provide energy during peak times, rather than adding to the peak with extra demands whilst charging. Policy makers of EV programs typically need to address the concern about grid stability, and broader need to address or facilitate the demands of grid operators to allow smart charging.

From objectives to performance indicators
Based on the stakeholder concerns in table 1 and related performance indicators the latter can be translated to the most prominent performance indicators, that can play a role in achieving these results. Performance indicators are discussed for the 5 goals associated with the respective stakeholders. Table 2 provides an overview of the KRI, KPI and possible interventions.

Sustainability goals -Municipality
Achieving sustainability goals A first and main objective for municipalities in the Netherlands in developing public charging infrastructure is to facilitate zero emission kilometres to contribute to air quality (reducing emissions of CO, NO x , PM) and to climate change (reducing CO 2 ). Contributing to the sustainability goals is directly related to the result indicator "amount of electricity charged (in kWh)", given that the indicator kWh provides a proxy for the amount of EV kilometres enabled by the charging infrastructure and thus for the amount of NO x , CO and PM prevented. Translation factors from kWh to amount of kilometres driven as well as on average emission factors of current car park are readily available to make relatively accurate estimations for air quality and climate change effects of the charging infrastructure.
Note that a limitation of this approach is that a significant amount of kilometres driven by the charged EV is likely to take place outside the city limits and therefore does not add to the air quality within the city itself. Localized measurements of air quality are required to compensate for these outer-city kilometres. Getting more insight in intercity traffic of EV users will become of growing interest; and could in time become an interesting intervention strategy for policy makers (e.g. funding charging infrastructure outside of its own city borders. The quest for policy makers then becomes to increase the amount of charged electricity, most prominently by simply adding charging stations. Another way is to stimulate more electricity charged per charging station, interesting for the ones with relative low performance on charge amount. Interventions may then lie in providing incentives to repark the EV once fully charged (thereby increasing the number of sessions and users per charging station). Indirect ways to influence charged electricity may include subsidy schemes to stimulate residents to purchase EVs, thereby increasing users and session on the CI. Another line for interventions could focus on increasing amount of electricity charged per session. Whereas currently an average session charges 8-9 kWh. Also in view of growing battery capacity of new EVs this is likely to increase autonomously, but may also be stimulated further in differentiated tariff structures (e.g. cheaper electricity prices above a certain amount of kWhs).

Achieving high effectiveness for mitigated emissions
In relative terms the effectiveness of investments versus the profits for air quality (euros per unit of NO x /Co/PM reduced ) or climate change (euros per ton CO 2 reduced) is an increasingly important result indicator for policy makers to legitimize the relatively high investments made in CI. This translates to the performance indicator "euro's per mitigated emissions". Possible intervention are similar to the ones under the sustainability goals, given that increasing the amount of charged electricity per charge station is a major focus. Naturally also interventions to reduce costs or increase benefits of CI can increase effectiveness of investments. Given the overlaps of the business case, this will be dealt with under goal number 4.

Facilitating electric mobility
A second category of objectives relate to the objective of municipalities to play a facilitating role for electric mobility. This largely relates to facilitating EV users in providing charging facilities but also candidate EV users, considering to buy an EV. Related result indicators include providing accessibility of charging infrastructure.
Accessibility to charging infrastructure is key to facilitating current (and future) EV users. One way to operationalize accessibility is coverage of charging stations in a city. Coverage can be operationalized from city perspective as in the m 2 covered by charging points based on a walking distance radius or by the mean meters walking from one charging point to the other. The limitation of this factor is that it measures supply capacity of charging in the city, and does not consider charging demand. Studies show how more affluent neighbourhoods tend to have higher share of EVs, leading to higher demand for charging. An even coverage of charging stations in a city would then likely lead to strong differences in competition between EV users to access scarce charging points (in neighbourhoods with many EV users). Therefore, the amount of relevant alternative charging points per user is an indication of accessibility as well.
The term accessibility refers to the existence of a good match between charge demand by EV users and the available capacity (supply) in a particular neighbourhood. Therefore policy makers will be largely concerned with managing the capacity utilization of current charge infrastructure: the percentage of time where charge points are occupied and hence not available for other EV users, where a capacity utilization of 100% indicates continuous occupation and 0% fully accessible all day. In cases of strong growth in EV sales (which is the case in many countries, amongst others the Netherlands) growth in capacity utilization (e.g. on a monthly basis) is a second indicator to monitor in order to be prepared to install new charge stations once a threshold level was reached.
Possible interventions obviously include adding charging stations in order to provide a better match between supply and demand. This can be monitored by measuring the number of unique users divided by the number of charge stations. Limiting factor is that users can be distinguished in different types with distinct charging behaviour, amongst others in inhabitants (night-charging), commuters (day charging) and visitors (occasional charging). In order to compensate for these different charging behaviour it may be considered to include a user-factor per every type of user, such as residents and commuters [16].
Analysis of the length of sessions in the city of Amsterdam shows that they have a large effect on the total connection time per charging point. Long sessions therefore cause an unnecessary decrease of accessibility for EV users. Monitoring this percentage per areas of scope (district, neighbourhood) is a necessity to take targeted measures.
A related KPI is the % of sessions with a low charge time ratio: the total amount of time charging divided by the total connection time. These sessions are mostly a burden during daytime and peak hours.
Interventions may include incentivizing removing fully charged EVs to increase accessibility for other EV users. Social interventions such as creating awareness for sharing charging spots might be helpful as well.
Given that accessibility is largely inhibited by long parking, interventions focused on removing fully charged EVs to make way for non-charged EVs can be powerful to achieve better utilization of the CI.

Optimizing capacity utilization
A third category of objectives for policy makers relates to public concerns about using scarce parking space for charging facilities. This largely translates to the earlier mentioned capacity utilization of charging infrastructure. Whereas optimizing accessibility concerned problems concerning over-utilization, scarce parking resources require the charging infrastructure not being under-utilized.
Under-utilization of a particular charging point tend to lead to complaints by residents about losing their scarce parking spaces due to charging points that are hardly used. Policy makers in several Dutch cities confirm that this regular practice, and that reacting to residents' complaints is a significant part of daily work; even though some of the claims may not hold true. Having actual data supports policy makers in responding quickly. The most important data then concerns the level op utilization of this charging station. A possible intervention could be the implementation of parking time windows that allow non-EV users to park cars on parking spots of charging stations from a certain time [17].
For policy makers finding the sweet spot between over-and under-utilization, or in KPI terms achieving sufficient level of utilization while retaining a sufficient level of accessibility for EV users. The topic of utilization is particular relevant on neighbourhood level or for a cluster of charging stations, given that policy makers decide upon further rollout of charging stations by observing (trends in) utilization of neighbouring charging stations.
More sophisticated KPIs could be developed to measure the effect of local competition on the utilization of charging points in a neighbourhood. For example the inter competition ratio can describes the competition on a charging point versus the competition between charging points. An imbalance of this ratio might indicate that EV users are fighting for one charging point while an alternative charging point is barely used. The charging point volatility of users within and between neighbourhoods can indicate the willingness of EV users to park their car in other areas due to competition for scarce resources.

Facilitating business case for charging infrastructure
Another concern for policy makers relates to improving the business case for public charging infrastructure, or somewhat broader, the facilitation of charging infrastructure development by commercial entities. Currently it is generally acknowledged that CI has a negative business case, reason for policy makers in numerous cities to have committed to laying the groundworks for a public charging infrastructure (overcoming chicken egg problem) [6], [7], [18]. Understanding and improving the business case is then high on the agenda of policy makers as well as the kind of actions municipalities may play in improving it. The business case in its most rudimentary form is basically made up of two factor: costs and benefits.

Costs of CI
The main costs for CI relate to hardware costs, preparation costs, installation, maintenance, electricity and grid connection costs. Particularly hardware and connection costs have a relatively high share in the total cost of ownership. Most of these costs (e.g. hardware, maintenance, installation, electricity) lie outside the span of control of municipalities. Nevertheless other cost factor provide opportunities for interventions and require monitoring to establish possible effects.
Typical interventions for municipalities may include lowering grid connection capacity -thereby reducing charging power but lowering grid costs. In Amsterdam it was decided to lower the grid connection from 3x35A (22kW) to 3x25A (11kW) so that per charging station 400 Euro per year was saved on connection costs (700 instead of 1100 Euro [19]). On a similar basis, municipalities may stimulate the rollout of hubsatellite systems, where a larger number of charging points are joined to one grid connection. Thereby the grid costs are shared among the number of sockets available. Another possible intervention to lower costs within the control of municipalities is lowering parking tariffs for EV users. Typical KPIs then include (i) cost-benefit ratios (to be calculated on a lifecycle basis), (ii) percentage of charging stations with positive business case, (iii) shelf life of current charging stations.

Benefits of CI
Benefits of charging infrastructure largely relate to price paid for electricity and associated tariffs incurred for parking. These may include starting tariffs, hourly tariffs, excess fees on popular charging times, but also (monthly/annual) subscription costs for services. Whereas in the Netherlands historically public charging stations merely had a flat fee (around 25 eurocents/kWh; mandated by municipalities to keep charging cheap), in recent years this market has become less regulated and particularly starting tariffs and hourly tariffs have been introduced by a number of CPOs. For municipalities the span of control lies in setting or relieving norms regarding electricity prices as well as introduction of other tariff structures that may increase the business case. Note that municipalities thus far have refrained from additional costs so as to stimulate the market for e-mobility.
Similar as the issue of accessibility, a positive business case is somewhat of a moving target and requires more hands-on result indicators to monitor to what extent one is moving forward. One way to do so is monitoring the percentage of charging stations with a positive business case over time and per area gives insight in the market readiness for both policy makers as well as CPOs. A more complex KPI is the potential amount of improvement of the business case that might be achieved after applying interventions of technological innovation. The potential amount of improvement can be simulated after applying interventions or technological innovation of historical data from charging infrastructure.

Preventing grid capacity problems
A growing concern for policy makers is the effect of charging behaviour on grid capacity. This is particularly the case with growing infrastructure and increased densities of charging stations in neighbourhoods. In the future this may lead to grid capacity problems, given that the exact effects of charging behaviour are currently not well understood. Adding charging stations automatically puts an extra strain on the grid; the question is to what extent this becomes intolerable for grid quality or requires further investments in the grid to allow growth in capacity usage. The responsibility for assuring the grid quality lies with DSOs; for municipalities the most important performance indicators relate to (i) peak power demand and (ii) peak shaving potential.
Reducing risks of power outage (result indicator) translates to peak power requirements (performance indicator) on neighbourhood level (e.g. on clusters of networks in the low voltage domain, or on the level of sub-stations of the grid): this is the most likely aggregate level where stress in substations of the grid occur and where peak loads should be reduced where possible. Related indicators include the frequency and length of power peaks to give an indication of how often and how large the problems occur.
The peak shaving potential relates to the potential for solutions such as smart charging to reduce power peaks. Possible interventions for a municipality include enabling smart charging, for instance by stimulating or facilitating delayed charging and flexible capacity charging (e.g. lower capacity charging during peak times in order to relieve the system).
Charging infrastructure has significant effects on the electricity grid, not only in terms of capacity and potential investments in its capacity, but also on congestion management, power quality (particularly in case of vehicle2grid solutions) and potential buffering for sustainable electricity production. These aspects are mostly the responsibility of the DSO but strongly relate to intervening power of municipalities.
Therefore, policy makers are advised to share these KPIs with the grid operators, particularly in case of intensified rollout strategies.
The level of balance of local power demand and local power supply in the portfolio of a municipality's charging infrastructure is a KPI that measures the effectiveness of the rollout strategy from the grid capacity perspective. For example, having 3-phase high power charging points in a neighbourhood where only 1-phase PHEVs with small batteries are present indicates a mismatch in the portfolio. Secondly a balance in supplied technology versus technological capabilities of EVs is to be achieved in the portfolio of the charging infrastructure. A well balanced portfolio might include standard charging points in areas with standard charging behaviour but also newly adopted technology in test areas to obtain insights in future adoption of EV technologies. Examples like local vehicle to grid solutions as test cases, smart chargers for e-taxis or scheduled charging could reveal directions for future rollout solutions to be implemented in cities. Table 2 provides an overview of the identified KRIs and KPIs. The list is not exhaustive in the sense that also other KRIs and KPIs can be found; but for the sake of selecting the 'key' indicators the endeavour was to keep the list limited in size. It can be observed that a number interventions focused on increasing the utilization of CI apply for several of the result indicators. Similarly applying smart charging policies contribute to several result indicators as well. The balancing act for municipalities is in providing sufficient accessibility while preserving sufficient utilization of and thereby the business case for the CI.

Reflection on performance indicators
KPIs vary from city level (overall infrastructure) to neighbourhood and even charging station level. A majority of KPIs can be extracted from transaction data from the charging infrastructure while a few of the KPIs may require simulations as input. Data derived from the use of charging infrastructure by EV users is essential for policy makers for effective rollout and optimization of the use of charging infrastructure. Therefore municipalities should set stringent requirements on the type of data they collect from the providers of CI and arrange support in analysing the data for optimization purposes. The above tables provide suggestions which type of indicators should be monitored to do this effectively.

Conclusions
Performance measurement of charging infrastructure is essential for effective rollout and operation. This requires development of Key Result Indicators (KRIs) and Key Performance Indicators (KPIs) that give policy insight to communicate with stakeholders and implement interventions. In this paper a two-step approach was followed to extract relevant KPIs by first analysing the stakeholders of policy makers (leading to a number of objectives and result indicators) and second to translate these objectives in KPIs and related intervention options. In total 10 KRIs and 14 KPIs were identified as relevant monitoring instruments with policy makers engaged in the rollout of charging infrastructure.
Recommendations for further work include further testing the approach with different municipalities in different stages of CI development, as required performance indicators may change within these different stages. Also KRI and KPI should be quantified where possible (including minimum and maximum values), so as to provide more practical steering opportunities for municipalities and have even clearer evaluations as to how well they are doing on their respective performance indicators (and to what extent interventions are working out). Based on the existing approach also KRIs and KPIs will be developed for other stakeholders in the value chain of CI, such as DSOs, CPOs and utilities; also to establish possible conflicts in interest in particular KRIs and how they may be aligned. Lastly it is recommended to measure the effect of interventions, e.g. by experimenting locally with implementing interventions to test the effect of interventions on KPIs might be needed in order to develop a context based optimal policy.