Operating and Investment Models for Energy Storage Systems

Abstract

In the context of climate changes and the rapid growth of energy consumption, intermittent renewable energy sources (RES) are being predominantly installed in power systems. It has been largely elucidated that challenges that RES present to the system can be mitigated with energy storage systems (ESS). However, besides providing flexibility to intermittent RES, ESS have other sources of revenue, such as price arbitrage in the markets, balancing services, and reducing the cost of electricity procurement to end consumers. In order to operate the ESS in the most profitable way, it is often necessary to make optimal siting and sizing decisions, and to determine optimal ways for the ESS to participate in a variety of energy and ancillary service markets. As a result, many publications on ESS models with various goals and operating environments are available. This paper aims at presenting the results of these papers in a structured way. A standard ESS model is first outlined, and that is followed by a literature review on operational and investment ESS models at the transmission and distribution levels. Both the price taking and price making models are elaborated on and presented in detail. Based on the examined body of work, the paper is concluded with recommendations for future research paths in the analysis of ESS.

1. Introduction

Liberalisation of the power sector caused electricity to become commodified and traded in the markets. However, unlike other commodities, electrical energy cannot be stored in its original form and the power systems are operated with the goal of maintaining the balance between the consumption and the production of electricity at all times. As our society recognised human influences on the environment and started to require more renewable energy sources (RES), maintaining power balance became a much harder task and consideration of the uncertainties caused by the intermittent energy sources became imperative. Several solutions for addressing RES intermittency exist: installing new, fast ramping generators such as gas power plants, building new transmission lines to secure power supply in the events of renewable energy shortage, designing demand response programs in which the demand is managed to meet the production and using energy storage systems (ESS) to store the surplus and supply the shortage of electricity. While the term ESS can generally represent a larger set of energy storing technologies, in this paper we use it to describe a set of technologies that enable storing of electricity in some other form: potential energy in pumped-hydro plants, kinetic in flywheels, electrochemical in batteries, etc.

ESS as a market participant changes its role from the generating unit to the consumer depending on the market conditions. This puts the ESS in a unique position and gives it an opportunity to strategically choose its market position in order to maximise profit more efficiently than producers and consumers, who can only sell or buy in the markets. Figure 1 presents a concise overview of the models representing the ESS either as a non-strategic player (price-taker) or a strategic player (price-maker). A price-taker has no influence on market prices and bids competitively. On the other hand, a price-maker is a strategic player that exercises market power by bidding over its marginal price or by withholding capacity. The two terms are not completely accurate because even a non-strategic ESS can influence market prices, as was shown in [1]. For this reason, we adopt terms “strategic” and “non-strategic” instead of “price-maker” and “price-taker.” In the investments phase, a strategic ESS, as opposed to a non-strategic one, tends to install larger ESS facilities. In the operational phase, strategic ESS, compared to the non-strategic one, is generally signified by higher profits earned through market participation.

Based on the system connection and size of storage, technical literature divides the ESS in two groups: transmission-level and distribution-level. Transmission-level ESS are large-scale installations connected to the transmission network such as pumped-hydropower stations, compressed air energy storage plants and large-scale battery storage plants. Sizes of these facilities range from a couple of megawatts to a couple of gigawatts. Distribution-level ESS are smaller systems connected to the distribution network which can be placed at the consumers’ premises (behind the meter) or be a part of a microgrid, virtual power plant or distribution grid operator’s (DSO’s) assets. The size of such facilities depends on the distribution system operator’s grid rules and is usually less than one megawatt. We adopt this approach as well, analysing transmission- and distribution-level ESS separately. The analysed body of literature consists of 57 articles on investments and 77 on the operating of ESS in transmission and distribution systems. Various markets in both investment and operating phases were considered for ESS participation in these papers, which is outlined in Figure 2.

Figure 3 shows a large gap between the number of papers dealing with transmission- and distribution-level ESS in operational phase. The gap is understandable if we take into account the fact that large numbers of papers on ESS operating in real-time markets employ optimal control algorithms, which are out of scope of this review.

Technological background for ESS can be found in [2,3,4]. Luo et al. [2] presented an overview of ESS technologies and listed possible applications for ESS in electrical power systems. A more recent paper by Koohi-Fayegh and Rosen [3] also focused on technologies and listed a smaller number of potential applications, concluding with a list of technological issues that researchers are facing, such as the need to increase the cycling ability for electrochemical storage and to discover new materials for all types of energy storage. The paper singled out hydrogen storage as the most promising technology for the future. A comparison of conclusions from [2,3], indicates that the majority of the earlier issues have been solved, but some still remain open, such as seasonal storage, especially at the distribution level. Khan et al. [4] presented not only an overview of ESS technologies but also the potential for storing primary energy sources, i.e., natural gas and coal. This is interesting in the context of multi-energy systems where ESS could be displaced by primary energy source storage.

ESS planning is a widely reviewed area. Awadallah and Venkatesh [5] presented a modelling framework and reviewed literature dealing with ESS planning and operation in distribution networks but did not address uncertainties in models. As a conclusion, they expressed the need for more general studies in the distribution systems—the reason being that the body of literature contains results that are hard to generalise. They also concluded that more research on market participation of the ESS in distribution network is needed and that development of techniques for long-term large-capacity ESS operation are necessary to enable seasonal price arbitrage. Lorente et al. [6] performed a short review of research papers on ESS planning published between 2016 and 2018 and concluded that the ESS impact on prices is commonly ignored in the planning models. They also stated that more research on ESS siting is needed. After a thorough literature review wherein they categorised ESS expansion planning models via different modelling approaches and listed objective functions and constraints, Sheibani et al. [7] recognised open issues in ESS expansion planning. These included the necessity of risk assessments for investors, determination of the optimal financial support for ESS expansion and consideration of different services to system operators. The authors stated, based on conclusions from several case studies, that it is not profitable for the ESS to participate in only one market and the investors must consider more revenue streams. This is in accordance with conclusion drawn by Zidar et al. [8], who reviewed solving methods for ESS siting and sizing in distribution grids, categorising them into: mathematical programming, exhaustive search, analytical methods and heuristic methods.

Optimal financial support for ESS was addressed by Miller and Carriveau [9], who gave a review of financing opportunities for ESS investors. They presented the state-of-the art of the financing schemes categorised as: governmental incentives, partnering with the renewable technologies and innovative finance models. Koohi-Fayegh and Rosen [3] confirmed that governmental policy support will play a large role in development of ESS technologies. However, the latest EU energy legislation, except for some special circumstances, forbids the transmission system operators (TSOs) and DSOs from owning ESS [10]. It remains to be seen whether this will slow down ESS integration in the European power system or will just shift the focus to the privately owned ESS.

Mejia and Kajikawa [11] performed data analysis of a large number of papers and patents. Their findings show an overlap in topics covered by the research community and industry that contains optimisation techniques for ESS operation and planning and various topics in the area of materials science.

Traditionally, ESS have been used for peak-shaving and energy arbitrage, but nowadays they are considered for balancing, congestion management and other purposes as well. As many ESS technologies are reaching maturity and their investment costs are decreasing, the following questions have arisen: how can they operate profitably in today’s markets; what capacity and which storage technology should be installed; and where should they be placed? To answer these questions, the scientific community uses mathematical models for simulation and optimisation. In this paper, we present the work aimed at answering these questions. We concentrate on the research of market participation of the ESS during the operational and investment planning phases. Our contribution to the body of literature is a detailed survey of mathematical models for the analysis of ESS used for said purposes. Based on the survey, we provide recommendations for future research in the area of market-participating ESS.

The paper is outlined as follows. The standard mathematical model of ESS is given in Section 2. Section 3 presents a detailed literature survey on ESS market participation, and Section 4 presents a literature survey on expansion planning. Section 5 describes the ways of dealing with the computational complexity of ESS models. We conclude the paper in Section 6.

2. Energy Storage System Models

This section presents a standard model that represents any type of ESS mathematically, without assuming any technological details. As a generic mathematical model, the measuring units associated with the variables and parameters are there for illustration purposes and can be scaled up or down. Depending on the modelling objective, some of the constraints from the following set can be left out or modified:

$$\begin{array}{ccc}\hfill & 0\le p_t^{\mathrm{ch}}\le {\overline{P}}^{\mathrm{ch}}·x_t,\forall t\in T\hfill & \end{array}$$

(1)

$$\begin{array}{ccc}\hfill & 0\le p_t^{\mathrm{dis}}\le {\overline{P}}^{\mathrm{dis}}·(1-x_t),\forall t\in T\hfill & \end{array}$$

(2)

$$\begin{array}{ccc}\hfill & e_t\le \overline{E},\forall t\in T\hfill & \end{array}$$

(3)

$$\begin{array}{ccc}\hfill & e_t\ge \underline{E},\forall t\in T\hfill & \end{array}$$

(4)

$$\begin{array}{ccc}\hfill & e_T\ge E_0,\hfill & \end{array}$$

(5)

$$\begin{array}{cc}\hfill & e_t=e_{t-1}+\Delta T{p}_t^{\mathrm{ch}}{\eta }^{\mathrm{ch}}-\Delta T{p}_t^{\mathrm{dis}}/{\eta }^{\mathrm{dis}}-\Delta T{p}_t^{\mathrm{loss}},\forall t\in T\hfill \end{array}$$

(6)

Equation (1) constrains the ESS charging power below its charging power rating ${\overline{P}}^{\mathrm{ch}}$ and Equation (2) does the same for the discharging power rating at ${\overline{P}}^{\mathrm{dis}}$. In the ESS siting and sizing models, the right-hand-side (RHS) coefficients can be variables instead of parameters. Binary variable $x_t$ ensures that the ESS is never charged and discharged at the same time. Generally, binary variables turn a model into a mixed-integer program, which complicates the solution procedure, which is the main reason for neglecting them in the models. Binary variables can be omitted without consequences if the considered market conditions are such that it would not be profitable for the ESS to be both charged and discharged at the same time. However, simultaneous charging and discharging is profitable, assuming imperfect efficiency of the charging/discharging cycle, during negative market prices.

In addition to constraints (1)–(6), Tejada-Arango et al. [12] constrained the ESS power by a ramping constraint for transition between charging and discharging mode as follows:

$$\begin{array}{cc}\hfill & \left(e_t^{\mathrm{dis}}-e_{t-1}^{\mathrm{dis}}\right)-\left(e_t^{\mathrm{ch}}-e_{t-1}^{\mathrm{ch}}\right)+r_t^{+}\le \tau RU\forall t\in T\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill & \left(e_t^{\mathrm{ch}}-e_{t-1}^{\mathrm{ch}}\right)-\left(e_t^{\mathrm{dis}}-e_{t-1}^{\mathrm{dis}}\right)+r_t^{-}\le \tau RD\forall t\in T\hfill \end{array}$$

(8)

In Equations (7) and (8) $e_t^{\mathrm{dis}}$ and $e_t^{\mathrm{ch}}$ are discharged and charged energy during one time period, $r_t^{+}$ and $r_t^{-}$ are up and down ramping capacity reserves of the ESS and $RU$ and $RD$ are ramping limits. Ramping constraints of the ESS are often ignored because of the assumed instantaneous change in the power input or output levels. It was shown by Poncelet et al. [13] that generators’ flexibility constraints play an important role in the ESS investment models. The same should be true for the ESS flexibility constraints, especially those ESS offering flexibility services to the system. Therefore, it is expected that more models with ESS ramping constraints will appear in the future papers.

Equation (3) limits the state of energy from above to its energy rating $\overline{E}$, and Equation (4) from below to $\underline{E}$. It is important to impose this lower limit to state of energy to decrease the rate of degradation for batteries or to preserve the minimum water levels in pumped-hydro storage units. Equation (5) ensures that the final state of energy is not lower than the initial one ($E_0$). This way it is certain that the ESS does not make profit by merely selling the leftover energy from the previous optimisation period and the model is simpler to incorporate in the long-term optimisation problems.

The last Equation (6) represents the state of energy calculation for all time periods of the considered time horizon. While Equations (1) and (2) constrain ESS power and Equations (3)–(5) constrain its energy levels, Equation (6) connects the two. Instead of state of energy (SOE), an absolute value that has a physical meaning in the field of power system economics, some authors calculate state of charge (SOC), which represents percentage of SOE relative to its energy rating. The first term in this equation represents the state of energy at the previous time step. This term is usually replaced with $E_0$ for the initial time period. Hemmati et al. [14], who included in their model initial state of energy as a decision variable, showed that the initial state of energy has an impact on planning decisions and should be selected with care. The second term is the amount of energy charged into the ESS during period $\Delta T$, and the third is the amount of energy discharged from it during the same period, with the assumption that the said powers are constant over the time period. The parameters ${\eta }^{\mathrm{ch}}$ and ${\eta }^{\mathrm{dis}}$ are charging and discharging efficiencies of the ESS. The efficiencies used in this equation are not known for electrochemical ESS so one round-trip efficiency is commonly used instead of the separate charging/discharging efficiencies [15]. The last term represents the lost energy. This term can also be represented as a percentage of the previous state of energy and written as a coefficient of $e_{t-1}$. Losses materialise because of the self-discharge of the ESS over time. When modelling the ESS short-term operation, such as participation in the day-ahead or intraday markets, this term is often ignored [16,17], but it is taken into account in the long-term models [18].

Simple as it is, the presented generic model is not the best representation for every ESS technology. It works well for the energy storage technologies where the state of energy depends linearly on the charging power. However, batteries are usually charged with the the constant-current–constant-voltage (CC–CV) characteristic. In the CV charging phase, with the constant voltage and decreasing charging current, this model is not accurate. Several different battery models have been developed to address this issue. It was shown that more accurate models perform more realistically in the market environment. The first such model comes from [19], and reads:

$$\begin{array}{cc}\hfill & p_t^{\mathrm{ch}}\le {\overline{P}}^{\mathrm{ch}}·x_t,\forall t\in T\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill & p_t^{\mathrm{ch}}\le {\overline{P}}^{\mathrm{ch}}·x_t·\frac{\overline{E}-e_t}{\overline{E}-E^{\mathrm{cc}-\mathrm{cv}}},\forall t\in T\hfill \end{array}$$

(10)

In these equations $E^{\mathrm{cc}-\mathrm{cv}}$ is the state state of battery’s energy at the CC–CV breakpoint. The authors here accounted for the decreasing charging current in the CV phase by assuming that the current is linearly decreasing and the maximum charging power is limited accordingly. Both Equations (9) and (10) limit the charging power, but if the state of energy is above the CC–CV breakpoint, only the (10) is the binding constraint.

The second approach, which uses the $\Delta e-e$ characteristic to represent how much energy is left to be charged to the ESS depending on the current state of energy, was presented in [15]. This approach uses a piece-wise linearisation of the $\Delta e-e$ function to limit the charging power at all time periods as follows:

$$\begin{array}{cc}\hfill & e_t=\sum _{i=1}^{I-1}{e}_{ti},\forall t\in T\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill & e_{ti}\le R_{i+1}-R_i,\forall t\in T,i\in I\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill & p_t^{\mathrm{ch}}=\frac{F_1}{\eta \Delta T}+\sum _{i=1}^{I-1}\frac{F_{i+1}-F_i}{R_{i+1}-R_i}·\frac{\Delta e_t}{\eta \Delta T},\forall t\in T\hfill \end{array}$$

(13)

In Equations (11)–(13) $F_i$ are breakpoints on the $\Delta e$ axis, and $R_i$ are breakpoints on the e axis.

Gonzalez-Castellanos et al. [20] defined non-linear charging and discharging characteristics and approximated them by a convex combination of the sampling points. This way, the structure of Equations (1) and (2) does not change and the non-linearity is addressed through the RHS coefficients alone, specifically maximum charging and discharging power and state of charge. The authors in [20] also recognised the non-linear connection of the battery efficiency to the state of charge and the charging/discharging power. They used the input and output power as a third dimension for the convex approximation of the charging/discharging characteristic and defined them as follows:

$$\begin{array}{cc}\hfill & P^{\mathrm{in}}=P^{\mathrm{ch}}·{\eta }^{\mathrm{ch}}\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill & P^{\mathrm{out}}=P^{\mathrm{dis}}/{\eta }^{\mathrm{dis}}\hfill \end{array}$$

(15)

Discharging power is sampled by $[P^{\mathrm{dis}},SOC,P^{\mathrm{out}}]$ and charging power by $[P^{\mathrm{ch}},E,P^{\mathrm{in}}]$.

In the short-term models, battery cycling is not considered, but for the mid- to long-term optimisation, the cycle life constraints should be included in the model as well. There are several ways to include the cycle life in models. Duggal and Venkatesh [21], Kazemi and Zareipour [22], He et al. [23], Xu et al. [24] and Padmanabhan et al. [25] included battery life duration in the short-term and long-term scheduling objective functions. Duggal and Venkatesh [21] and He et al. [23] defined battery lifetime as exponential function of its depth-of-discharge (DOD) and showed that the battery lifetime decreases as the number of daily cycles increases. Padmanabhan et al. [25] linearised this function and showed that the linear expression is a good enough approximation of the original. The lifetime variable was used to analytically calculate the annual operating and maintenance cost that is, in turn, used to calculate daily operating cost, which is included in the objective function. Similarly, Kazemi and Zareipour [22] maximised the ESS profit function, calculated as a difference between the short-term revenues and the annual investment cost. Lifetime used to calculate the annual investment cost was approximated by the rainflow counting algorithm. This algorithm is commonly used to determine stress caused by cycling in any system. For batteries, the stress is proportional to the number of cycles and DOD [26]. Vejdan and Grijalva [27] included cycle life in the objective function as a coefficient calculated by dividing the capital cost per unit of capacity by double the number of life cycles. Qiu et al. [28] included battery lifetime in their model as a capacity coefficient in the state of energy calculation. This coefficient is calculated for each year as a combination of calendar and cycling ageing and decreases as years go by, reaching zero when the ESS is at the end of its life. Hajia et al. [29] included a nonlinear lifetime characteristic for battery ESS depending on the number of cycles and DOD in an expansion planning model. A different approach was presented by Gantz et al. [30] who limited the number of cycles in a month through a constraint. The limit in this constraint is calculated from data found in [31] by dividing the expected number of cycles by design life (in months). Mohsenian-Rad [32] used a constraint to limit the number of daily cycles as well and showed that choosing the number of daily cycles is important for finding a trade-off between the yearly profit and the battery lifetime.

Notice that all constraints mentioned until now refer to the real power modelling. However, papers that also consider reactive power injections by ESS have started to appear. These are mostly models focused on distribution grid applications of ESS, where AC power flow is more common than in the transmission-level studies. Some papers incorporate AC network constraints without considering possibility of ESS to offer reactive and real power [33]. However, there are also those that implement AC constraints for ESS [34,35,36,37,38]. Models with AC power flow are more rare because the non-linearity of the constraints renders them harder to solve. While DC power flow models are good enough for economic analysis, AC power flow is necessary for investigation of technical aspects of ancillary services such as voltage regulation. In addition to Equations (1)–(6), such models contain the following constraints on reactive power flows [39,40]:

$$\begin{array}{cc}\hfill & p_t=p_t^{\mathrm{dis}}-p_t^{\mathrm{ch}}\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill & p_t^2+q_t^2\le s_t^2\hfill \end{array}$$

(17)

where $p_t$ is net real power flow, $q_t$ is reactive power flow and $s_t$ is maximum apparent power flow of the inverter.

3. Market Participation of the ESS

In this section, an overview of the current state of the research on market-participating ESS is given. Figure 4 shows markets in which the ESS participation was investigated in the reviewed literature. It is evident from Figure 4 that most of the literature considers ESS participation in energy markets and fewer number of papers consider ancillary service markets. A big gap between these numbers is understandable because most papers in which ancillary service markets are considered also include energy market participation. The reasoning behind this is that it is cheaper for the ESS to buy electricity in the energy markets to prepare for offering ancillary services to the system than to participate only in an ancillary service market. Figure 4 also shows that there is much less literature that covers the market participation of distribution-level ESS. This is not a surprise, as the wholesale markets often have rules that limit the participation of smaller distributed resources. The number of papers addressing market-participating ESS at the distribution system level is expected to grow with the development of flexibility markets at this level.

3.1. Different Applications of ESS in Power Systems

The reviewed literature can be divided into three large groups based on the application of ESS: market arbitrage, supporting RES integration and long-term self scheduling. In recent years, both the scientific community and the industry are considering the ESS for other applications, such as voltage regulation and black starts. This section gives an overview of the key findings on these applications. A detailed overview of different ESS applications can be found in [41].

3.1.1. Market Arbitrage

The most common market operation of an ESS is price arbitrage, i.e., buying energy when the prices are low and selling it when the prices are high. In electricity markets, this generally coincides with the low and high power consumption so the price arbitrage is also the energy arbitrage. However, the term energy arbitrage usually denotes operation outside of wholesale markets, e.g., balancing energy fluctuations from RES or variable loads in microgrids. Arbitrage was explicitly considered by many researchers. Thatte et al. [42] used robust optimisation to determine the optimal bidding strategy for ESS performing arbitrage in the day-ahead market. Xia et al. [43] showed by simultaneous perturbation method that an ESS in unit commitment can be used for energy and emissions arbitrage. Mohsenian-Rad [44] considered using distributed ESS for price arbitrage in a coordinated way so some ESS can be buying while others are selling electricity. The presented case study indicates that congestions are beneficial for this behaviour of ESS. Shafiee et al. [45] had an ESS performing arbitrage while taking into account uncertainty of price forecasting. Wang and Zhang [46] modelled ESS performing arbitrage in a real-time market as an arbitrage maximisation problem. Ciftci et al. [47] placed an ESS within a microgrid to be used for energy arbitrage and load following. They concluded that the possibility to offer multiple services to the microgrid might encourage investments in ESS.

A special type of arbitrage is the inter-temporal arbitrage between two time-scales, e.g., the day-ahead and the real-time markets, which was investigated in [48,49,50]. Braun [48] modelled a pumped-hydro storage and showed that inter-temporal arbitrage allows storage operators to exploit price differences between the two markets to optimise their short-term positions. Krishnamurthy et al. [50] showed that ESS performing inter-temporal arbitrage between day-ahead and intraday markets result in higher profits than those performing arbitrage only within the two markets. Zakeri and Syri [49] modelled a battery storage performing arbitrage between day-ahead and intraday markets in Nordic countries and concluded that a high share of hydropower plants in these markets reduces profitability potential for batteries.

3.1.2. RES Balancing

Another business case for an ESS is balancing the production from intermittent resources. Many papers investigated the possibility of applying an ESS for balancing variable wind production through coordinated optimisation or market mechanisms such as unit commitment. Castronuovo et al. [51] recognised three major ways in which an ESS and an RES can be coordinated: (1) as one facility in which the ESS cannot purchase power from the market so it is only used for wind power balancing, (2) ESS is independent of the wind power plant but can provide reserve to balance its production and (3) both systems are a part of the same virtual power plant. This categorisation is on a trace of the general categorisation of ESS used for RES balancing. They can either be a part of the same or separate facilities. Usaola [52] analysed the profit potential of a thermal solar plant with liquid salt storage participating in the day-ahead market. Rahimiyan and Baringo [53] modelled combined ESS-wind power plant facility performing arbitrage between the day-ahead and intraday markets. Shu and Jirutitijaroen [54] concluded that for very small wind power plants (a few kilowatts) it would not be profitable to install an ESS for balancing. Yuan et al. [55] showed that capacity of an ESS in such combined facility influences profit and should be a decision factor along with the bidding strategy. Garcia-Gonzalez et al. [56] compared performances of the uncoordinated and coordinated operation of an ESS and a wind power plant and concluded that the coordination of the two increases profits for both facilities. The same conclusion was reached by Khodayar and Shahidehpour [57] and Daneshi and Srivastava [33], who compared the profits of a wind power plant and an ESS owned by the same company with and without coordination. Coordinated operation of an ESS and a wind power plant was also analysed by Sánchez de la Nieta et al. [58] who developed bidding strategies and tested them on Iberian market data. The same analysis was performed by Thatte et al. [59] on West Denmark market data. Jiang et al. [60] considered using ESS to mitigate risks related to wind forecast errors and to minimise system operation cost in centralised unit commitment. Li et al. [61] showed that ESS in unit commitment models with wind power plants decreases wind curtailment, load and reserve shortfalls and total system operation cost.

3.1.3. Self-Scheduling

Generalisation of the models that include coordination with intermittent resources are the self-scheduling models. The need for self-scheduling comes from the balancing responsibility and the goal of profit maximisation of market participants in decentralised markets. Varkani et al. [62] proposed a self-scheduling strategy for coordination of a pumped hydro plant and a wind power plant participating in the day-ahead market, while the pumped hydropower plant participates in the regulation reserve market as well. Parvania et al. [63] compared results of self-scheduling models for decentralised and centralised ESS, and the results indicate that the centralised ESS is more beneficial for the system. Self-scheduling models are often used for modelling the long-term operation of ESS. Such models are less common in the literature, which confirms that the long-term operation of ESS is still an open issue [2,3]. While Kazempour et al. [64] modelled a pumped-hydropower plant as part of a cascade to maximise profit of the whole system, Kazempour et al. [16] considered a self-scheduling pumped-hydropower plant aiming to optimise amount of energy to be store for one week ahead. Baslis and Bakirtzis [65] presented a pumped-hydropower plant in a long-term self scheduling model aware of its influence on prices and showed that such facility utilises strategy led by long-term objectives, independent of the short-term water inflows. Specifically, long-term contracts for large volumes can motivate ESS to behave counter-intuitively in the day-ahead market, dropping the price during the discharging hours because the energy is actually sold at the forward-contracted price. Thatte et al. [42] optimised the bidding strategy of an ESS performing arbitrage for one day and for period of 90 days. Kazemi and Zareipour [22] focused on the long-term scheduling of a battery considering the impact of the short-term operation on the battery lifetime in the long run. Pandžić et al. [66] analysed the influence of the optimisation period on profits of an ESS operator and concluded that it is best for a virtual storage plant to optimise operation over the scheduling horizon of at least two days. Alvarez et al. [18] used the future cost function to model the influence of ESS’ long-term strategy on its short-term operation. They showed that fixing the ESS’ state of energy to the previously scheduled long-term amounts increases operational costs and therefore decreases profit of the virtual power plant (VPP).

3.1.4. Ancillary Services

Besides energy arbitrage and RES balancing, an ESS can offer various other ancillary services to the network operators. Unlike the ESS performing arbitrage, those ESS that offer predominantly ancillary services make smaller numbers of cycles and therefore last longer. This was confirmed by Fleer et al. [67] who assessed profitability of ESS offering primary frequency reserve in German markets considering the uncertainties of reserve prices and investment costs. Thien et al. [68] considered ESS participation in German reserve markets but here the emphasis was placed on the market rules, showing that the decrease of duration of the traded products from 30 to 15 min is beneficial for the ESS. German balancing markets also allow ESS participation, the profitability of which was investigated by Olk et al. [69]. ESS considered for frequency reserve was already incorporated in many market environments, as is evident from

Table 1. Literature survey on the competitive ESS operating at the transmission level. (Abbreviations: CAP, capacity; EN, energy only; VR, voltage regulation; RM, reserve market; BM, balancing market).

Modelling Technique	Network Unconstrained	Network Constrained	Trading
Deterministic	[70]		CAP
	[48,49,52,64,71,72,73,74,75]	[20,25,33,43,63,76,77,78]	EN
	[71]		VR
	[49,64,71,75]	[25]	RM
	[64,71]		BM
Stochastic	[1,23,50,54,56,58,62,79,80]	[32,57,61,81]	EN
	[23,62]		RM
	[1,23,62]		BM
Interval		[82]	EN
Robust	[16,22,60,83]		EN
	[16,22,83]		RM
	[16,83]		BM
Chance-constrained	[58]	[51]	EN
Risk-contrained	[16]		EN RM BM

,

Table 2. Literature survey on the ESS operating at the distribution level. (Abbreviations: EN, energy only; VR, voltage regulation; RM, reserve market; BM, balancing market).

Modelling Technique	Network Unconstrained	Network Constrained	Trading
Deterministic	[87,88]		retail
	[30,78,89,90,95]	[35,71]	EN
		[71,85]	VR
		[71]	RM
		[35,71]	BM
Stochastic	[18,94]		EN
Robust	[53,96]		EN
Chance-constrained	[47]		EN

and

Table 3. Literature survey on the strategic ESS operating at the transmission level. (Abbreviations: EN, energy only; RM, reserve market; BM, balancing market).

Modelling Technique	Network Unconstrained	Network Constrained	Trading
Deterministic	[72,73,100,102,103,104,105]	[17,66,76]	EN
	[102]		RM
	[102,105]		BM
Stochastic	[27,55,65,106]	[18,44,86,101]	EN
	[106]		RM BM
Robust	[42,59,107]	[44]	EN
Chance-constrained	[55]	[92]	EN
Risk-contrained	[45]		EN

.

Other ancillary services are usually not procured through centralised markets, so the literature is less thorough in the areas of the market-based profitability of black start, voltage regulation and other services. There are, however, many examples of researchers investigating the possibilities of using ESS for such purposes. Black start is an interesting option for the ESS but even more so for hybrid facilities comprised of an ESS and a generator. Li et al. [84] presented a method of configuring an ESS combined with a wind power plant for the purpose of establishing the voltage and frequency for starting-up thermal power plants. Large shares of RES installed in distribution grids can cause nodal voltages to increase or flicker. The idea of using ESS to regulate voltages has been gaining popularity in recent years.

Most of the research in this area deals with voltage control in distribution networks, where ESS can impact voltages by changing its charging/discharging level. Sugihara et al. [85] proposed a subsidy programme offered by the DSO to the ESS owners in order to control their ESS when there is a need for voltage control and proved that it is possible to use customer-owned ESS for voltage regulation. Opathella et al. [71] showed that an ESS of any size can collect revenue by selling various ancillary services. Their results indicate that the largest sources of revenue for ESS are voltage and frequency regulation, while energy arbitrage, reserve and black start bring much smaller shares of the profits.

3.2. Market Opportunities for ESS at the Transmission and Distribution System Level

Based on the voltage level a facility is connected to, ESS are categorised as transmission- or distribution-level ESS. ESS connected to the high-voltage transmission networks are large-scale facilities that can participate in wholesale markets or offer various services to the power system and its users. For most of transmission-level ESS applications, DC load flow is an acceptable approximation and reactive power flows are not modelled. However, this does not stand for the distribution-level ESS that are connected to the medium and low voltage levels. They are generally smaller in size and thus unable to participate in wholesale markets directly. However, there are various solutions that allow their indirect participation, e.g., via aggregators. Directly, facilities from this category can offer energy and ancillary services in retail or local energy markets.

Retail markets were traditionally set up only for electricity consumers to procure necessary energy, and the adoption of the prosumer paradigm has started only lately. Markets at the distribution system level, both energy and ancillary services, are still not as developed like those at the transmission level; thus there are far fewer profit opportunities for the ESS there. Furthermore, while the transmission-level markets can have price-setting mechanisms based on network constraints, current distribution-level market design is much more complicated and specific to the location. In other words, the influence of the location on ESS profits is easy to calculate at the transmission system level, as opposed to the distribution system level where each DSO can have its own congestion-management mechanism and incentive scheme set in place.

In this work ESS are categorised as transmission or distribution-level based on the distinction in size and possible applications as the asset.

3.2.1. Transmission System-Level ESS

As shown by many researchers who modelled ESS operation at the transmission level, placement of an ESS within the grid influences its profitability. Li and Hedman [79] showed that transmission contingencies can limit the amount of power that an ESS can deliver or store, increasing the wind spillage. Wang et al. [76] showed that congestion is beneficial to the ESS profit in the markets where locational marginal pricing (LMP) is used. The same conclusion is supported by Mohsenian-Rad [44]. The LMPs are used in all papers covered by this study but one, where zonal markets are considered. Weibelzahl and Märtz [77] investigated how ESS influence optimal zonal decomposition of a transmission system. They compared optimal zone allocation for cases with and without ESS and concluded that storage facilities change absolute value and direction of transmission flows and zonal prices and can therefore greatly influence the optimal number of zones and their boundaries. Mohsenian-Rad [32] investigated the impact of many other parameters, such as seasonality, storage efficiency, charging and discharging rates and battery life on the ESS profit. Nasrolahpour et al. [86] considered the impact of ramping constraints of conventional generators on ESS operation and concluded that profit of an ESS is higher in systems with less flexible resources. Similar conclusions were reached by Poncelet et al. [13], who analysed relevance of flexibility constraints in unit commitment for planning models.

Most of the papers on the transmission-level ESS consider participation in energy-only markets (see Figure 4), both on the day-ahead and the real-time scale. However, the body of literature that considered ancillary services and capacity markets is expanding due to a general conclusion that an ESS can hardly be profitable by trading only energy and the investors should consider participation in different markets to maximise their profits [8,16]. Table 1 contains information on markets that are considered for ESS participation in the papers reviewed for this study.

ESS participation in capacity markets is generally constrained by its energy rating. Therefore, most of the existing capacity markets which allow for ESS participation have mechanisms to prevent ESS bidding more capacity than they can deploy. Opathella et al. [70] defined the capacity market for the ESS participation by assigning a capacity factor between zero and one to each market participant. The factor is zero for the ESS if at the considered moment the expected energy supply is greater than the demand, and if the demand is greater than the supply, the factor is calculated by dividing the necessary capacity by the available capacity of the ESS, capped off at 1.

3.2.2. Distribution System-Level ESS

Although electricity markets that allow participation of distribution-level resources are uncommon, an ESS at this level has many opportunities to ensure profit, both at retail and wholesale levels. Moreira et al. [35] analysed services ESS can provide to the distribution grid and the possible conflicts between them. Babacan et al. [87] modelled distribution-level ESS participating in the time-of-use pricing retail market, and Jiang et al. [88] investigated the possibility of load shaping through dynamic pricing. Gantz et al. [30] and Tushar et al. [89] considered a shared ESS, also called cloud energy storage, and optimal ways to divide its capacity among the users. Atzeni et al. [90] considered using behind-the-meter ESS and distributed generators to facilitate demand side management, objective being cost minimisation for each individual consumer. The result of this consumer behaviour is a flatter load curve, which shows that even selfish actors can be beneficial to the system if the goal is congestion relief or peak shaving. Gil-González et al. [91] showed that using an ESS in a distribution grid with high penetration of RES reduces its operating costs. Nazir et al. [40] explored the possibility of using ESS for loss minimisation in unbalanced distribution grids with high penetration of PV.

An ESS placed within the distribution grid can participate in the wholesale markets for energy and ancillary services through aggregators. Parvania et al. [63] investigated how decentralised ESSs scheduled by an aggregator influence the transmission grid and concluded that this way of scheduling might be prone to rescheduling after the power flows through the network are realised. Contreras-Ocaña et al. [78] modelled interactions between an aggregator and ESS units under its control and between an aggregator and a wholesale market. The interaction between the aggregator and the ESS was modelled as a Nash bargaining game. Their results showed that rational aggregator is always beneficial to the system but the same is not true for a strategic aggregator. Therefore, they developed pricing schemes to prevent the aggregator from exercising market power. Mortaz [92] demonstrated the impact of geographical diversity on an ESS aggregator’s profit through a risk-measure: more diverse portfolios are almost always more efficient in handling the RES production. Wang and Kirschen [93] presented a two-stage model of an aggregator enabling trade between commercial consumer-owned ESS and day-ahead and real-time markets.

An ESS in the distribution system can participate in markets as a part of a microgrid or a virtual power plant. In such models, the aim is to optimise the operation of the system by utilising the ESS potential for energy arbitrage. Pandžić et al. [94] considered an ESS as a part of a virtual power plant offering in the day-ahead and balancing markets. Giuntoli and Poli [95] modelled a virtual power plant consisting of distributed generators, ESS and loads. They took into account grid locations of said resources but used a DC network model, an approximation which works well for transmission-level models, but is not very accurate for distribution-level ones. Ju et al. [96] investigated how participation of a virtual power plant containing ESS, distributed generators and loads in different types of demand response programmes, can benefit the grid. Their results showed that, while the incentive-based demand response has greater influence on the demand curve, it is most beneficial for the grid to introduce both the price and incentive-based programmes simultaneously. Ciftci et al. [47] modelled ESS used for load following and energy arbitrage within a microgrid with the objective of energy cost minimisation. Alvarez et al. [18] compared behaviour of a virtual power plant with and without and ESS and showed that ESS does increase the VPP’s profit.Liu et al. [97] optimised a cloud ESS in a microgrid and define service pricing mechanisms for it. The microgrid in this paper contains households with rooftop PVs and a cloud energy storage that trades with them.

Table 2 presents the literature review of the distribution-level ESS. It shows that distribution-level ESS mostly participate in energy-only markets.

3.3. Strategic Market Participation

Two most known types of market competition are Cournot’s and Bertrand’s models. Cournot competition is signified by quantity bids and horizontal shifts of supply functions. On the contrary, Bertrand competition is signified by price bids and vertical shifts of supply functions. While Bertrand competition guarantees social welfare maximisation because no-one is motivated to bid more than their marginal cost, Cournot competition can lead to an equilibrium in which social welfare is not at the optimum. However, it was proven that for markets with a large number of competing players Cournot and Bertrand optima are equal [98].

Supply functions are the middle ground between the two extreme cases of competition. Supply function bids are the most common way the electricity markets are organised. Depending on the steepness of the supply function, the behaviour of the market participants can be said to follow the Cournot model (steep) or the Bertrand model (flat). Most electricity market participants follow a mixed strategy, biding their marginal cost to achieve Bertrand optimum and withholding capacity to exercise market power in Cournot context. This was confirmed by Lundin and Tangerås [99], who categorised the behaviour of suppliers in the Nordic power market as Cournot competition because of the horizontal shifts of supply functions between time-periods. Similar behaviour can be observed in the Alberta market by Shafiee et al. [100]. However, conclusions in [99] were based on the the changes in participants’ behaviour between the day-ahead and intraday markets, thereby concluding that the increase in prices between the markets is a result of capacity withholding when it can easily be a consequence of capacity shortage.

ESS can arbitrarily behave as producers or consumers. Therefore, they are not simple market participants and can change market conditions in unexpected ways. Nasrolahpour et al. [72] and Sioshansi [73] compared the behaviour of an ESS as a strategic and non-strategic agent. They showed that adding a new ESS to an imperfectly competitive market can decrease social welfare, which is the opposite of what is expected in Cournot’s model of competition, where an increase of the social welfare is expected when new participant joins the market.

As ESS do not have fuel costs which mostly comprise marginal costs of generators, the optimal Bertrand strategy for an ESS is to bid zero price for charging and market cap for discharging [66]. Nonetheless, it was shown that an ESS which strategically chooses prices and quantities can influence market price and increase its profit. Nasrolahpour et al. [86] proposed a bi-level model of an ESS that strategically sets prices and volumes in the upper level. They showed that the ESS utilising this strategy increased the market price while discharging and decreased it while charging, thereby maximising its own profit. A similar model was used by Wang et al. [76] to show that strategic ESS influence LMPs and therefore increase profit. A supply competition curve was modelled by Krishnamurthy et al. [50] as a price–quantity strategy for an ESS performing inter-temporal arbitrage. They showed that profits of an ESS utilising this strategy are higher on average than of an ESS bidding only quantities. Fang et al. [101] showed that an ESS strategically choosing when to charge or discharge chooses quantities in a way that does not alter LMPs. Shafiee et al. [100] noted the importance of assessing the impacts of ESS on prices during an economic analysis to avoid behaviour which causes ESS to be less profitable when behaving as a price-maker than as a price-taker. Zou et al. [102] found Nash–Cournot strategy for an ESS supporting large-scale RES. With the assumption that RES tend to increase the difference between peak and valley prices and strategic ESS tend to exploit this difference, they concluded that strategically behaving ESS can provide flexibility for the RES if they are driven by selfish objectives.

All these findings are true if only one strategic participant is assumed to exist in the market. In most of the literature only one strategic player is modelled and the other market participants are assumed to be behaving competitively. Pandžić et al. [66] modelled competition between three independent merchant-owned storage facilities via the diagonalisation algorithm. They showed that profitability of a strategic player significantly decreases when strategic behaviour of other market participants is neglected.

Table 3 presents a concise overview of the strategic ESS models. While strategic operation is mostly investigated for energy-only markets, there are several examples of ESS behaving strategically in ancillary service markets as well.

3.3.1. Bi-Level Models

The most common way of modelling strategic market participation is through a bi-level structure shown in Figure 5. This is a practical way of modelling a Stackelberg game. Stackelberg game is a strategic game with a leader and followers where the leader takes the first move to which the followers respond by optimising their position within the given circumstances. Therefore, multiple lower-level problems can appear, as in [86], where there is a lower-level problem for the day-ahead market and an additional one for each stochastic scenario in the real-time market. There can also be more than two levels in a model, as in [92] where the three levels optimise the positions of: (1) an aggregator, (2) an ESS owner and (3) a day-ahead electricity market operator.

A strategic ESS is represented by the upper-level profit maximisation model, while the lower level represents the market clearing process. The lower-level problem is included in the upper level as a set of constraints. Decisions from the upper level can be included in the lower-level problem as parameters. The following is a general mathematical representation of a bi-level problem, where the upper level minimises function $F(x,y)$ of the upper-level (x) and lower-level (y) decision variables, subject to the set of constraints represented by function $G(x,y)$; and the lower level minimises function $H\left(y\right)$ of lower-level decision variables, subject to set of constraints represented by the function $I\left(y\right)$:

$$\begin{array}{cc}& \underset{x,y}{min}\mathrm{F}(x,y)\hfill \\ & \mathrm{subject}\mathrm{to}:G(x,y)\ge 0\hfill \\ & \{\underset{y}{min}H\left(y\right)\hfill \\ & \mathrm{subject}\mathrm{to}:I\left(y\right)\ge 0\}\hfill \end{array}$$

(18)

To be able to solve this problem by a linear programming method, e.g., the simplex algorithm, the bi-level problem is transformed into a mathematical problem with equilibrium constraints (MPEC) using the primal-dual transformation and the strong duality constraint or Karush-Kuhn-Tucker (KKT) conditions. The resulting non-linear program is linearised using the KKT conditions and other linearisation techniques, such as Fortuny–Amat substitution [108]. The final program is solved by linear programming or mixed integer programming solvers, depending on the structure of the upper-level problem. The prices in these models are the dual variables of the lower-level power balance equations. To be able to apply KKT conditions, the lower-level problem must be convex. Having an ESS modelled using binary variables in the lower level is an issue because these types of models are non-convex. This issue is avoided either by modelling the ESS without binary variables [74] or by using some decomposition technique to solve the problem.

One way to model a price-maker ESS is to place the full ESS model defined in Equations (1)–(5) in the upper level. Some papers in which this approach is used are [17,27,72,73,76,86,101,104,106]. A different approach is to include in the upper level only the objective function and an auxiliary variable that represents a strategic decision on bidding. This is the approach taken by Ye et al. [104], who concluded that while the strategic bidding increases the ESS profit, it decreases the social welfare. Another conclusion drawn in this paper is that the higher power rating of an ESS causes more strategic behaviour, and the higher energy rating causes more competitive behaviour. Ye et al. [74] used the same formulation to model a strategic generator in a bi-level setting and to investigate the impact of a price-taker ESS on the generator’s market power. The conclusion was drawn that the ESS reduce the market power during peak and increase it during off-peak hours.

3.3.2. Price Quota Curves

A second approach to the strategic market participation modelling is the definition of the price quota curves (PQC), used by Shafiee et al. [100] and Shafiee et al. [107]. For the generators, the PQC is a step-wise decreasing function that shows dependence of the market price on the generated power, and for the demand it is a step-wise increasing function of the market price dependence on the power consumption. A PQC shows how the market price increases with the growing demand and decreases with the growing generation. In these models, the prices are variables, functions of the charging and discharging quantities. Baslis and Bakirtzis [65] used PQCs to model a strategic pumped-hydro plant that withholds capacity in peak hours to induce price spikes and discharges in off-peak hours, thereby increasing its profit.

3.3.3. Strategic Price-Setting

A third approach to modelling the strategic market participation is to predict prices from the historical data and place bids in a way that would maximise the profit. This results in an unusual strategy for electricity markets, bidding maximum available capacity at all times and choosing prices strategically. This is the approach taken by Thatte et al. [59], Sousa et al. [103] and Thatte et al. [42]. The major deficiency of this way of modelling the ESS behaviour in the markets is the limited accuracy of predictions.

4. ESS Investment Modelling

Capacity expansion planning is used to ensure that the system has enough resources to supply the demand at all times. While in the past, capacity planning mostly considered investments in transmission lines and generators, recent research in this area has taken ESS, demand response programmes, and other new technologies into account as well.

When planning investments in ESS, capital costs are generally separated into costs for energy and costs for power rating. Intuitively, this can be explained by pumped hydropower station investments where dam construction is considered energy cost while costs incurred by turbine and generator installation are power rating costs. On the other hand, when it comes to stationary battery storage, the costs of batteries themselves are energy costs, while the cost of a bidirectional AC/DC converter is reflected in power costs.

The objective of the model depends on the intended usage of the ESS. If the goal of the investor is performing price arbitrage between different time-scales or markets, the objective is profit maximisation and there are no budget constraints [72]. If, on the other hand, it is supposed to be used as a system asset, e.g., for congestion relief or load shifting, the objective is usually cost minimisation and one of the constraints is the total budget [109]. Dvorkin et al. [110] used a slightly different approach, constraining profit from below. They showed that, to ensure profitability of the ESS, this lower boundary should be set to the value of the investment cost.

There are two basic types of energy storage investment decisions: siting and sizing. Siting refers to the decisions on the optimal ESS placement within a grid, while sizing refers to the decisions on its power and energy ratings. These decisions are modelled as continuous variables for the continuous decisions or as binary variables for the yes/no decisions. Examples of continuous storage investment decisions include [14,111,112,113,114,115]. Storage investment decisions modelled as binary variables can be found in [109,116,117].

Siting decisions were made flexible by Kim and Dvorkin [36], where the influence of the mobile energy storage on a distribution grid was investigated. Models of mobile ESS include models of the transport systems used to move the storage around, such as the railway in Sun et al. [118].

Sizing-only decisions were considered by Nasrolahpour et al. [112] and Qiu et al. [28]. This is in accordance with the conclusion made by Lorente et al. [6], stating that siting is more critical than sizing due to complexity of the siting models. On the other hand, Zhao et al. [119], Chakraborty et al. [120] and Pandžić [121] considered only sizing decisions at a premise of a consumer as the placement of the ESS in these models is behind the meter.

4.1. Strategic ESS Investments

Until now, only a few papers have considered strategic ESS investment decisions. As in operational models, these decisions are modelled as bi-level mathematical programs. Nasrolahpour et al. [112] compared results of strategic and perfectly competitive ESS sizing decisions to show how said decisions depend on the chosen strategy. They came to conclusions similar to those of Ye et al. [104] about energy and power ratings of installed ESS. A strategic investor will choose a facility with higher power, and a competitive one will choose one with a higher energy rating.

Dvorkin et al. [113] investigated the impact of transmission line investments on profitability of a strategic ESS investor, noting that investments in transmission lines reduce a number of opportunities for the ESS. Pandžić et al. [122] similarly investigated coordinated investments in transmission lines and energy storage but they gave the system operator a choice between investments in lines or storage. The system operator was placed in the upper level, where it can anticipate decisions of the strategic ESS investor who was placed in the middle level of the model. The lower level presented market clearing process. Their results showed that the system operator favours transmission line over ESS investments even for low ESS investment costs. A strategic investor invests more in storage than the system operator because it can ensure profits more securely by actively participating in the markets, which is forbidden for the system operator-owned ESS. Huang et al. [123] analysed the same situation if the system operator and ESS investor switch places so that the ESS investor is in the upper level and the system operator is in the middle level. The strategic investor takes an even bigger share of the market in this configuration. They showed that system operator-owned ESS has no profit for any investment cost scenario, which is in accordance with the assumption that the system operator uses its storage in the same way as transmission lines, for social welfare maximisation, and not turning profit.

The impact of optimal allocation of both strategically and non-strategically behaving ESS on price volatility in energy-only markets was investigated by Masoumzadeh et al. [124]. They showed that, although an ESS is able to decrease price volatility, it does not remove it completely, and the positive impact stops when the ESS reaches the profitability limit. Next, they showed that a non-strategic ESS has larger influence on the price volatility than a strategic one, which is explained by the former’s social welfare maximisation objective, opposed to the latter’s profit maximisation.

4.2. Transmission System-Level ESS

Investments in ESS can be planned independently or in coordination with other technologies. The technologies used for coordinated approaches depend largely on the applications of the ESS. At the transmission system level, besides the stand-alone approach, ESS planning has been investigated in coordination with transmission networks, generators and renewable energy sources.

Table 4. Literature survey on the ESS investment planning models at the transmission level.

Modelling Technique	Network Unconstrained	Network Constrained	Stages
Deterministic		[125]	multiple
	[129]	[12,126,127,128]	two
		[110,111,113,115,122,123,130,131,132,133]	single
Stochastic		[28]	multiple
	[112]	[14,114,134,135]	two
		[124,136,137]	single
Robust		[116]	multiple
		[117,134,138]	two
Chance-constrained		[139]	two

presents various modelling approaches for ESS at the transmission system level. There are not many models without network constraints, which can be explained by the fact that the ESS exploits network congestions and LMPs to achieve profit. However, it must be noted that not all of these models include siting decisions.

4.2.1. Investments in Standalone ESS

Pandžić et al. [125] considered ESS for minimisation of operating costs within a unit commitment model to determine the necessity for ESS at each bus. The model in this paper is divided into three stages: first the siting decisions are made; then sizing; and last is the operation stage. Dvorkin et al. [110] also considered ESS for congestion management and other transmission services, but they modelled the investments as a bi-level problem, just like Pandžić et al. [111]. Here, a bi-level formulation was used to model the relations of the siting and sizing decisions and LMPs from the perspective of a merchant trading off between energy and reserve markets. Hemmati et al. [14] modelled three time-scales of ESS operation: daily, weekly and seasonal, as three levels within the planning model.

Several private investors in distributed ESS were considered by Saber et al. [139]. The investors were differentiated by their treatment of risk and their zone within the transmission grid. The model structured this way can also represent one investor that treats risk differently in different bidding areas of a zonally structured market.

Zheng et al. [128] studied optimal ESS allocation within transmission system. Unlike most studies on the transmission system that consider only DC power flows, in this paper AC power flows were adopted. The model was of a bi-level structure where the upper level represented siting and sizing decisions and the lower level operational phase.

4.2.2. Coordinated Investments in ESS and Transmission System Assets

While the transmission lines have longer lifetimes, they are more costly and take longer to build than most ESS technologies. For this reason, an ESS is often considered as a substitute or support for the transmission lines. Hu et al. [131] showed that deploying ESS can reduce transmission grid investment costs and MacRae et al. [133] demonstrated that an ESS can be used to postpone investments in transmission system. Aguado et al. [137] showed that the net social welfare increases when ESS are included in transmission expansion planning. The relationship between the transmission lines and ESS expansion was further investigated by Bustos et al. [127], who concluded that the complementarity of the two depends on many system parameters, such as nodal demand, generation capacity, congestion and prices. A three-level model was proposed by Zhang and Conejo [116] for coordinated investments in transmission lines and ESS. The first level determined the investment decisions, while the second and the third modelled long-term and short-term operations. Nikoobakht and Aghaei [134] proposed a continuous-time model for coordinated planning of ESS and transmission network in order to better capture intermittent RES variability, and concluded that the proposed model utilises ESS in the operational phase better. The lowermost level constraints of the three-level problem used by García-Cerezo et al. [117] to model coordinated transmission and ESS expansion planning contained binary variables. Such a problem cannot be solved by standard methods, i.e., KKT optimality conditions, so the authors proposed a nested column-and-constraint generation algorithm. They solved the model with and without binary variables and showed that simultaneous charging and discharging occurs if binary variables are not used.

4.2.3. Coordinated Investments in ESS and Generators

Joint optimisation of generators, transmission lines and ESS was considered by Carrión et al. [135] where the ESS provision of frequency response is included in the planning stage. Wu et al. [129] considered investment planning for generators and pumped-hydro storage constrained by the low-carbon requirements of the system. Opathella et al. [126] analysed the influence of generator contingencies in a long-term planning model. Tejada-Arango et al. [12] proposed a change of approach for unit commitment used in planning models from energy-based to power-based. This new approach takes into account more granular data inputs in form of power demanded at each moment instead of hourly energy demand. The results show that this approach results in lower total investment and operating costs.

4.2.4. Coordinated Investments in ESS and RES

Energy storage was considered useful for incorporating intermittent wind production in the power system by Xiong and Singh [114] because it reduces daily operating costs by reducing wind spillage for high wind production scenarios and prevents load curtailment for low wind production scenarios. Fernández-Blanco et al. [136] also considered using ESS to reduce renewable energy spillage and tested the sensitivity of siting and sizing decisions on various model parameters, such as penalties for renewable spillage, marginal costs of conventional generators and maximum energy rating. They showed that ESS does not always reduce renewable spillage if this reduction is not a part of the objective function.

4.3. Distribution Level ESS

We consider distribution-level ESS all those connected to the low or medium voltage network. Just like with transmission-level ESS investments, the approach to investment planning depends on the intended usage of the ESS. Even when specific ESS purposes are considered, general decisions can be drawn. Hajia et al. [29] considered joint expansion planning of distributed generators and ESS. The nonlinear model was solved by various heuristic algorithms. They showed that the energy arbitrage opportunity is the most important factor in ESS sizing.

Table 5. Literature survey on the ESS investment planning models at the distribution level.

Modelling Technique	Network Unconstrained	Network Constrained	Stages
Deterministic	[140]	[141]	multiple
	[119,120]		two
	[121,142,143,144]	[145,146,147]	single
Stochastic		[109,148,149,150,151,152]	multiple
	[153]	[36]	two
	[121,154]	[34,37]	single
Robust	[121]		single
Chance-constrained	[155]		multiple
		[156,157]	single

presents different modelling approaches to distribution-level ESS investment planning.

4.3.1. Coordinated Investments in ESS and Distribution System Assets

Xing et al. [151] researched expansion planning of a distribution grid already containing distributed ESS, but did not consider ESS investments. Similarly, Saboori et al. [141] used a multi-stage planning model to investigate the impact of ESS on distribution grid expansion. Installation of ESS is shown to decrease the number of new lines. It was also shown that ESS have positive impact on grid voltages and reduce congestions. These benefits are increased with the size of ESS. Quevedo et al. [109] considered the impact of electric vehicles on distribution grid expansion planning, showing that additional demand from electric vehicles can incur high costs for system operators by causing need for network expansion, which can be avoided by installing stationary ESS. Besides ESS, the authors in this paper considered installation of distributed generators, substations, transformers and electric vehicle charging stations. Hassan and Dvorkin [146] investigated how ESS capital costs, distribution grid line ratings, penetration of photovoltaics within the distribution grid and placement of wind power plants within the transmission grid influence ESS siting and sizing decisions in a distribution grid operated in coordination with transmission grid. The coordination was represented as a bi-level model wherein the transmission system operation was in the upper level and distribution system operation in the lower level. They showed that, if only one-way power flow is allowed between the transmission and distribution systems, placement of RES within the transmission grid does not influence investments in distributed ESS.

Joint expansion planning of energy storage and the distribution grid was modelled by Shen et al. [150], Akhavan-Hejazi and Mohsenian-Rad [156] and Iria et al. [38]. Shen et al. [150] showed that a distribution grid relies on ESS for peak shaving and reliability enhancement. Akhavan-Hejazi and Mohsenian-Rad [156] took the research a step further by modelling both real and reactive power flows and considering ESS for voltage compensation within an active distribution network. Voltage regulation was also considered by Das et al. [147] who took into account both real and reactive power injection by ESS. Iria et al. [38] considered investments in ESS and on-load tap changer transformers in a distribution network with high penetration of renewable generation. The installation of the new network assets was performed for the purposes of congestion and voltage problem mitigation.

Recently, idea of active distribution networks has gotten very popular. These networks include controllable distributed resources such as generators and ESS. Compared to traditional distribution networks, active ones require changes in the planning approach. Nick et al. [34] investigated optimal allocation of distributed ESS in active distribution networks for various purposes: voltage control, congestion management, network loss and load curtailment minimisation. This work was taken a step further by Nick et al. [39] by incorporating grid reconfiguration possibility in the model. Active distribution network planning was further investigated by Kim and Dvorkin [36] and Abdeltawab and Mohamed [37], who researched the possibility of using mobile ESS for enhancing distribution grid stability, especially through voltage control. Li et al. [152] used a three-level structure to model coordinated investments in active distribution grid, RES and ESS. The upper level presented network structure planning, the middle level was the allocation of RES and ESS and the lower level was system operation model. Sekhavatmanesh and Cherkaoui [158] developed a method for using the ESS to ensure fast grid restoration of active distribution networks, a service similar to black start in transmission networks.

4.3.2. Investments in ESS for RES Integration

In order to accommodate large shares of RES, investment models for coordinated or stand-alone RES and ESS are considered. Santos et al. [148] and Santos et al. [149] is a two-part paper dealing with different RES-enabling technologies placed at the distribution grid level. Zhang et al. [157] investigated optimal ESS allocation in distribution grids with high wind power penetration. Their model contained wind curtailment cost in the objective function and minimum wind utilisation constraint. Their results showed that if more wind utilisation is required, a larger ESS needs to be installed. Xiao et al. [145] built a siting and sizing model for distributed energy storage systems in distribution grid to accommodate distributed RES. The grid was represented by an AC model which requires a solving approach able to handle non-linear models. The genetic algorithm was used in this case.

4.3.3. ESS in Aggregators’ Investment Models

In most planning models that consider ESS interactions with an aggregator, ESS siting and sizing is not considered. Exceptions are the shared storage models. Zhao et al. [119] modelled an aggregator operating a shared storage aiming to maximise its profit. This model improves the utilisation of energy storage which means that the aggregator can invest in a smaller facility and still serve same number of customers. Shared storage optimisation was the objective in Chakraborty et al. [120] as well, where the cost sharing was modelled as a coalition game.

4.3.4. Investments in ESS in Microgrids and Vpps

Considering organisation, microgrids and VPPs are similar structures. Both structures are optimally run by a central operator with the aim of minimising operating costs or maximising profits. The difference between the two is structural—while a microgrid is a small portion of a distribution grid connected to the main grid through one point of common coupling, a VPP is spatially distributed set of resources. Yang and Nehorai [140] considered investments in a hybrid generator-storage facility within an islanded micro-grid at different geographical locations. Their results showed that the type of climate at the location influences selection of installed technologies. The planning problem in Khodaei et al. [159] focused on microgrids with distributed resources and energy storage and assessed the possibility of islanded operation. It was shown that, for shorter expected durations of the islanded operation, installation of an ESS is not justified. Cao et al. [155] presented a model for multi-stage microgrid expansion planning for stand-alone microgrids. Jacob et al. [144] proposed a sizing method for ESS within a microgrid with photovoltaic (PV) production based on design space approach. The method considered time-scale classification of the ESS and variability of PV output. Different time-scale storage requirements were met with different technologies: fuel cells and flow batteries for long-term, lithium and lead acid batteries for mid-term and flywheels and supercapacitors for short-term storage.

4.3.5. Behind-The-Meter ESS Investments

In behind-the-meter applications of ESS, the objective is usually minimisation of electricity cost. These costs can be a result of wholesale market participation, incentives paid by the system operators or participation in demand response programmes. These models can include RES and controllable loads as well. Sharma et al. [143] modelled a nearly zero-energy residential home with a PV system. Necessary power and energy ratings of the ESS were calculated for each time interval by subtracting the load from the produced power. Optimal ESS capacity was then determined by minimising overall costs heuristically by using the genetic algorithm and two other minimisation functions provided in MATLAB. Although this approach might lead to a local optimum, all three techniques converged to the same solution, so the authors concluded that they reached the global optimum. They showed that after the installation of ESS, energy exchanged with the grid decreased significantly. Zhu et al. [142] presented a method for sizing ESS within distribution grid with high PV penetration and tested the method on three types of ESS: behind-the-meter, utility owned and merchant owned. They showed that it is more economical for the DSO to procure services from the latter two types of ESS. They also showed that existence of demand side management reduces the size of installed ESS. Pandžić [121] optimised investments in ESS for a hotel participating in a two-tariff retail market. Bayram et al. [154] developed an analytical method for sizing a shared ESS used by consumers to achieve various benefits, such as improved power quality and cost reduction, by participating in demand response programs and avoiding peak power charges. Wang and He [153] presented a model for making optimal decisions on behind-the-meter ESS installation and demand response programme participation. The model was suitable for commercial consumers and the paper presented two case studies for a smaller and a larger consumer. Depending on the distributed generator’s capacity, the model makes different decisions on ESS size and participation in a demand response scheme. They showed that for low electricity prices no ESS is installed.

5. Computational Complexity of ESS Models

The computational complexity of a model depends on its various properties, such as the number and types of variables, the number of time periods and whether or not uncertainties are considered.

Table 6. Types of mathematical models used in the considered literature. (Abbreviations: LP, linear programming; MILP, mixed-integer linear programming; SOCP, second order cone programming).

Operation	Planning	Solving Technique
[20,22,23,30,43,46,49,50,51,53,54,57,59,73,74,78,88,89,90,105]	[119,125,130,156]	LP
[1,16,17,18,20,25,32,33,35,44,47,50,52,56,57,58,60,61,63,65,66,70,71,72,76,79,80,81,82,83,86,92,94,95,96,100,101,102,104,106,107]	[14,28,109,110,111,112,113,114,115,117,121,122,126,127,131,133,134,135,136,137,139,148,149,155,156,159]	MILP
[40,91]	[34,36,38,39,116,146,151,158]	SOCP

shows three main types of models encountered during this literature review. Linear programming (LP) is the simplest type of models, as it contains only continuous variables and therefore can usually be solved by algorithms like Simplex. In using these types of models for ESS modelling there is a risk of simultaneous charging and discharging in certain cases, as discussed in Section 4.3.1. Mixed-integer linear programming (MILP) models contain integer variables. In the case of ESS modelling, these are usually binary variables used to prohibit simultaneous charging and discharging of the ESS or investment indicators. In the next level of complexity, second order cone programs (SOCP), are used for modelling AC power flows. They are being used more and more for modelling ESS operation within distribution grids. These three types of models are not the only ones used in the literature. For example, Shafiee et al. [45] used a mixed-integer non-linear model, Huang et al. [123] used a mixed-integer quadratic programming model and Abdeltawab and Mohamed [37] used a mixed-integer convex programming model.

The main ways to deal with intractability issues in ESS models involve choosing the right modelling and solving techniques. The intractability of models is caused by large numbers of input parameters. Although many clustering techniques were developed to deal with large numbers of input parameters in power system modelling, they are not appropriate for ESS modelling. Intertemporal constraints, especially for long-term energy storage, cause large errors in models where clustering techniques are applied. For this reason there are not as many papers dealing with long-term storage operation as there are with short-term operation in the intraday and day-ahead markets (Figure 6). However, ESS-friendly clustering techniques are being developed. To reduce computational burden of planning models, Pineda and Morales [130] proposed a chronological time-period clustering different from the standard hours-days-weeks clustering. The proposed technique separates longer time periods into clusters in which smallest time-period is one hour and only consecutive and similar clusters can be merged. Tejada-Arango et al. [132] also presented clustering techniques for decreasing computational burden of the planning models. They proposed improved system states and representative periods clustering techniques and showed that they do indeed shorten the computation time.

5.1. Modelling Techniques

Power system planning solves investment problems by taking into account technical and economic constraints, and it has always been concerned with uncertainties. For a long time, the main sources of uncertainty in the planning models were load growth and production of hydro-power plants. Nowadays, the largest source of uncertainties is the production of intermittent resources, i.e., wind and solar power plants. This is evident from Figure 7, which shows that, while load is the dominant source of uncertainties in planning models, price is considered to be uncertain more often in operational than planning models. RES as a source of uncertainty is commonly considered in both the operational and the planning models. Label “none” in Figure 7 signifies the share of deterministic models where no uncertainties were considered.

There are several ways to include uncertainties in a mathematical model. The three main types of mathematical programming methods for dealing with uncertainties are robust, stochastic and chance-constrained optimisation. Comparisons of the methods were performed with general conclusions that stochastic optimisation yields larger problems and takes longer to solve than the other methods. It was also concluded that robust optimisation gives good results for border-line cases, but for most cases it is better to use stochastic optimisation. It was shown by Khodayar and Shahidehpour [57], who compared the results from deterministic and stochastic problems, that neglecting uncertainties generally results in higher expected profits. The uncertainties in the models stem from the stochastic nature of RES output, load levels and market prices.

Besides mathematical programming methods, the approach of machine learning for dealing with uncertainties in computationally tractable way is gaining momentum in recent years. Until now, it was applied only to a few ESS modelling problems. Machine learning techniques are mostly used to forecast values of the uncertain parameters such as wind power by Varkani et al. [62]. Another useful application of machine learning algorithms is as a heuristic used to solve complex stochastic programming models. For example, Yuan et al. [55] used a neural network with genetic algorithm to solve a large stochastic problem of coordinating wind power plant and ESS.

However, reinforcement learning can be used to model market-participating ESS. This approach takes on the uncertainties by "teaching" the model to respond to them, instead of modelling them mathematically. The technique that has gotten the most attention from the power systems modelling community is Q-learning. Q-learning is a model-less reinforcement learning algorithm that describes the behaviour of a model by a Markov decision process with actions to which rewards are assigned and the goal is to maximise cumulative reward. This algorithm was used by Wang and Zhang [46] to model market participation of a strategic ESS. Although machine learning algorithms are widely used for data fitting and forecasting, Wang and Zhang [46] were the first ones to use this approach to model the ESS market participation. Q-learning was also used by Ye et al. [160] to model the strategic market participation of a generator.

The biggest flaw of this approach is that the model trained on specific data set learns to behave only in the circumstances described by the data. This can be problematic in a changing environment, e.g., a market in which new players are appearing.

5.2. Solving Techniques

Based on the reviewed literature, the general conclusion is that solving mathematical models with uncertainties is quite complex. This is especially the case with multi-stage models, which can be intractable even for small number of stages. The complexity of the models with ESS is even greater because of the intertemporal constraints and the necessity for the inclusion of binary variables. Many different solving techniques were developed with an aim of simplifying this process. Some of them are:

• Heuristic algorithms [55,128,143,147];
• Decomposition techniques [43,105,122], especially Benders decomposition [22,57,60,106,112,134] and column-and-constraint generation [92,113];
• Dynamic programming [16,18,27,54,161].

Some of these techniques are based on duality theory, and are therefore unsuitable for solving models with binary variables, common in ESS modelling. For a detailed analysis of solving techniques used for complex planning problems with ESS, an interested reader should refer to Lorente et al. [6] and Zidar et al. [8].

6. Conclusions

The scope of the literature dealing with ESS is vast, which indicates that a significant number of issues have already been solved in both the operation and the investment planning models including ESS. However, the potential for improvements in model development exists in both areas. New use cases for ESS need to be created, especially for distribution-level ESS to ensure reliable grid operation in the context of active network design, distributed generation and demand response programs. Allowing distributed resources to participate in the wholesale energy and ancillary service markets, and the deployment of innovative local energy markets, should open even more possibilities for ESS. At the transmission level, different ownership structures of ESS (privately or TSO-owned) necessitate different operating strategies so there is room for inventiveness as well.

As a conclusion, some of the specific areas where study of ESS can be expanded are:

• ESS operating in zonal market structure, especially considering uncertainties. While many electricity markets are structured zonally (Europe, Australia), not so many studies on ESS participating in zonal markets were conducted.
• Studies on strategic ESS will have us believe that it is very simple to ensure large profits for investors by employing an aggressive strategy during operational phase. More studies on competition between more than one strategic ESS are therefore necessary to reconcile the results of ESS with general economic theory.
• Strategic investment models, on the other hand, do not deal with bidding decisions and only siting and sizing decisions are strategic in these models. Nonetheless, competition between more than one strategic investor was not yet investigated.
• Market participation of the distribution-level ESS was not widely investigated in the past. The design of distribution-level markets will have a large role in enabling profitability of distributed ESS. Aggregation, local energy markets and peer-to-peer trading between distribution-level ESS must be tested thoroughly before they are implemented in real-world situations.
• Although load, RES and prices have been considered as the main sources of uncertainties so far, other sources are appearing in the models, such as contingencies and EV loads. Although ESS has proven to be of most use in situations where flexibility of the system is jeopardised, their application in mitigating various uncertainty risks should be analysed.
• Tractability of models is a challenge even without ESS. However, long-term operation of ESS does not play well into most clustering techniques used to simplify models with time-spans of a year or longer. Therefore, it is necessary to develop ESS-oriented clustering techniques which would enable including long-term operation in operational and investment models with ESS.