The Methods of Assessing the Efficiency of a Virtual Power Plant—Case Study

Abstract

In the case of new investment in RES technology, there are two issues related to efficiency assessment. The first one is how to join technical, financial and economic efficiency assessments in one. An investment feasibility study is usually conducted through a financial analysis to obtain the internal rate of return and the net present value. However, a new RES investment is typically financially unprofitable unless the environmental and social aspects are included. The second one is a lack of required financial data. The new RES investment is often innovative and neither the owner nor other entities have financial data on the operating costs and expenditure for the last periods. Therefore, in this paper, we proposed two methods of efficiency assessment. The first one is based on the avoided costs theory. Furthermore, the second one belongs to heuristic methods and is based on the experts’ assessment of different kinds of parameters. The purpose of this study is to assess the efficiency of the pilot project of VPPs using two recommended methods. This paper emphasizes the advantages and disadvantages of each method. The actual technical and financial data for the period of six months from the pilot study were calculated.

1. Introduction

In recent years, documents in the area of climate protection, decarbonization and climate neutrality have been of particular importance in EU legislation. The most recent and relevant document is the ‘Fit for 55′ package that was announced in July 2021, which will streamline actions to achieve a significant reduction in greenhouse gas emissions of 55% by 2030 [1]. The new proposed EU energy targets for 2030, informally agreed on March 2023, include the following: further increasing the share of renewable energy in energy consumption up to 45% and further reducing primary and final energy consumption in the EU. The consequences of the climate and energy policy goals set at the European level are the introduction of national laws and regulations aimed at supporting the development of renewable energy investments, increasing energy efficiency in buildings and processes, as well as triggering new investments in low-emission or less-emission energy and heat sources. Renewable energy technologies, like no other, align with this policy in all aspects.

Along with the development of renewable energy technologies and increasingly stringent regulations on the energy neutrality of economies, there is a need for changes of an organizational nature in local energy markets. Virtual power plants (VPPs) are one such solution. The previous work focused on the technical [2] and economic [3] approach to the VPP, where the economic efficiency of the investigated VPP was calculated by using the net present value (NPV) method, and the break-even point (BEP) was separately identified for different scenarios. Additionally, a sensitivity analysis was conducted. It considered factors independent of the VPP (weather and market price). Finally, the authors assessed and compared the economic efficiency of integrating various users (such as a hydropower plant, rooftop PV, prosumer and an energy storage system (ESS)) into a VPP.

The method used has its shortcomings because it does not comprehensively capture investment efficiency from the perspective of the organization (energy company). It focuses only on financial efficiency. On the other hand, methods for assessing social efficiency include cost–benefit analysis or the ExternE method presented in the European Commission’s program. The methodology of ExternE (External Cost of Energy) considers the external costs of generating electricity from a given fuel (e.g., nuclear, coal) throughout the production cycle. Studies still lack methods for calculating the efficiency of innovative technological solutions holistically, which VPP undoubtedly is.

In response to this demand, this article proposes a new approach for assessing profitability (efficiency). Therefore, the main aim of this paper is to present two methods for assessing VPP efficiency. The first is based on the concept of the so-called avoided costs. Two sources of benefit are included in the calculation: from savings on the hydropower plant’s own needs and from charging the storage at a lower price and discharging at a higher price. The second method is based on scoring by experts. Here, the basis for assessing efficiency is management priorities. Both methods consider the characteristics of such an investment as a VPP.

The commonly used methods of calculating the profitability of an investment have disadvantages and limitations. The use of classical methods such as NPV or avoided costs to calculate the profitability of an investment (such as VPP) leads to conclusions about the unprofitability of this type of investment. This situation is often observed in the case of innovative investments characterized by a high level of risk and uncertainty of financial data. However, there is a widespread conviction that there are environmental or social benefits for the region and future generations associated with the implementation of VPPs. Furthermore, these investments are based on the newest technologies which have not yet been tested on a large scale in economy and they need improvement. More advanced companies, according to these technologies (VPPs), would not share their know–how. In the long run, the results of developing these technologies will be also improve financial efficiency; however, to achieve this state, these technologies should be implemented and studied in the natural (actual) environment first. The analogous situation posits a solution with regard to PV micro-installations or wind turbines and a continuous improvement of their effectiveness. Hence, a combination of the classic method with another investment evaluation method was proposed in order to obtain a more complete picture of the profitability of the investment decision. These two methods are used independently but complement each other. Each is based on different inputs, but both were used to analyze the same case study. In the realm of VPPs, a fresh investment typically lacks financial viability unless it incorporates considerations of environmental and social impact or relies on subsidies. Given our experience of working together with energy company managers, different decision makers need different information. For example, financiers focused on the financial aspects (e.g., NPV, avoided costs), but the technical department would like to know about the technical improvements and the technical feasibility of an investment, and HRM and employees would like to know how the investment affects the number of employees. Considering the specificity of an innovative investment (such as an VPP) and the availability of information, a combination of the avoided cost method with a heuristic method was proposed in order to consider as many factors as possible and obtain a holistic approach for assessing the profitability of the investment. Numerous papers have focused on calculating the economic efficiency of new investments in various RESs, renewables combined together in a hybrid model, and battery storage for smart home or microgrids, but only a few papers focused on VPPs. In the literature, there is still a lack of comprehensive methods for calculating the efficiency of innovative investments in power systems from an organizational perspective. Holistic methods for calculating the effectiveness of innovative technological solutions are still lacking. The approach proposed in this paper is a step in this direction as it combines two methods: (1) the heuristic method, which can be tailored to the type of investment and the objectives of the decision makers, and (2) the avoided cost method, which is usually used in case of missing or poor-quality financial data for a new energy investment. This work fills the gap by proposing to combine the avoided cost method with a heuristic approach to consider as many factors as possible. This aims to obtain a holistic approach for assessing the profitability of an investment.

2. Literature Review

2.1. Efficiency or Profitability

The terms “efficiency” and “benefit” or “profitability” are closely synonymous and sometimes used interchangeably in the literature. The key term in determining the efficiency of any venture, organization or process is the concept of “efficiency.” However, it must be related to the object for which it is determined, such as economy, enterprise, investment, activity or process.

From an economic perspective, efficient techniques (e.g., manufacturing) are defined in terms of resource (factor) allocation. Efficiency itself is explained in two terms. The first one involves seeing it in the context of a “something-for-something”(trade-off) relationship in terms of effects with full use of resources. A graphical illustration is the production capacity curve. Moreover, efficient combinations of resources are those combinations of resources that produce the maximum quantity of one type of goods for a given amount of production of another type of goods. However, with regard to the example under review, this approach is not applicable.

In the second approach, efficient manufacturing techniques are defined as those combinations of resources used that make it possible to produce the same quantity of product (e.g., electricity) with less involvement (consumption) of at least one resource. An illustration of the efficient combinations of resources comprises the points located on the isoquant. This approach could be applied to assess the efficiency of a VPP’s work in economic terms. Thus, according to this economic view, one can say that a technique is more efficient if the consumption of a resource is reduced for the same amount of production. Thus, more efficient production methods save resources (less consumption). Hence, efficiency can be equated with thrift–frugality with no change in the satisfaction level of needs. It is worth noting that saving is not necessarily related to efficient use (reducing the input while maintaining the same effect) but to reducing both the input and the effect (e.g., the level of satisfaction of a need).

Efficiency is most often framed as a comparison of effects to inputs. Such an approach is close to the concept of productivity. If the input and effect are measured in the same unit (e.g., a monetary unit), then efficiency can be measured by the difference of the two quantities. An example of such a measure of efficiency is profit being the difference between revenues and costs or EBIT. The comparison can take the form of a quotient, just like profitability ratios, such as return on equity (ROE) (profit/equity) or electricity produced from 1 ton of coal. Efficiency is one of the basic process parameters in physics or the characterization of technical devices. Also, in management, according to the praxeological approach, efficiency refers to action. It is assumed that action is efficient when it is performed in the right way, i.e., such a way that the effect is adequate to the result [4] (p. 408).

Regarding the estimation of investment efficiency in financial and economic terms, a distinction is made between the so-called static methods, e.g., ROE, and dynamic methods (e.g., NPV), i.e., those that consider the changing value of money over time.

Another approach that differentiates efficiency accounts is its scope. Efficiency can be analyzed from the perspective of a single entity, such as an investor. Then, the most common way is to calculate efficiency in financial terms with such indicators as net present value (NPV), internal rate of return (IRR), return on assets (ROA) and ROE.

However, it is possible to carry out efficiency calculations in macroeconomic terms, answering the question of the efficient allocation of resources in the economy (Pareto efficiency) or considering social and environmental effects. In these cases, for example, the following methods are used: cost–benefit analysis, ecological footprint over the entire life cycle of an investment or good, or the ExternE method, which has already been mentioned. Including social and environmental effects in the calculation increases the economic efficiency of this type of investment compared to financial efficiency. Thus, the need for this type of investment is justified. However, due to their financial unviability, they are subsidized with public funds. Instead, in this type of analysis, one speaks about benefits.

An important issue in determining efficiency is defining the criteria for the efficiency account as well as defining its scope.

From the perspective of the analyzed example, it should be noted that efficiency (more considerations and comparisons of definitions can be found, for example, in [5].) can be expressed by one dimension (e.g., in terms of financial efficiency) [3] or from a more holistic perspective. In the latter case, the methods used to assess efficiency can be, for instance, CBA, an ecological footprint or (from the perspective of a particular organization) a balanced scorecard.

The assessment of efficiency can be carried out as follows:

• Ex ante—as the basis for the decision to launch a given investment;
• In media res (current)—as a basis for deciding whether to continue or cease the investment, which is particularly important in economic terms, so as not to incur the so-called sunk costs;
• Ex post—conducted primarily for control and planning purposes.

What emerges from these short characteristics is that the very concept of efficiency is defined differently and measured using different methods. The choice of the methodology depends on the subject of the efficiency measurement—the subject who is interested in a given analysis, i.e., decision makers and/or beneficiaries. In business practice, access to data is a significant constraint. This is especially true for innovative investments, such as VPPs.

2.2. Profitability of Investment in RESs and VPPs

The RESs are preferred over conventional power systems so the researchers proposed RESs as the sole power generation for commercial and residential sectors in geographic regions where energy infrastructure is not laid down. In addition, numerous works have been performed to prove the feasibility and economic benefits of new investments in renewable energy installations operating independently or integrated in existing grids.

Conclusions on the profitability of investment are most often formulated on the basis of measurements, such as NPV, IRR, payback period (PBP) or levelized cost of energy (LCOE). Numerous papers focus on using NPV to assess the economic efficiency of the investment in PV installations [6,7,8], wind generation [9], various micro-RESs [10,11,12], battery storage for smart home [13], particular renewables combined together in a hybrid model [14,15,16,17] or even microgrids that consist of photovoltaics and an energy storage system [18].

This article is about VPPs, so it is necessary to define the object of the efficiency assessment, i.e., VPPs, which can be defined in two ways. In narrow terms, they are algorithms that control or support the decision-making process of switching the distributed generation capacity to meet the energy needs in real time at the lowest possible cost. In this case, the efficiency of an algorithm’s operation will be equated with improving the ability to control connected resources optimally and minimizing the cost of generating and delivering energy to consumers.

More broadly, VPP is defined not only as an algorithm that optimizes the switching of one of the cheapest sources at a given time, but also as the entire grid infrastructure, including distributed generation capacity. Nowadays, these are most often distributed renewable electricity generation facilities owned by entities external to the VPP owner. In this case, when calculating efficiency, it is necessary to consider the costs of operating individual energy sources or the costs of settlements with their owners. From the investor’s perspective, such efficiency should be calculated analogously to that of infrastructure investment. More definitions of VPPs can be found in [19,20].

It is also essential to consider a feature of the VPP, such as its networking, which can be analyzed in three dimensions. Firstly, the VPP is characterized by the networked nature of the electrical technical infrastructure (the individual sources are connected by electricity grids). Secondly, monitoring and controllability are fully automated due to the possibilities offered by the IT network. Thirdly, the physical entities that constitute the VPP are open (new sources and new customers can be connected) and they enter into relationships with one another, which requires appropriate contracts. They can form a supply chain-type network (cooperative linkage). Moreover, individual entities can compete with one another (VPPs at any given time will supply the sources with the lowest production costs). R. Kucęba [20] believes that the VPP is an organized infrastructure with network connections formed by small power plants together with cooperating both supply side and demand-side market participants and controlled by IT and ICT systems, including artificial intelligence.

In the literature, there are no developed models of business enterprise efficiency for assessing a network structure controlled by ICT systems. The most similar approach is presented in [21] (p. 64), where methods for assessing IT ventures include total cost ownership (TCO), total economic impact (TEI), real options method (ROM), information technology scorecard (ITS), information economics (IE), expected value of information (EVI) and applied information economics (AIE).

Only a few papers focus on the economic efficiency of the VPP [22,23]. In paper [22], the net present value of economic efficiency is assessed from the perspectives of VPP service providers and power companies. In paper [23], the authors compared the economic efficiency of integrating various flexibility technologies (e.g., DER) into a VPP.

An overview of the selected literature on the measurement of VPP efficiency from a managerial decision perspective is presented in

Table 1. Methods and scope of measuring the efficiency of VPPs—literature overview.

Measurement Criterion	The Method Used	Authors
The maximization of a VPP profit and the maximization of both functions energy and exergy efficiency	Optimization model by using a genetic algorithm (GA): the equal weigh multi-objective method	Elgamal, Vahdati and Shahrestani (2022) [24]
Energy management efficiency by using the VPP	An overview of various methods used to optimize the operation of the VPP algorithm, including graphing the operation of individual energy sources, among others: linear programming, learning-based algorithms	Rouzbahani, Karimipour and Lei (2021) [19]
Return of investment in the VPP	Payback period (PBP); net present value (NPV)	Li, Gao and Ruan (2017) [25]
The energy utilization efficiency of the renewable	Game theory, Nash-seeking algorithm, scenarios, comparative analysis	Lin, Chen and Li (2021) [26]
Energy and economic efficiency	Technical efficiency (productivity); profitability index as present value of future cash flow by initial investment	Minutillo, Perna and Sorce (2019) [27]

Source: authors’ work.

.

2.3. Avoided Costs

When traditional methods of calculating the profitability of an investment cannot be used due to the lack of financial data, the avoided cost method may be applicable. This method is well-known and widely used in various fields, including the energy industry. The following describes the examples of how the method can be used to estimate costs in small thermal power plants, hydroelectric power plants, wind power plants, RESs and even battery energy storage. In most cases, avoided costs analysis is used for power enterprises. However, it has also found its way into private power plants.

The authors in paper [28] estimated the avoided costs in the electric utility in Slovenia for different load-shape power-conservation measures (including self-production). Furthermore, the economic effects of the time shifts of the various planned generation units (thermal and hydroelectric power plants) consider the optimal operation of the system over the planned period.

The avoided costs related to the application of wind energy in The Netherlands were investigated in [29]. The avoided costs are calculated based on proposed figures for the future costs of fuel, investment, operation, maintenance and advanced techniques to reduce the emission of NO_x, SO₂ and CO₂. The avoided costs were translated into the break-even costs of wind energy.

There has been controversy in the United States and Canada over the calculation of avoided costs for off-grid electricity generation [30,31,32], and energy enterprises have adopted different interpretations and implementations.

In paper [33], a suitable method was recommended for assessing the long-term avoided costs of a private power plant under uncertain conditions for the power utility. The break-even cost of the private power plant can be treated as a long-term “avoided cost”. The authors showed that the break-even cost is a useful measure in the economic assessment of private power plants.

The avoided costs method was used [34] to calculate a reasonable selling price for the electricity generated by a cogeneration system connected to the electric grid. Based on the proposed cogeneration price function and the avoided costs concerning the loss and upgrade of transmission lines, computer simulations were conducted. The avoided costs method is used worldwide to determine the price of electricity produced from cogeneration.

A feed-in tariff (FIT) is an energy supply policy that supports the development of new renewable power generation. One of the methods of setting FIT payments is by estimating the value of renewable energy. This value can be determined in several ways, for example, according to the avoided costs of the energy company [35].

In the paper [36], the social cost–benefit for hydropower generation is calculated from the avoided costs and the external costs of carbon and other gas emissions. The avoided cost comprised three costs: the electricity production cost of fossil fuel plants, the energy cost or direct variable cost and the capacity cost or direct fixed cost. The authors suggested that in formulating policies to promote hydropower, the government should allocate this social benefit back to electricity users.

The avoided costs method is used in the optimization of the planning and control of the consumer-scale battery energy storage system connected behind the meter [37]. The objective function of the optimization problem is formulated in the form of avoided costs, defined as the difference between the costs of consumed electricity with and without an energy storage system.

2.4. Heuristic Methods

The group of heuristic methods has its origins in antiquity. The Sophists understood heuristics as the art of discussion to detect the truth. These methods are used for creative problem solving, forecasting or prediction. More recently, they have found application in computer science as methods most often combined with machine learning and linked to artificial intelligence. There is rich literature on heuristic methods themselves and their applications, e.g., in [38]. These methods include brainstorming, the Delphi method, Gordon’s synectics, the morphological method, CERMA techniques and the 20–80 method (Pareto’s decomposition) [39].

Heuristic methods are applicable under the conditions of uncertainty and lack of historical data. Therefore, they are suitable for assessing risks and the direction of development of innovative ventures. Many of them use expert knowledge, associations, distributed knowledge and tacit knowledge. Thus, they are categorized as creative methods. Rouzbahani et al. explicitly write that “Heuristic techniques aim to achieve a specific result for a problem that is tested by considerable experiments making a trade-off between the solution accuracy and the computation cost and speed” [19] (p. 7).

In this article, due to the lack of accurate and reliable historical data, a modified Delphi method combined with information economics (IE) was used to value the efficiency of a VPP. In expert methods (e.g., the Delphi method), the criteria for selecting experts are always set individually depending on the research subject and its scope, but the most essential criterion for selecting experts is their competence. In a 2010 study by Japan’s NISTEP, which has been systematically conducting foresight studies since the 1970s, the number of experts with the highest level of knowledge in the most innovative technologies was a few. For example, the thesis entitled “Generating energy from solar power in space and transmitting it to earth” was assessed by eight experts with superior knowledge in the field. In fact, 253 experts assessed the thesis in the first round of surveys but 22 had intermediate knowledge and 73 had a low level of knowledge. A total of 243 experts assessed this thesis in the second round of surveys, of whom 19 had intermediate knowledge and 78 had low knowledge [40]. It is standard in the Delphi method to conduct several rounds of surveys of independent experts, so that the results obtained (mean or median) have an acceptable spread for the researchers. The standard is three rounds. After each round of surveying, each expert receives feedback on the average responses and their spread as well as information on how his/her answer ranks against this. In the next round of surveying, he/she can change his/her previous answer. The assumption is that if the expert is confident in his/her answer, then he/she will be less likely to change it; if not, then he/she is likely to change it. Arriving at a common position among experts is ensured by the Delphi procedure, which is used to average the results while minimizing their spread.

IE is a comprehensive method of assessing the efficiency of IT projects [21,41]. This is one of the multi-criteria methods proposed in 1988 by M. Parker and R. Benson. In this method, the impact of the project on the enterprise is considered holistically and both the business and technology domains are considered. The efficiency of an IT project measured by the IE method is the sum of three types of weighted average, i.e., a simple rate of return, business domain and technology domain. It is essential to establish ranks for these components. A more detailed description can be found in [41]. Ranks have values from 0 to 5. Weights are also established to reflect the organizational value system. The sum of weights for all assessed criteria must not exceed 20. In a simplified version of this method [42,43], only domains are valued. The advantage of this method is the holistic assessment of efficiency. The disadvantage is its discretionary nature. This article attempts to reduce recognition by using the expert method. In addition, what was proposed in each domain was its own adequate criteria for assessing the efficiency of the VPP more broadly than just the information system.

3. Materials and Methods

This article deals with a real VPP that operates in Poland, in a region called Lower Silesia. The virtual power plant consists of a fragment of the distribution network on medium voltage and low voltage (for more details see [2]). The main distributed energy resources that are integrated into the virtual power plant are a 1.25 MW hydropower plant (HPP), a 0.5 MW battery energy storage (ES) and photovoltaic (PV) micro-installations. The HPP has three synchronous generators and is located above a water reservoir. HPP and ES are connected to a medium voltage level. The ES unit is located at the hydropower plant. The PV micro-installations consist of 16 pairs of different PV modules working together with a total capacity of 6 kWp, 17 micro photovoltaic installations (with capacities ranging from 1 to 5 kWp) with a total capacity of 48.99 kWp, 3 small PV installations with capacities 17.4 kWp, 28.78 kWp and 29.7 kWp, and a horizontal-axis wind turbine with a capacity of 1.5 kWp. The total capacity of the PV micro-installations is 132.37 kW.

3.1. Method of Economic Efficiency of the VPP

The method developed as a part of the project to assess the economic efficiency of the operation of the VPP can be divided into two key stages. The first stage involves developing a renewable generation profile for the generation units within the VPP, which includes a schedule of their operating times and production quantities. As a result of the conducted research [44], models that allow for the creation of short-term point and probabilistic forecasts based on point forecasts obtained using the quantile regression averaging (QRA) method for the electricity production of individual generation sources or their groups were developed. These models were used to forecast the production and regulatory capacities of the sources entering the VPP within a time horizon of up to 48 h.

The second stage involves optimizing the operation of the VPP, which takes place in the following two phases:

• Optimization to obtain the offer curve, which will be used to make decisions regarding the volume of energy offered for sale on the DAM.
• Optimization of VPP operation for balancing sales and purchases on the balancing market.

This stage utilizes N different scenarios of forecasted DAM prices and generation from production sources. A target function was proposed [45], which involves maximizing profits using historical data and constraints related to the characteristics of the individual sources belonging to the VPP:

$$\max \left\{\sum _{t=1}^{48}\sum _{i=1}^n{\pi }_i\left({\lambda }_i\left(t\right)\ast P_i\left(t\right)-\mathrm{C}\left(VPP\left(t\right)\right)\right)\right\}$$

(1)

where ${\pi }_i$ represents the probability of the i-th scenario occurring, ${\lambda }_i\left(t\right)$ and $P_i\left(t\right)$ correspondingly denote the obtained energy and energy price according to that scenario, while $C\left(VPP\left(t\right)\right)$ represents the costs incurred by the VPP in accordance with the decisions made.

Both phases of optimization aim to maximize the profits obtained by the VPP within a specific, appropriately long time horizon, while considering operational constraints. All technological constraints of the ES, including those stemming from battery management system procedures, are considered in the characteristics of the storage unit’s operation. These characteristics encompass aspects such as the power for charging and discharging the storage, the state of charge and generation efficiency. Charging the storage unit with full power is only feasible during HPP operation. Similarly, all technological constraints of the HPP and those arising from the distribution network are considered, such as the schedule of HPP operation along with its associated constraints, turbine operation constraints due to staffing hours, time intervals between generator startups, water level in the reservoir and minimum water discharge times from the water reservoir. For more details see [46].

The problem was solved using linear programming with an arbitrary number of variables. The created schedule for the VPP undergoes verification during execution, which occurs on an hourly basis. The applied optimization algorithm was described in the report [47].

During the validation of predictive algorithms and the optimizer, issues arose with the input data. Due to the absence of installed measuring devices along the course of river inflows to the HPP water reservoir, electricity production forecasts relied on seasonal data. Furthermore, a significant variability in the amount of water entering the water reservoir was observed, with a span of values exceeding sixfold, making it difficult to forecast inflows based on it [47]. The provided data did not allow for predicting inflows in a way that would improve the optimizer’s performance. This can lead to the following problems. Firstly, there is a possibility of overestimating the generation capacity and submitting too high a selling offer on the DAM, which may result in the need to purchase additional energy from the balancing market, often at unfavorable prices. Secondly, there are technical restrictions regarding the maximum duration of shutting down generation units in the HPP in conjunction with the minimum time they must operate. If the inflow is overestimated, there could be a scenario where, despite forecasts allowing for it, there would not be enough water in the reservoir to start them up within the required minimal timeframe.

Additionally, there was an issue with the compatibility of measurement data from PV micro-installations with meteorological data. Predictive algorithms require data of sufficient length, which was not met. There were also errors in the measurement data from the ES, which hindered the proper operation of the optimizer. These errors were related to interruptions in the operation of measuring devices and the operator’s actions not aligning with the planned schedule. The low-quality data resulted in significant root mean squared error (RMSE) in the energy price forecasts on the DAM, with the RMSE for the best forecast reaching a level of 16.9 [47].

In search of alternative methods for assessing the effectiveness of a new investment, the method of avoided costs and a heuristic method were proposed.

Calculation Model for Avoided Costs Method

Two sources of benefits were considered in the calculation of economic efficiency. The first one is from the savings on the hydropower plant’s own needs, which are met by the ES when the generators are not running and the ES is charged. The second benefit is from charging the reservoir from the hydropower plant when the price in the day ahead market (DAM) is low, and from discharging it at a high price.

The avoided costs method focuses on comparing the net revenue generated from two variants, A and B. Variant A is when the HPP works alone, while Variant B is when the HPP works together with the ES, representing the simplest version of the VPP. By choosing Variant B, we expect to benefit from savings on HPP’s own needs, which are met by the ES when generators are not running, and from the benefit of charging the ES at a lower price and discharging it at a higher price. The sum of these benefits represents the avoided cost, that is, the cost we do not incur if we select the VPP work variant. We calculate this cost by comparing net revenues for the two options. By net revenue, we mean revenue-less costs associated with the HPP and ES work’s own needs.

Avoided costs were calculated as the daily difference of the sum of the net revenues of Variant B and Variant A for 24 h, considering the optimization algorithm. The daily net revenue (${TR}_N$) is the sum of the net hourly revenue vector $z_i\left(h\right)$ for a given variant, as follows:

$${TR}_i=\sum _{h=1}^{24}{z}_i\left(h\right)$$

(2)

where $z_i\left(h\right)$ represents the net revenue for hour h and means the type of option A or B.

Monthly avoided costs (K) are calculated as the sum of daily avoided costs, as follows:

$$K=\sum _{d=1}^{30}{(TR}_A^d-{TR}_B^d)=\sum _{d=1}^{30}\left(\sum _{h=1}^{24}{z}_A^d\left(h\right)-\sum _{h=1}^{24}{z}_B^d\left(h\right)\right)$$

(3)

where

• K—monthly avoided costs;
• ${TR}_i^d$—daily net revenue from the i-th variant;
• $z_i^d\left(h\right)$—net revenue from the i-th variant for day d and hour h.

In Variant A, if the HPP operates and produces more than its own needs, it meets its own needs from this volume of production and sells the surplus on the DAM. If the HPP does not satisfy its own needs, it must buy the missing energy from the grid at a price resulting from the B23 tariff (B23 is a name of one group of tariffs in Poland. The analyzed HPP buys electricity at this tariff.). The computational scheme of the algorithm for Variant A is shown in Figure 1.

In comparison to Variant A, the appearance of the energy storage in Variant B changes the energy and revenue flows. Since it was agreed that the cost side of this solution should not be considered, the proposed solution focuses on the revenue. Specifically, the benefits of being able to meet its own needs by the storage (${PW}_{HPP2}\left(h\right)$) or being able to charge the storage $Q_{ES}\left(h\right)$ and sell the energy at a higher price in another hour. In this variant, the HPP’s own needs can be met by the HPP (${PW}_{HPP1}\left(h\right))$ or by the ES when the HPP is not operating or generates less than its own needs.

Therefore, the HPP’s own needs vector was divided into two vectors. Including the own needs vector means that when the ES is unloaded and the analyzed HPP is not operating, the ES’s own needs are met by the distribution network. The ES’s own needs vector does not appear in the algorithm because the energy sold from the ES is net energy and the ES’s own needs are met by the ES as the maximum discharge level is 20%. The portion of energy and revenue flows related to the HPP remains unchanged. The inclusion of the own needs vector means that when the ES is unloaded and the HPP is not operating, the ES’s own needs are met by the distribution network. The computational scheme of the algorithm for Variant B is shown in Figure 2.

The model for calculating avoided costs was built and tested using data from April 2020. The model was then used for calculations using data from six months, that is, from May to October 2020 when the VPP system was tested. The actual price of energy from the DAM market was taken from data from the Commodity Energy Exchange website on hourly energy prices by the delivery date for the fixed rate I (PLN/MWh). The price of energy drawn from the grid according to the B23 tariff was assumed at 500 PLN/MWh, according to the information from the power company.

3.2. Heuristic Method

The proposed heuristic method belongs to the group of scoring methods for IE-based efficiency assessments. In the information economics method, inputs (costs or expenses) and positive effects (benefits) are considered [21]. It involves estimating the weights and ranks of each criterion and then summing the products of their values. The Delphi method was used to estimate the weights of the criteria and the ranks of each criterion. The efficiency of the pilot plant (E) VPP was estimated based on the following formula:

$$E=\sum _{i=1}^n{w}_i·r_i$$

(4)

where

• w_i—the weight of the i-th category, assuming that the sum of weights for all categories is 20.
• r_i—the rank of the i-th category, r EUR {0, 1, 2, 3, 4, 5}.
• n—the number of categories.

E can take values from 0 to 100 where the investment is more efficient the closer the value of the E index is to 100.

The research was conducted in multiple stages. The main stages were as follows:

• Establishing criteria for assessing the efficiency of VPP installations based on consultations with the staff of the power company;
• Estimating the weights of each criterion based on the Delphi method and a survey questionnaire, with which the first group of experts was surveyed;
• Estimating the ranks for each criterion based on surveys among the second group of experts;
• Calculating the value of the E index according to Formula (4).

The following sections will briefly describe each stage of this study. However, the experts will be characterized first.

3.2.1. Experts

Two groups of experts participated in this study. The first group participated in a Delphi survey to estimate the weights of the sub-criteria and the second group assessed the ranks of each criterion. The experts assessing the weights were the employees of the energy company who held management positions or were high-ranking specialists in fields related to the VPP performance criteria under study. Knowing the enterprise’s strategy, they were able to identify relevant criteria for the entire organization. In order to do this, experts filled out questionnaire no. 1 according to the procedure of the Delphi method.

The second group of experts assessed the extent to which the pilot plant affects the parameters of each sub-criterion, that is, they assigned ranks to each sub-criterion. Therefore, this group of experts had to have access to data on the operation of the pilot VPP. Hence, the second group of experts consisted of (1) the employees of the power company who were involved in the project or had information about the operation of the pilot VPP, (2) the employees of the company developing the VPP optimization algorithm and (3) the scientists studying the effects of the pilot VPP. Each expert had access to the relevant data (e.g., finance staff for financial data and those assessing the results of readings from individual meters for technical data). The advantage of such a selection of experts was to minimize the risk of obtaining detailed information (e.g., financial) by unauthorized persons. This group of experts filled out survey no. 2 and, in the survey metric, they indicated what knowledge (theoretical and/or practical) they had in the field.

3.2.2. Establishing Criteria

The proposed criteria are assigned to the business or technology domain of the VPP. The general structure of the criteria is presented in Figure 3. The list of criteria was reviewed by the VPP staff. A total of 34 sub-criteria were adopted (

Table 2. Sub-criteria and their weights adopted on the basis of the second round of surveying.

Code Criterion	Criterion	Coefficient of Variation		Weights after the Second Round
		Round I	Round II
B.1.	Financial
B.1.1.	Financial efficiency	15.56	14.89	0.6
B.1.2.	Avoided costs (related to optimization and energy storage)	12.50	14.57	0.6
B.1.3.	Employment costs	13.75	13.32	0.5
B.1.4.	Costs of environmental fees	14.14	17.82	0.5
B.2.	Organizational and structural
B.2.1.	Making the organizational structure more flexible	27.36	21.30	0.5
B.2.2.	Specialization of work (growth)	28.28	26.80	0.5
B.2.3.	Flattening the organizational structure	28.34	25.20	0.5
B.3.	Human resources
B.3.1.	Number of employees	20.33	14.43	0.5
B.3.2.	Employee motivation	26.35	25.39	0.5
B.3.3.	Staff qualifications	18.90	17.68	0.6
B.4.	Knowledge and information resources
B.4.1.	Access to databases	16.56	15.47	0.6
B.4.2.	Reliability of data	11.17	11.11	0.7
B.4.3.	Data flow control	25.66	23.92	0.6
B.4.4.	Data security	16.45	15.91	0.7
B.4.5.	Streamlining the process of information exchange between selected departments within the enterprise (e.g., sale, production) and management levels	16.56	17.64	0.6
B.5.	Key processes in an energy company
B.5.1.	Streamlining the production planning process	18.92	17.93	0.6
B.5.2.	Streamlining the production control process	18.92	20.41	0.6
B.5.3.	Streamlining the sales planning process (including rostering)	20.15	18.95	0.6
B.5.4.	Streamlining the sales process	14.43	14.29	0.5
B.5.5.	Streamlining the process of controlling sales	13.36	13.98	0.6
B.6.	Relations with the environment and competitive position
B.6.1.	Increasing the number of customers (entities participating in the VPP)	16.45	15.91	0.7
B.6.2.	Expanding the scope of offerings (providing new products to the market, e.g., consulting, optimization)	11.17	11.07	0.6
B.6.3.	The prestige of the energy company as an innovative entity	16.56	15.47	0.6
B.6.4.	The prestige of the energy company as an integrator of distributed RESs	16.56	15.47	0.6
B.6.5.	The prestige of the energy company as an enabler of the energy society	17.93	18.84	0.6
B.6.6.	Environmental effect	14.41	14.14	0.5
	Technical criteria
T.1.	Technical optimization of the operation of distributed renewable energy sources—improvement of coordination of operation of distributed and individually owned RES generators	11.07	10.88	0.6
T.2.	Ability to remotely control the operation of the system	10.47	15.41	0.8
T.3.	Reducing the response time to random changes (e.g., changes on the DAM) and analysis for an extensive big data system	10.47	10.91	0.8
T.4.	Increasing balance sheet capacity	16.56	17.64	0.6
T.5.	Improving quality parameters (e.g., current voltage, power)	35.79	33.70	0.5
T.6.	Automating the energy production processes	15.42	17.68	0.6
T.7.	Reducing energy transmission and distribution losses	35.32	32.92	0.6
T.8.	Continuity of electrical energy supply	34.40	32.92	0.5

Source: based on [47] (pp. 249–254).

).

3.2.3. Estimating the Weights of Particular Criteria

The weights of each criterion were estimated by the first group of experts. The target sum of the scales was 20. However, for psychological reasons, a linguistic scale was used in survey no. 1. Each expert assessed the criteria by answering the following question: “How relevant (important) is the criterion in assessing the efficiency of the VPP?”. When performing the assessment, he/she could have chosen the following options:

• Critical;
• Very important;
• Important;
• Minor;
• Totally unimportant;
• I have no opinion.

Criteria weights were estimated based on two rounds of surveys. Eight experts participated in the first round of surveys

After the first round of questionnaires, it turned out that coefficients of variation calculated according to Formula (5) were too high were for some criteria.

$$variation coefficient=\frac{standard deviation}{sample average value}·100\mathrm{\%}$$

(5)

It was assumed that low variation is when the ratio does not exceed 20%. According to the procedure, the experts were notified of the results of the survey: median, mean and standard deviation. They also had their own answers marked.

The second round of surveys was conducted to reduce the variation in the experts’ opinions. The answers concerning whether they change their opinions or retain their previous opinions were not sent by two out of eight experts. After interviewing the staff, it became clear that the two experts had not changed their previous opinion so their previous answers were accepted for calculation. Additionally, a conversation with one more expert was held. In total, the opinions of nine experts were taken to assess the weights. For most criteria, the variation decreased. Due to the lack of consensus among experts in assessing the weights of criteria T.5 (the names of criteria are in Table 2), T.7 and T.8, for which the spread was about more than 30%, an interview was conducted with an employee of the energy company to explain such disparities. Information was obtained that the pilot plant, due to its small scale, has a negligible impact on technical parameters in criteria T.5, T.7 and T.8. However, as new sources are connected to the VPP plant, experts believe it will have a greater impact on quality parameters and reduce grid losses and continuity of electricity supply. Due to the study of the pilot installation and the resulting limitations, another survey round was not conducted as it would not have the intended effect (resistance from respondents and their stiffening of opinions).

The median was used to assess the weights of each criterion, and then the weights were rescaled so that their sum was 20 (formula 6).

$$w_i=\frac{M_i·20}{\sum _{i=1}^{m}Mi}$$

(6)

where

• w_i—the weight of the i-th criterion in the scoring assessment;
• M_i—the median for the i-th criterion;
• m—the number of sub-criteria; m = 34.

The established weights after the second round of surveying are presented in Table 2, which also provides the coefficients of variation after the first and second round of surveying. Even after the second round of surveys, there are still criteria for which weight variation is medium. However, for two criteria, the variation decreased from medium to low.

3.2.4. Rank Estimation

The ranks of the individual criteria were estimated by a second group of experts by rating each criterion on a six-point scale (from 0 to 5), which was described as follows:

• “0”—the installation of a VPP has a very negative impact on the criterion;
• “1”—the installation of a VPP has a negative impact on a criterion;
• “2”—the VPP installation has no impact on the given criterion (neutral);
• “3”—VPP installation has a minor positive impact on a given criterion;
• “4”—the VPP installation has a medium positive impact on the given criterion;
• “5”—VPP installation has a very high positive impact on a given criterion (significantly improved quality/value for a given criterion).

The scale is asymmetrical, containing two negative and three positive values and a neutral rating (2). This scale was used due to the following reasons:

• The desire to obtain only an opinion that the impact is negative and not the degree of this impact, as opposed to information about the positive impact of the VPP on the individual efficiency criteria.
• Not planning an in-depth statistical analysis due to the relatively small number of respondents.

Due to the varying thematic scope of the assessed criteria, each sub-criterion was assessed by a different number of experts (

Table 3. The number of experts in selected thematic areas.

The Thematic Scope of Knowledge	Total
Finance	4
Organization and structures of enterprises and market issues (including, electrical energy markets, sales processes at a stock exchange)	7
Human resources (human relations)	3
Data acquisition and processing (widely understood knowledge and information)	8
Planning, organizing and controlling production	5
Planning, organizing and controlling sales and marketing in an energy company	4
Issues of ecology	7
IT technology	5
Electricity and power grid/Automation of transmission and distribution of electricity	7
Concerning optimization issues (including balancing) and forecasting	10

Source: [47] (p. 258).

). A total of 13 experts provided their answers. However, a different number of experts gave their responses for each criterion due to the extent of their knowledge.

4. Results

4.1. Avoided Costs Method

The aggregate results of the monthly avoided costs calculations since May to October are shown in

Table 4. Monthly avoided costs.

Month	A	B	Avoided Cost	EW Production
May	PLN 15,287.16	PLN 15,305.16	PLN 18.01	94.76 MWh
June	PLN 63,441.21	PLN 63,004.88	PLN −436.33	255.48 MWh
July	PLN 27,837.59	PLN 27,628.19	PLN −209.39	109.05 MWh
August	PLN 8655.22	PLN 8011.74	PLN −643.48	39.82 MWh
September	PLN 35,551.47	PLN 35,076.41	PLN −475.05	144.34 MWh
October	PLN 162,326.57	PLN 161,837.34	PLN −489.23	668.94 MWh

Source: [47] (p. 246).

. Positive values of avoided costs mean that Variant B is better than Variant A, while negative values indicate that Variant A is more favorable. Negative values were obtained for all months, which means that the variant of independent HPP is more favorable. This value shows the loss due to the operation of the energy storage.

In the analysis of the consecutive days in each month, it was noted that the values of avoided costs were negative on most days, but there are days on which these costs were positive. In May, positive values were obtained for 11 days, in June for 9 days, in July for 14 days, in August for 6 days, in September for 11 days and in October for 10 days. The monthly range of variation in avoided costs is shown in

Table 5. Range of variability of avoided costs.

	Avoided Costs
Month	Minimum	Maximum	Average
May	−110.00	113.65	0.58
June	−136.75	87.80	−14.54
July	−127.96	87.91	−6.75
August	−103.43	89.06	−20.76
September	−110.92	104.31	−15.84
October	−120.03	95.36	−15.78

. The minimum values range from −136.75 to −103.43, while the maximum values fall within the range of 87.80 to 113.65.

Figure 4 and Figure 5 show the daily avoided costs for May and July. We selected the months in which the incurred loss was the smallest, that is, for which positive avoided costs were received on the largest number of days.

The avoided costs in this model have two sources, as follows:

• From savings on the hydropower plant’s own needs, which are met by the ES when the generators are not running and the ES is charged.
• From charging the ES from the hydropower plant when the price on the DAM market is low and discharging is at a high price.

The analysis of the data obtained showed that the benefit of savings based on its own needs was small due to the small capacity of the power plant; in addition, it was noted that its own needs often had to be met with the energy taken from the grid at the price of 500 PLN/MWh, resulting from the B23 tariff.

4.2. Heuristic Method

From the enterprise management’s estimated weights, the most important criteria for the VPP are those of the “knowledge and information resources” group, the technical criteria and the relationship with the environment and the competitive position. Financial criteria are less important. The least important criteria are human resources and organizational and structural aspects.

Table 6. The medians, arithmetic means and standard deviations.

	Criterion	Median	Mean	Standard Deviation
B.1.	Financial
B.1.1.	Financial efficiency	4.5	4.50	0.50
B.1.2.	Avoided costs (related to optimization and energy storage)	5	4.67	0.47
B.1.3.	Employment costs	4	3.57	1.05
B.1.4.	Costs of environmental fees	3	3.00	1.29
B.2.	Organizational and structural
B.2.1.	Making the organizational structure more flexible	4	4.00	0.82
B.2.2.	Specialization of work (growth)	4	4.14	0.83
B.2.3.	Flattening the organizational structure	4	3.60	1.02
B.3.	Human resources
B.3.1.	Number of employees	3	3.40	0.49
B.3.2.	Employee motivation	3	3.50	0.76
B.3.3.	Staff qualifications	4	4.29	0.70
B.4.	Knowledge and information resources
B.4.1.	Access to databases	5	4.55	0.50
B.4.2.	Reliability of data	5	4.55	0.50
B.4.3.	Data flow control	5	4.44	0.68
B.4.4.	Data security	5	4.33	0.82
B.4.5.	Streamlining the process of information exchange between selected departments in the enterprise (e.g., sales, production) and management levels	5	4.75	0.66
B.5.	Key processes in an energy company
B.5.1.	Streamlining the production planning process	5	4.60	0.49
B.5.2.	Streamlining the production controlling process	5	4.40	0.80
B.5.3.	Streamlining the sales planning process (including rostering)	5	4.40	0.92
B.5.4.	Streamlining the sales process	4	3.90	1.14
B.5.5.	Streamlining the process of controlling sales	4	4.00	1.00
B.6.	Relations with the environment and competitive position
B.6.1.	Increasing the number of customers (entities participating in the VPP)	5	4.71	0.45
B.6.2.	Expanding the scope of offerings (providing new products to the market, e.g., consulting, optimization)	5	4.75	0.43
B.6.3.	The prestige of the energy company as an innovative entity	5	4.67	0.67
B.6.4.	The prestige of the energy company as an integrator of distributed RES	5	4.78	0.42
B.6.5.	The prestige of the energy company as an enabler of the energy society	4	4.22	0.92
B.6.6.	Environmental effect	4	3.89	0.87
	Technical criteria
T.1.	Technical optimization of operated distributed renewable energy sources’ improvement of the operational coordination of distributed and individually owned RES generators	5	4.67	0.62
T.2.	Ability to remotely control the operation of the system	5	4.67	0.47
T.3.	Reducing the response time to random changes (e.g., changes on the DAM) and analysis for an extensive big data system	5	4.31	0.82
T.4.	Increasing balance sheet capacity	5	4.54	0.50
T.5.	Improving quality parameters (e.g., current voltage, power)	5	3.90	1.64
T.6.	Automating energy production processes	5	4.70	0.64
T.7.	Reducing energy transmission and distribution losses	5	3.60	1.85
T.8.	Continuity of electrical energy supply	4.5	3.80	1.60

Source: [47] (pp. 258–260).

presents the experts’ assessment of the ranks for each group of criteria and Figure 6, Figure 7, Figure 8, Figure 9 and Figure 10 illustrate the selected ranges according to the median and the mean value. The lowest value is shown in red in Table 6. Means and standard deviations, when the spread from the average value is relatively large, are presented on grey backgrounds. This means that the experts have different opinions.

The business (B) domain criteria received quite varied ranks, but they were positive. Considering the median, the mean value (more than 4.7) and a spread of less than 10%, the three criteria in group B.6 were rated highest. These are as follows:

• B.6.3: Prestige of the energy company as an innovator.
• B.6.2: Expanding the scope of offerings (providing new products to the market, e.g., consulting, optimization).
• B.6.1: Increasing the number of customers (entities participating in the VPP).

From the point of view of the avoided costs analysis method, it is noteworthy that this criterion (B.1.2) also received a high-rank score and a relatively small spread (10.1%) (Table 4), which suggests and confirms the results obtained from the efficiency calculation using the avoided costs method. From the group of business criteria, the criterion of the cost of environmental fees (B.1.4) received the highest disagreement in the experts’ assessment as well as the lowest value of median and the mean, but this value is positive. The reason could be that the VPP uses the same renewable sources, such as the HPP, before investing in the VPP.

Based on the median, it can be said that all technical efficiency criteria of the VPP were assessed positively. The impact of the pilot plant on technical efficiency is high or very high. However, looking at the arithmetic mean of the ranks and the standard deviation, it is essential to note the relatively large spread of responses in relation to the three criteria (T5, T7 and T8). The reasons for the inconsistency of responses among the experts were the same as for the estimation of the weights. The experts considered that the impact of the pilot VPP alone is negligible on the criteria listed, but that it will increase with the expansion of the VPP. What is essential is that the impact itself is positive.

The economic efficiency of the pilot VPP was estimated according to Equation (4), taking the interpretation of the obtained efficiency factor values E as in

Table 7. Interpretation of the E-measure for individual ranges.

Efficiency	Interpretation of E-Measure Values
Very high	[90, 100]
Medium efficiency	[70, 80)
Low efficiency	[41, 60]
Zero efficiency	40
Slight inefficiency	[21, 40)
Very high inefficiency	[0–20]

Source: [47] (p. 267).

.

The values of the E-measure calculated on the basis of the results obtained using the heuristic method for the three rank variants are presented in

Table 8. Value of the aggregated economic efficiency index.

Adopted Scoring Basis	Aggregate Indicator Value (E)
Median for all experts	92.0
Median adjustment for all experts	86.7
Mean score of all experts	85.5

Source: [47] (p. 267).

. The ranks were assumed to be equal to the following:

• The median (Table 4).
• The adjusted median, where the adjustment of the median consisted in assigning neutral values (i.e., 2) for the criteria with high variability, i.e., B.1.4, T.1, T.7 and T.8.
• The arithmetic mean (Table 4).

In the experts’ assessment, the pilot VPP achieved high efficiency.

5. Discussion

In an earlier work [3], it was assumed that battery energy storage would be charged and discharged twice a day. At the same time, they should be charged from the hydropower plant when the price in the energy market is the lowest and discharged at the peak when the price is the highest. This condition is necessary to achieve positive economic efficiency.

EW’s actual energy production data are for a 6-month period. Assuming that energy production in the second half of the year will be comparable, an annual production volume of 2624.78 MWh was obtained. With respect to the assumptions presented in the paper [3], the energy production from the HPP during the study period is less than the average annual energy production for typical weather, which was 3053.7 MWh. The calculated the 5-year modified NPV for the scenario with one energy storage was PLN −751 thousand. Based on the obtained results, the annual value of avoided costs was calculated, which equaled to PLN −4470.94. This value is significantly lower than that calculated with the NPV. However, these values are still negative.

By analyzing the results shown in Table 1, it can be seen that there is no relationship between the amount of hydropower production and the losses from the operation of the ES. This means that the negative values depend mainly on the difference in energy prices between the charge and discharge of the ES. The difference is too small on most days to reach positive values, even if the ES is charged and discharged twice a day. The following examples illustrate this problem.

On 21st September at 8 p.m., the highest price on the DAM was recorded, which was PLN 449/MWh. On that day, the ES was charged between 6 p.m. and 7 p.m. when the DAM prices were PLN 296.40/MWh and PLN 367.55/MWh, respectively, and discharged throughout the day (at 9 kW per hour) at different prices, which were higher and lower than the charging price, as shown in Figure 11. As a result, the total daily avoided costs were PLN −15.17. An even larger daily loss of PLN −81.66 was recorded on 18th September and the cause was very similar (cf., Figure 12). However, this shows the high potential for positive avoided costs.

The avoided cost method was proven to work well when costs are a key factor affecting the cost effectiveness of a project. Using this method can help us to focus on minimizing costs. However, it is important to note that it is crucial to compare the two variants—Variant A, which corresponds to the scenario where the HPP operates independently, and Variant B, which refers to the situation where the HPP collaborates with the ES (the most basic version of the VPP). The obtained data show that the pilot plant is not financially profitable, despite previous analyses showing that a positive outcome is possible.

The problem that arose here was the small magnitude of the savings on own needs, which was caused by several factors. Firstly, the low energy production of the HPP is due to the low water level in the reservoir resulting from the drought. Secondly, the installed capacity of the ES was small, and, in addition, the ES was only charged and discharged once a day instead of twice a day. Thirdly, the actual operation of the VPP did not follow the operating schedule resulting from the optimization algorithms, with the result that own needs had to be met with energy drawn from the grid at a price higher than the market price. The results obtained with the avoided cost method are not as expected, which is mainly caused by the fact that the pilot VPP did not operate according to the optimization schedule. The significant deviations from the operating schedule were mainly due to the human factor and also weather conditions. While the human factor can be eliminated by introducing full automation of the plant, the weather conditions are beyond our control.

The use of the heuristic method for assessing the efficiency of a new investment (VPP) allows non-financial effects to be taken into consideration from a new investment, which are important for employees as well as managers. The employees talked about their fear related to the potential layoff of workers as a result of the implementation of the investment. The experts pointed out that this issue, linked to human resources, is the important group of criteria for assessing the efficiency of the investment. Thus, the B3 set criterion was added. However, the research discovered that two sub-criteria (B.3.1 and B.3.2) from this set achieved medium weights and their ranks reached the following values, respectively: (a) for B.3.1: median = 3; mean = 3.4; (b) for B.3.2: median = 3; mean = 3.5. On the basis of the piloted installation, the experts assessed that the VPP will have a minor positive impact on human resources, but not negative as feared by employees.

The experts assessed relatively high weights for the technical criteria focused on improving the parameters which simplify the job and support decision-making process related to the optimization of purchase, production and sales. Such improvements do not always take into consideration the financial profit, but they improved the quality of information and reaction speeds regarding market changes. The assessment of the efficiency by using the NPV method or the avoided costs method do not take these issues into consideration.

The advantage of the heuristic method is the possibility of fitting it to the individual preferences of decision makers and the type of investment. The decision makers verified the criteria. Thus, the method is useful for the decision-making process. This approach is similar to the consumer utility theory. Its disadvantage is the lack of the possibility to compare the efficiency of investment between different companies because the managers have different preferences.

One problem was the lack of possibility to reduce the standard deviation due to the very strong personality of one expert. He would not change his opinion in the second round of surveying. Thus, the assumed spread of the experts’ answers was not achieved.

Each of the two presented methods is appropriate for different conditions and investments (

Table 9. Conditions and assumptions related with investment (‘+’ means appropriate, ‘-‘ means inappropriate).

Conditions	Avoided Cost Method	Heuristic Method
The investment does not create revenue	+	+
The aim of investment is minimalization of costs	+	-
The costs and benefits of investment and its products or results are impossible to measure in money	-	+
Social benefits are priority	-	+
Ecological benefits are priority	-	+
Technological development is priority	-	+
The investment influence on different department and functions in organization	-	+
The investment change business model	-	+
Uncertainty and complex environment	-	+

Source: author’s work.

). Based on the table, we can see that these methods complement each other.

6. Conclusions

The results confirm that in the case of pro-ecological and innovative investments, more comprehensive methods should be used to assess efficiency than just the financial efficiency method because of the following reasons:

• Investments of this type have a small scale of production and coverage and are therefore financially inefficient.
• Executives expect investments not only to be financially efficient, but also to improve the efficiency of certain processes, which will not always be reflected in the financial data. Examples include easier access to data or easier remote control of the plant.
• An important role is played by the enterprise’s reputation—the perception that it is a forerunner of certain processes. The effect of improving the enterprise’s prestige can be attracting new customers, which will translate into financial results in the long term.
• The type of investment: the VPP is a kind of platform which is based on network cooperation. Its value is higher and higher when the number of users (e.g., consumers, prosumers) increases.

In turn, the heuristic method includes a lot of the dimensions of efficiency assessment which the financial account and the avoided costs do not take into consideration. This concerns aspects such as (1) the company’s participation in a process focused on technological development and the improvement of the company’s image; (2) the improvement of the information flow process, which influences positively quality of work; and (3) the improvement of coordination focused on the electricity use coming from dispersed RESs. These activities will contribute to the RES development in local areas as well as the use of energy sources localized close to energy users, thus reducing grids waste and greenhouse gases.

In the heuristic method, the following sub-criteria are directly related to social issues:

• Number of employees (B.3.1) because employees fear redundancies.
• Data security (including prosumers) (B.4.4).
• Expending the scope of offerings (e.g., consulting) (B.6.1).

In turn, the environmental issue is included directly in the sub-criterion labelled ‘environmental effect’ (B.6.6) and indirectly in following sub-criteria:

• Technical optimization of the operation of distributed renewable energy sources—improvement of coordination of operation of distributed and individually owned RES generators (T.1).
• Reducing energy transmission and distribution losses (T.7).

In addition, in oligopolistic markets—the market in which the company under study operates—the enterprise’s reputation serves an important marketing and strategic function by enabling strategies to deter competitors and create additional barriers to entry.

The assessment of investments using the heuristic method confirms the results obtained in the assessment of efficiency using the avoided costs method.

The difficulty in using the heuristic method is to achieve consistency in assessments when they are conducted ex ante. This is primarily due to the limited number of experts, which is necessary if the company wants to preserve trade secrets. Other factors seem to be the limited range of data obtained by the pilot installations and the resistance of experts to change, such as the fear of losing their jobs.

Future research should focus on the development of the heuristic method to fit the criterion of assessment, their wages and a measuring scale fitting the VPP investment. Therefore, the results of the future research should be a universal measuring tool dedicated to the type of investment, such as a VPP. With the emergence of an increasing number of VPPs globally as well as the lack of capacity to show tangible financial benefits from these investments, which are beneficial to the environment and society (including a local one), it seems crucial to have a tool that enables a comparative analysis to be conducted for different projects belonging to the same group of investments. Different kinds of authorities (e.g., managing public funds and co-funding these investments or supporting competitiveness and innovativeness of companies, regions and economics) should be interested in a combination of these methods and a tool related to the heuristic method. Furthermore, future research should work towards developing a holistic method that combines not only these two methods, but also others that cover areas that these two methods do not.