Market Value and Agents Benefits of Enhanced Short-Term Solar PV Power Generation Forecasting

Abstract

Renewable energy sources such as PV solar or wind power are intermittent and non-dispatchable. Massive integration of these resources into the electric mix poses some challenges to meeting power generation with demand. Hence, improving power generation forecasting has raised much interest. This work assesses the market value of enhanced PV solar power generation forecasting. Then, we analyse the different agents present in the electricity system. We link the studied agents to the proposed market values based on both analyses. Improving the accuracy of RES forecasting has massive potential as the sector grows and new agents arise. It can have reactive values like reducing imbalances or proactive values such as participating in intraday markets or exercising energy arbitrage. However, accurate forecasting can also lead to opportunistic values that can be exploited by malicious agents if they are not adequately regulated.

1. Introduction

The energy transition implies leaving behind fossil fuels to mitigate the effects of climate change and increase the renewable energy sources (RES) share in the electricity grid and markets. These RES are spread across the territory, favouring the decentralisation of the energy system and the participation of small consumers with active roles. In addition, combining a decentralised system with the development of the Internet of Things (IoT) and Smart Grids drives the energy system’s digitalisation, enabling new solutions to connect and organise system elements and stakeholders. RES are also becoming the most cost-effective electricity available, according to the International Energy Agency [1].

Nonetheless, its implementation has significant challenges as wind and solar are intermittent and non-dispatchable energy sources. Thus, generation cannot be guaranteed to be coordinated with demand to meet it. There will be many moments with power surplus or deficit posing risks to electricity availability [2]. This mismatch leads to additional storage capacities to balance electricity supply and demand over time [3]. Nonetheless, the integration of RES requires other developments such as flexibility on the demand side, complex data acquisition systems or precise generation forecasting.

Here, we focus the analysis on the value of improving short-term forecasting of PV solar generation. This technology has the most significant potential for growth in decentralised systems power generation and Smart Grids. However, small generators are blind to their expected generation. At the same time, “evidence suggests that firms possess private information that allows them to significantly improve the precision of the forecasts of their own plants’ available capacities” [4]. Therefore, accurate forecasting will close the profitability gap between large and small generators, making the latter more active in the energy system and, thus, the energy transition.

Research has mainly focused on developing methods to improve PV generation forecasting [5,6,7]. In this sense, Stylianou et al. [8] studied the use of cloud cover to forecast PV power generation. In turn, Konstantinou et al. [9] forecasted PV generation 1.5 h ahead using machine learning. Regarding the value of these techniques, Kaur et al. [10] studied the benefits of improving short-term forecasting of the energy imbalance markets, concluding that it notably reduces the probability of imbalances.

However, to be best of our knowledge, the market value of short-term forecasting PV generation has not been assessed for all markets and agents in the electricity system. Hence, we answer the following research objectives with this work:

We define the different market values of short-term (less than two hours ahead) forecasting PV solar power generation.

We identify and map the market values with the potential benefits for the different energy market agents.

On the one hand, the results allow analysing the potential benefits of investing in forecasting techniques for different actors and discussing how to exploit them. Some agents may be unaware of all the benefits of an accurate short-term solar generation forecast. On the other hand, results also point out the threads that this technology could pose for the electricity system agents, especially those under the European Electricity Market structure.

The rest of the paper’s structure is as follows. Section 2 provides the value analysis of improved Solar PV prediction in the market. Section 3 and Section 4 present the conceptual framework for analysing the energy system’s agents and market structure. Section 5 discusses the different values of Solar PV for the system and Section 6 how these values correlate with each agent of the system. Section 7 presents a case study of real-time solar PV forecasting. Finally, Section 8 presents conclusions.

2. Review of the Solar PV Generation Forecasting Methods

This section reviews the main PV solar power generation forecasting methods in the literature. Researchers often classify them into persistence models, physical methods, statistical techniques and hybrid approaches.

The persistence model is popular for short-term forecasting, especially for one-hour ahead horizons [6]. It has a low computational cost, time delay and reasonable precision [5]. This technique assumes that the climate conditions of the hour or day ahead will remain similar to the current conditions. The technique’s accuracy declines notably with the cloud cover and predictions longer than one hour. Researchers use this model as a benchmark to compare with other forecasting approaches for accuracy evaluation.

Physical methods involve numerical weather predictions (NWP), sky imagery and satellite imaging [7]. NWP consist of a set of mathematical expressions that define the state and dynamics of the atmosphere [6]. Literature categorises NWP methods according to the scale into global models considering the whole atmosphere and mesoscale models for restricted areas [5,7]. These methods are reliable for long-term forecasting up to 15-days ahead. However, the performance is affected mainly by sharp changes in meteorological variables [8].

Sky imagery provides a sky image horizon-to-horizon to detect clouds, measure their height and determine their motion. Sky imagery analysis has four main elements: obtention of the sky image, image analysis to recognise clouds, estimation of the cloud motion vector and prediction of short-term cloud cover, irradiance and power generation [7]. Satellite imaging is similar to the sky imagery technique. This approach uses sensors based on satellite images instead of a digital camera on the surface. Researchers use satellite methods effectively for irradiance predictions from 1 min to 5-h ahead [7].

Statistical techniques use historical time series and real-time data to forecast PV solar generation, and they can be time-series-based models or machine learning techniques. These techniques perform better short-term predictions than NPW methods [5].

Time-series-based models involve autoregressive moving average (ARMA), regression and exponential smoothing. The ARMA approach combines the autoregressive and moving average models. ARMA is popular because it can extract the statistical characteristics and adopt the Box–Jenkins method. Researchers have improved the technique with the introduction of ARIMA. ARIMA can capture irregular data as irregular climate patterns, but it is more computationally intensive than ARMA [6].

The regression method determines a relationship between response (dependent) and predictor (independent) factors [6,7]. This method is a repetitive process in which the outputs are assessed to modify the inputs. The weakness of this method is that it requires a mathematical model and several explanatory variables [6].

The exponential smoothing method makes predictions from statistical analysis of historical time series data [5,11]. This method imposes an unequal set of weights on the data. Thus, the data weights decay exponentially from the most recent to the most distant values [6].

The most widely used machine learning techniques are Artificial Neural Network (ANN) and Space Vector Machine (SVM) because they solve complex and nonlinear forecasting models [7]. Other algorithms are also helpful when adequately trained [12,13,14]. ANN is the most effective method when the data have complicated and nonlinear bonds. This method allows for excellent accuracy because it can be effectively trained [9,15]. However, ANN increases the method’s complexity by employing the multi-layered network architecture [6].

The SVM has excellent performance compared to ANN and other traditional statistical methods [7]. SVM is a supervised machine-learning method based on the structural risk minimisation principle. Researches apply SVM to time series regression; thus, forecasting PV power generation is an appropriate problem for the method. SVM’s strength is learning without depending on prior knowledge and simplifying the complex mathematical problems related to PV forecasting [6]. Nonetheless, SVM is highly sensitive to the parameters used in the forecasting model. These parameters are the penalty factor, the tube radius, and the kernel function parameter [6].

Finally, researchers have developed hybrid approaches to combine two or more techniques [16,17,18,19]. This combination has introduced the ability to combine linear and nonlinear approaches, enhancing the performance of these methods compared to individual techniques [7]. However, developers of hybrid models should use superior attributes to improve forecasting. The poor execution of a single technique in the hybrid architecture may impact the results negatively compared to the individual approach [6].

As a result of the literature review, it is now possible to accurately forecast the generation of distributed PV systems; the closer in time, the more accurate. Moreover, this new knowledge can be automated through IoT in smart grids and big data analysis to design algorithms that exploit its full potential. The following sections analyse the benefits derived from this data by the different agents operating in the electricity market in most countries.

3. Agents in the Power System

The power system is composed of multiple actors performing specific roles [20]. Figure 1 depicts the agents that can intervene in the electricity markets, connected by the wholesale and retail markets. The wholesale market refers to the energy markets where the big generators, consumers and retailers trade energy. The retail market refers to the commercialisation between retailers and final consumers.

3.1. Consumers

Energy end-users locate at the bottom of the illustration: small and large consumers and aggregators that combine purchase offers of various consumers. Consumers can connect to the distribution or transmission grids at low or high voltage. Residential or small commercial and industrial customers usually connect to the low voltage distribution grid. In contrast, large industrial and commercial customers often connect to the high voltage distribution power system or a high voltage transmission or sub-transmission power system. In addition, consumers can access the wholesale market by presenting economic guarantees or contracting the supply through a bilateral deal outside the regulated market. These alternatives usually are only accessible to large consumers.

Traditionally, end users acted only as passive costumers and receivers of electricity from centralised generation. Nonetheless, it is now common for customers to build their generation resources to become active consumers, usually called prosumers [21,22,23]. Thus, consumers can sell electricity to the market and offer flexibility in owning storage systems.

3.2. Intermediary Agents

Intermediary agents involve aggregators and retailers. Aggregators are emerging in the electricity markets, empowering consumers and small-scale generators by facilitating their access to the electricity markets [24]. An aggregator acts as an intermediary between owners of distributed energy resources (DER) installations, end-users and the agents in the electricity markets that serve these end-users or exploit the services provided by these DER [25]. Aggregators’ ability to connect their customers’ assets to the market is their most essential strength. In addition, they encourage the entrance of many demand-side providers or system services, facilitating the complex decision-making required to generate flexibility value. Aggregators combine resources and technologies to ease each other’s technical constraints and profit from scale effects. In this way, aggregators manage peak demand, arbitrage between low and high energy prices hours, and buy or sell storage capacity according to the system’s needs.

Retailers are responsible for providing electricity to consumers who do not participate in the wholesale electricity market. Retailers can obtain electricity from multiple sources like bilateral contracts, future markets, self-production, and the pool-based electricity market [26]. With the rise of renewable energy prosumers, retailers can increase their revenues by selling their surplus energy in the real-time market. Nonetheless, RES intermittency also leads to additional uncertainties in the decision-making strategies.

3.3. Operators

Operators are in charge of controlling the well-functioning of the system. The figure of the system operator guarantees secure and continuous operation of the production, transmission, and distribution systems. The Transmission System Operator (TSO) owns and manages the transmission grid. Hence, it is a natural monopoly and is highly regulated [20]. The TSO is also in charge of the market settlements, predicting demand to control the evolution and develop the transmission grid and scheduling the operation of production facilities. In addition, the TSO operates the adjustment services to resolve technical constraints, allocate complimentary services for frequency and voltage control of the grid, and manage the deviations. Additionally, it controls the electricity exchange with neighbouring systems and assesses the state of international connections.

The distribution system operator (DSO) or distributor carries the electricity between the transmission grid to the final consumers connected to the distribution grid. Traditionally, distributors’ only objective was to create and maintain the physical infrastructure between the transmission grid and the energy consumers. Nevertheless, now they must manage more significant amounts of information due to the smart meters. Therefore, distributors are now information providers and own an extensive advanced metering infrastructure to collect valuable information for the system’s proper operation.

Moreover, the surge of DER is increasing the complexity of operating these systems [27]. Hence, distributors must implement changes in their business vision to survive. Flexibility markets are one of the system’s new features available for the DSO. DSO-TSO coordination is essential to exploit flexible markets efficiently [28].

Another agent involved in the operation of the system is the market operator. The market operator’s main objective is to manage the electricity markets and ensure compliance with the obligations of all participants. Thus, this agent is in charge of coupling the market by matching the sell and buy offers. Researchers expect the creation of local electricity markets (LEM). LEM will implement local trading or peer-to-peer [20] and require their market operator to enable more dynamic electricity trading.

Finally, the regulator is responsible for ensuring competitiveness and transparency in the electricity sector. The regulator also gives its opinion on severe sanctions committed, coordinates market supervision and issues reports of various kinds, among other things.

3.4. Generators

Electricity generators’ main activity is to produce electricity for the consumers. They are in charge of building, operating and maintaining power plants that convert primary energy resources into electricity. These generators can sell their energy in the different electricity markets or on bilateral contracts.

Conversely, virtual power plants (VPPs) are new agents emerging to provide electricity alternatively to the traditional generator. VPPs are cloud-based distributed power plants that combine the capabilities of diverse DER to pool power generation and trade or sell electricity on an open market [29], behaving like a conventional generator [20]. VPPs are one of the most promising and effective methods in DER management [30] because VPPs make traditional and distributed power plants operate in harmony [31], creating positive synergies and interactivity [32]. In addition, energy consumers become active participants in the electricity system thanks to the VPPs [33].

The storage agent is another agent currently developing on the system. This agent can store energy and help balance and flatten the electricity load curve. Their fast response enables them to deliver load following, capacity mechanism, frequency response, or black-start capability. Storage is critical in power systems to ensure reliable, extensive renewable penetration [20].

4. Energy Markets’ Structure

The European electricity markets are divided into three blocks: first, the unorganised market consists of private parties that agree freely upon bilateral contract conditions. Second is the organised market, where the market operator manages standardised contracts. Figure 2 shows a general approach to how these organised markets occur from when the agents purchase energy to the moment of delivery.

4.1. Forward or Future Markets

The forward market consists of a daily session where agents negotiate operations from four years to two days before the exchange. The better the agents’ forecasts of the daily market price, the more efficient their operations in the futures market will be.

4.2. Daily Markets

The market operator manages the daily market, also known as the pool. The market operator requires all agents with enough generation capacity to submit bids for the daily market. Afterwards, the market operator orders the purchase bids from highest to lowest and the sale bids from lowest to highest. With this process, the demand is covered first with the cheapest and then with the most expensive bids until the point where the supply and demand curves intersect. In most European countries, the electricity market is a marginalist market.

4.3. Intraday Markets

Intraday markets offer the possibility to correct deviances from the original predictions and operations made in the daily market. This way, agents can thus avoid deviations that can lead to high costs. Several intraday market sessions occur daily, and each session will apply to different scheduling horizons. A second option for adjusting bids is the continuous contracting or Single Intraday Coupling (SIDC) market, which allows energy to be traded continuously between different European regions.

4.4. Operation Markets

The SO assigns adjustment services through market mechanisms, generating the operating markets. The adjustment services are processes used to guarantee the continuity and security of supply, maintaining controlled voltage and frequency values at all points in the system. These services are divided into three groups:

• Solution of technical constraints: these services aim to correct incompatibilities between the results of the different electricity markets and the technical capacity of the transmission grid to carry out physical energy exchanges;
• Ancillary services: SO use ancillary services to control the frequency and voltage at the various points of the transmission grid within safe limits [34]. For frequency control, the active power demanded by consumers must be equal to that produced by the various generators in the system, plus system losses. The delivery or consumption of reactive power controls the voltage;
• Unbalances management: this mechanism aims to balance large deviations (greater than 300 MW) between scheduled generation and forecast consumption at the closure of each intraday market session. This mechanism is essential as unbalances generate network costs by overloading components [35]. Flexibility services are increasing their presence in the balancing markets. Thus, DSOs and TSOs can acquire the flexibility to redispatch their grids and manage congestions. Schittekatte and Meeus [36] show that “flexibility markets can operate as intraday markets, rather than service markets”. In addition, LEM can also profit from flexibility-based services [28].

Flexibility and capacity markets have arisen from the rejection of traditional regulation favouring more competitive markets. Nonetheless, they are replicating traditional market vices due to mistakes in demand forecasting and opaque administrative processes [36]. These vices imply a loss of competitiveness and, ultimately, increase the final cost of electricity for end-users. Fixes in the regulatory process and improving RES power generation forecasting can alleviate, at least partially, this situation.

5. Enhanced Solar Generation Forecasting Market Values

We have classified the market values that enhance solar power generation forecasting offers as reactive, proactive or opportunistic.

Table 1. Overview of the market values of enhanced solar power generation forecasting.

Reactive Value
R1. Reduction of imbalances
R2. Reduction of grid feeding (maximise self-consumption)
R3. Multi-energy systems optimisation
R4. Competition regulation
Proactive Value
P1. Participation in intraday markets
P2. Participation in balancing markets
P3. Energy arbitrage
P4. Realistic prediction of generation to avoid congestion
Opportunistic Value
O1. Market power
O2. Congestion generation

offers an overview of the different market values. The reactive values refer to the increased reaction capacity of any agent to deviations from the original predictions. The proactive values relate to the increased power of agents to make beneficial decisions on the different electric markets. The opportunistic values are those opportunities for the agents to abuse their market power using the enhanced solar PV generation forecasting. We define market power as the capacity to raise and maintain the price of products or services above the level of a perfectly competitive market.

5.1. Reactive Values

The reactive value has a decisive role in improving the resilience of an energy system against deviations in the planned power production and the expected incomes. The ability to act according to real-life situations beyond the design parameters can significantly impact the installation’s economic performance. We have identified the following reactive values: reduction of imbalances, reduction of grid feeding (maximise self-consumption), multi-energy optimisation, and competition regulation.

5.1.1. Reduction of Imbalances (R1)

Imbalances generate high costs for generators, retailers and large consumers purchasing energy directly at the wholesale electric market. Imbalances are the deviations produced in the generation or consumption of the contracted energy. If a generation plant does not feed the accorded power, the difference will constitute an imbalance. Similarly, when a retailer or large consumer consumes more or less power than anticipated, they generate an imbalance. Finally, generators and retailers compensate economically for these imbalances, thus implying high costs for them.

Therefore, imbalances are undesired, and agents will try to minimise them as much as possible. Thus, improved forecasting can help understand the real-time generation and match it with the contracted energy flows for each hour. This way, generators and consumers can anticipate imbalances and get another chance to correct them.

5.1.2. Reduction of Grid Feeding and Self-Consumption Maximisation (R2)

When there is a solar PV system for self-consumption, the generated power can be consumed onsite or fed to the grid. However, consuming the power instead of feeding it to the grid is usually more cost-effective because of the price difference between purchasing and feeding electricity.

Thus, it would be optimal to adjust demand as much as possible to the generation to take profit as much as possible from the self-consumption installation. Although residential demands are generally inelastic [37], some industrial and large consumers can adapt their energy consumption to meet the generation. They can do that by scheduling processes, storing energy or taking profit from inertia in heating and cooling or air-conditioning. This value is especially relevant for off-grid installations where the energy supply must always meet demand.

Hence, enhanced forecasting is essential to anticipate the upcoming generation and act consequently. In addition to maximising self-consumption, accurate forecasting can flatten the grid’s demand curve and reduce the contracted power of prosumers.

5.1.3. Multi-Energy Systems Optimisation (R3)

Multi-energy systems (MES) are systems in which electricity, heat, cooling, fuels, transport, and others can interact optimally [38,39]. Thus, solar generation and other energy carriers can be optimised, especially dispatchable ones like hydro or biomass. Namely, MES employs other technologies when the solar generation leaves a gap between generation and demand. In addition, storage is considered in these systems to increase the system’s autonomy [40]. When there is an excess in the electric generation coming from PV installations, it can be stored. In addition, using price signals and demand flexibility measures can improve the performance of the MES [41].

Power generation forecasting is essential to optimise the MES. With accurate predictions, the MES operator selects the required actions effectively, improving the system’s performance technically and economically.

5.1.4. Competition Regulation (R4)

Even though RES are stated to increase competition and reduce market power, these technologies do not present per se the elimination of market power. However, there are two critical distinctions between conventional and renewable technologies: RES generation marginal cost is virtually zero, and RES plant availability is uncertain and intermittent [4]. Hence, if market power with fossil technologies resembled Bertrand competition (competition with prices), market power with RES relates more to Cournot competition (reduction in capacity bidding).

Control over the available power and agents’ behaviour towards it is needed to regulate competition. This control is especially relevant to monitoring large producers that could hold their capability to increase market prices. In addition, evidence suggests that climate data predictions are usually far less accurate than generators’ predictions with their onsite information [42].

Therefore, improved generation forecasting offers real-time data analysis allowing regulators and market regulators to understand the actions of generators, study the market and be more proficient in regulating competence. This market value is related to the opportunistic values explained later to restrain those abusive activities.

5.2. Proactive Values

The proactive value allows the agents to act proactively to achieve and maximise their goals instead of letting the system run by inertia. We have identified that the improved forecasting grants the following proactive values: participation in intraday markets, participation in balancing markets, energy arbitrage and realistic prediction of generation to avoid congestion.

5.2.1. Participation in Intraday Markets (P1)

Although daily and intraday markets set similar prices in general, there are moments of the year when generators sell electricity in intraday markets at moderately higher prices. Currently, for a PV generator, it is hard to know how much power can offer in the markets due to uncertainties in the generation forecast and the risk of generating imbalances. However, accurate power generation forecasting can help these installations to participate in intraday markets.

5.2.2. Participation in Balancing Markets (P2)

Power systems require ancillary services to ensure the stability of the grid. In this sense, different markets and services exist to ensure the system’s reliability. These markets comprise the secondary reserve, tertiary reserve, and imbalance management markets, which are managed by the Transmission System Operator (TSO). Nowadays, advances in IoT, Smart Grids and granularity control are opening these markets to RES and demand management [34].

We expect an increased need for ancillary services as a more variable renewable generation gets into the system [25]. Furthermore, there is considerable potential in opening to decentralised generation and demand management at the distribution level, as is currently under research [37].

Therefore, improving power generation forecasting helps to understand the future behaviour of generating facilities so they can participate in these markets at an individual level (in the case of large plants) or in aggregated terms through the combination of DER. VPP, aggregators or electricity retailers’ future development will mainly perform these participations in the markets. Moreover, the forecasting potential will increase if flexibility, energy or operating markets are developed at the local level in the future since these markets would function at closer times to dispatch.

5.2.3. Energy Arbitrage (P3)

The increasing need for generation and demand response will increase price volatility in electricity markets. As market prices reflect the conditions in the system, this arbitrage can help automatically keep the grid balanced and reduce the differences between supply and demand.

Again, active demand management and better solar predictions will allow agents to actively and accurately participate in these markets to arbitrage their energy.

5.2.4. Realistic Prediction of Generation to Avoid Congestion (P4)

Distribution grids present increasing penetrations of solar PV at the mid and low-voltage levels. These generation facilities that inject electricity into the system can alter its stability if they are not well managed and balanced. Moreover, the topology and vastness of distribution grids have made them more challenging to digitalise, monitor, and control. Moreover, the electrification of the energy demand and new electricity devices such as electric vehicles will further increase the need for a more proactive and precise operation.

Currently, DSO operates distribution grids manually in many countries, and many of their costs are related to replacement and new infrastructure needs. Therefore, the digitalisation and proactive management of the grid can reduce the need for early replacement and new assets. Distributors and DSOs bear these costs, but ultimately, they transfer them to consumers. Thus, improving forecasting can help DSOs proactively understand the grid’s situation in the following hours and reduce potential imbalances, voltage drops, or extreme grid situations before their occurrence.

5.3. Opportunistic Value

In a transition period to a new market structure and energy model, regulations can present flaws catching up with the technological evolution. These flaws generate opportunistic values that some agents could exploit to increase their profit without providing efficiency or actual value to the system. Hence, it is relevant to consider the possible opportunities that arise. This way, there can be room to act proactively before any agent takes advantage of it.

5.3.1. Market Power (O1)

Traditionally, large market concentrations and the usage of market power characterise energy and electricity markets. Thus, market power is associated with fossil and hydro generation. The capacity to hold production and raise prices has occurred in a system where generation capacity is known while marginal costs are not.

This lack of information will shift from costs to capacities in future power systems with increasing renewable generation. Renewable energy sources have uncertain capacities but known marginal costs (close to zero). In general terms, the transition from a fossil to a renewable system reduces the capacity of generators to exercise market power, but it does not suppress it. In particular, firms could withhold renewable generation output if they realise that their capacity is large or increase their bids among marginal prices [4]. In this sense, exchanging capacity information between firms could allow coordinated bids to increase prices by reducing the capacity offered or increasing the bid price.

As said, accurate power generation forecasting will increase the information available. Therefore, the available information could be quickly exchanged or understood to effectively exercise market power at all levels and in all markets.

5.3.2. Congestion Generation (O2)

As previously mentioned, the increasing penetration of both energy demands and energy generation capacity at the medium and low distribution systems will increase the complexity and management of these systems. DSOs incur costs to pay agents to increase, reduce, withhold demand or generate capacity to overcome imbalances and congestion. At some critical moments, these payments can increasingly rise as unsuccessful management may end up with overloaded transformers or lines. A specific way of market power in power systems has been the artificial creation of congestion. By creating congestion, the DSO needs to pay agents to manage it. Therefore, the same agents that can be creating the congestion are the ones that benefit from solving the issue.

Hence, accurate forecasting could inform agents to modify their generation or consumption patterns and generate congestions. Then, they can profit from their previously known solar capacity to answer the needs and requirements of the DSO, which would unnecessarily increase its management costs paid by consumers in the end.

6. Potential Benefits for the Agents

We have described many potential market values for improving solar power generation forecasting. Nonetheless, not all the agents will benefit from all the market values. In this section, we intend to identify the benefits for each agent. To summarise,

Table 2. Market values from which each agent can benefit.

Actor	R1	R2	R3	R4	P1	P2	P3	P4	O1	O2
Small Consumers		X	X
Large Consumers	X	X	X		X	X			X	X
Generators	X				X	X			X	X
Retailers	X				X	X
Aggregators	X	X	X		X	X			X	X
VPP	X		X		X	X			X
Storage							X		X
TSO								X
DSO								X
MO
Regulator				X

shows the market values for each agent.

6.1. Consumers

Small consumers benefit from the reactive values as they do not have much operative capacity or market power. Nonetheless, small consumers can profit from accurate forecasting by maximising self-consumption (R2) through changes in the demand to meet power generation. Similarly, it can help small consumers optimise multi-energy systems (R3) by adapting the consumption to the production.

However, accurate solar power generation forecasting is even more valuable for large consumers. They can profit from the same values offered to small consumers. At the same time, as large consumers can participate in the wholesale market, they can reduce their imbalances (R1) in the intraday markets when they detect deviations from the forecasted power self-generation. Moreover, large consumers have proactive market values available. Namely, they can participate in intraday markets (P1) to profit from potential higher prices in intraday markets. Likewise, large consumers can participate in balancing markets (P2). From the opportunistic point of view, they could also leverage market power (O1) and generate congestion (O2) by changing their consumption patterns.

6.2. Intermediary Agents

Retailers and aggregators act as intermediaries between the markets and the end-users. The retailers can reduce their imbalances (R1) if the demand from their customers is higher or lower than expected. They can also participate proactively in the intraday (P1) and balancing (P2) markets to obtain lower electricity prices.

Aggregators can take advantage of the same market values as the retailers (R1, P1 and P2). At the same time, as they can operate energy systems, they can also exploit the same market values as large generators. Thus, aggregators can maximise self-consumption (R2) and optimise multi-energy systems (R3) thanks to enhanced generation forecasting. They can also participate willingly in intraday (P1) and balancing markets (P2) to obtain better purchasing and selling prices for their energy. In addition, aggregators manage many consumption and generation sites, so they could change the patterns to exercise market power (O1) and generate congestion (O2).

6.3. Operators

Regulators include the TSO, DSO and the market operator. For the TSO and DSO, accurate solar power generation forecasting can help them avoid congestions (P4) under variable climatic conditions. However, we have not found any market value of enhanced forecasting for the market operator.

The regulator can use better generation forecasting to regulate competition (R4). If the regulator can predict how much power each installation generates, they can identify generators holding capacity to increase market prices.

6.4. Generators

The precise forecast allows traditional generators to reduce their imbalances (R1) by participating in intraday and balancing markets. They can also proactively decide to sell part of their generated power in the intraday (P1) or the balancing markets (P2), seeking higher prices. This capacity might also lead to abuses of their market power (O1) and the generation of congestion (O2).

As plants with distributed generation, VPPs can profit the same as generators and some extra features. VPPs can reduce their imbalances (R1), and as they operate multi-energy systems, accurate generation forecasting will help them optimise it (R3). To improve profitability, they can also access intraday (P1) and balancing markets (P2). Like traditional generators, precise predictions could lead to abuses of market power (O1). Nonetheless, as they consist of relatively small distributed generation plants, they will not likely generate grid congestions.

Finally, storage agents can buy or sell at any moment and exercise energy arbitrage (P3) to profit from the price differences during the day.

7. Case Study: The PROGNOSIS Project

To illustrate the concepts, we discuss in this work short-time PV forecast models and the potential benefits for the agents in the power system, and we discuss an actual case study called the PROGNOSIS project. The project target is to forecast PV power output for 1.5 h ahead. The model used for forecasting does not utilise any exogenous data from satellite data or specialised equipment. Instead, the inputs to the model are the power production of a dense multipoint grid of connected PVs [9]. First, researchers estimated clear-sky PV electricity production [43]. Hence, they integrated the continuous PV input into energy maps over various regions to calculate and predict the motion and development of cloud cover in time to measure the shadow clouds cast over PV installations [8].

The forecasting model uses a machine learning approach: a long-short-term memory (LSTM) network. Researchers used historical data in Cyprus from four years of PV data to train and test the model [44]. Results show that the trained model captures trends and fluctuations, providing good predictions (see Figure 3) with a Root Mean Square Error (RMSE) of 0.09394 with a standard deviation of 0.01616 [9].

PROGNOSIS is essentially a real-time decision-making tool for the energy sector, although it can be helpful in several industries. The resulting forecasting facilitates the decision-making for the visualization, management and optimization of microgrids and electricity systems.

Additionally, they have developed a database to collect and store all the relevant data [45] using open-data sources or software to follow EU standard protocols to integrate it with other databases. Therefore, it promotes democratizing decision-making over PV management as the data are open and available to any agent in the system. As it can be seen in the figure, the real-time prediction allows to use all the values described to participate in markets and optimise the responses and usage of decentralized Solar PV.

8. Conclusions

RES integration in our electricity systems poses various challenges. One of the most concerning is matching load generation with demand due to the intermittent and non-dispatchable nature of solar and wind energy sources. Thus, technologies like energy storage, demand-side flexibility and solar and wind forecasting are raising attention. Combining IoT, smart grids and big data analysis, accurate forecasting of short-term solar PV power generation is now possible, and in this work, we analyse its potential market value.

Results indicate that improving solar generation forecasting has a significant market value in a sector under a complex transition to becoming more decentralised, with near-zero marginal costs, and significantly digitalised. Namely, it can have reactive values like reducing imbalances and proactive values such as participating in intraday markets or exercising energy arbitrage. In addition, digital tools that allow automatic and rapid actions will be increasingly important. Electricity markets are evolving, and forecasting may provide more value than suggested. However, it can also create some opportunistic values that malicious agents in the energy system could exploit. Hence, policymakers and regulators must be well aware of these potential threats (for them) and design tools to avoid and control them.

Short-term solar PV power generation is an essential technique for the future energy markets in which non-dispatchable technologies have an important role. This work points to potential market values for different agents. Different agents will benefit from this analysis to put forward and develop their solar forecasting techniques, boost solar PV integration into the electricity system, and speed up the energy transition.