A Novel Approach for the Short-Term Forecast of the Effective Cloud Albedo

The increasing use of renewable energies as a source of electricity has led to a fundamental transition of the power supply system. The integration of fluctuating weather-dependent energy sources into the grid already has a major impact on the load flows of the grid. As a result, the interest in forecasting wind and solar radiation with a sufficient accuracy over short time horizons grew. In this study the short-term forecast of the effective cloud albedo based on optical flow estimation methods are investigated. The optical flow method utilized here is TV-L1 from the open source library OpenCV. This method uses a multi-scale-approach to capture cloud motions on various spatial scales. After the clouds are displaced the solar surface radiation will be calculated with SPECMAGIC NOW which computes the global irradiation spectrally resolved from satellite imagery. Due to a high temporal and spatial resolution of satellite measurements the effective cloud albedo and thus solar radiation can be forecasted from 5 minutes up to 4 hours with a resolution of 0.05 ◦. In the following there will be a brief description of the method for the short-term forecast of the effective cloud albedo. Subsequently evaluation results will be presented and discussed. Finally an outlook of further developments will be given.


Introduction
The power supply system is in a fundamental transition.The replacement of fossil fuels with renewable energies evolves with enormous speed.Weather-dependent energy resources such as wind and solar energy play a major role within this scope.Their integration into the grid has a huge impact on the load flows and for this reason the forecasts of solar radiation and wind have to be more precise including the short-term forecast up to 3-4 hours.Hence, a forecast based on satellite observations, is also referred to as nowcasting, depicts a reasonable choice for this issue.These forecasts show better results than the numerical weather prediction model (NWP) for the first few hours, for instance up to 3-4 hours for photovoltaic power forecasts (PV-forecasts) [1].Further, NWP runs with data assimilation need usually 3-6 hours of computing time.Thus, the results of the numerical weather prediction model are only available with a time delay of several hours, whereas satellite based forecasts are available near real time.
The temporal short-term variation of the solar surface irradiance is predominantly defined by clouds in Central Europe.Thus, an accurate short-term forecast of cloud properties is the most important step.The main challenge here is to derive the location and the shape of clouds for the next few hours from satellite data.In our work the optical flow method TV-L 1 provided by the OpenCV library is used for the calculation of cloud motion vectors and thus for the forecast of clouds [2].Optical flow is a widely used, evaluated and established technique for image pattern recognition within the scope of traffic, locomotion and face re-detection.An overview of different optical flow methods and its application is given by Sonka et al. [3].TV-L 1 and other optical flow methods are so far only sparsely used for the calculation of cloud motion vectors in the research field of meteorology.To the knowledge of the authors, one of the first applications has been the utilization of the optical flow for radar images, as described by Peura and Hohti [4].Also at the German Weather Service ("Deutscher Wetterdienst"), optical flow has been recently implemented for the short-term forecast of radar reflectivity.The success of the estimation of the optical flow of radar images indicates that the method could be successfully applied also to clouds in general.A promising candidate for the forecast of cloud properties is the effective cloud albedo (CAL).CAL is a satellite observable characteristic derived from the reflectivity of the visible channel of satellites.The advantages of the effective cloud albedo are numerous.It can be directly observed from space with the use of satellites, without the need of any model or other external information [5] (see section 2).For effective cloud albedo values up to 0.8 the cloud transmission is simply defined by 1 − CAL [5].Thus, the effective cloud albedo provides a clear and direct information on the cloud effect on the solar surface irradiance.The reflection of the Earth's surface is already filtered allowing optical flow to focus on clouds.Satellite observations enable the retrieval of the effective cloud albedo, and thus the solar radiation, with high spatial and temporal resolution and a large areal coverage.Further information concerning the retrieval of the effective cloud albedo can be found in Müller et al. [5].
Straightforward methods for the calculation of cloud motion vectors are based on the minimization of the root mean square error (RMSE) or the absolute difference between a shifted image in x-y direction at t 0 and the subsequent image at t 1 .The cloud motion is then defined by the shift in x-y direction which minimizes the RMSE or absolute difference between the images.This can be applied for various spatial scales and is called multi-scale-approach. Multiple scales lead to a dense vector field, however, more scales increase the needed computing time.
The cloud motion vector applied for the satellite weather at the German Weather Service [6] is an example of a straightforward multi-scale approach, which is based on the minimization of the absolute differences.Another example is the method of Schmetz et al. [7].Here a cross correlation method is used for the cloud motion vectors.In this method, image filtering, also known as slicing, is applied to enhance the highest cloud tracer suitable for tracking.This filtering leads to a relative low density of cloud motion vectors, which is a significant disadvantage for energy meteorology applications.NWCSAF uses a similar approach, namely a gradient method, to define the cloud edges and cross correlation for the calculation of the motion vectors [8].This methods leads also to a low density of cloud motion vectors and is therefore not appropriate for the utilization in the field of energy meteorology.Originally, the main application of satellite-derived cloud motion vectors was the use as wind fields in the data analysis for numerical weather prediction [7].Only a dense field of cloud motion vectors enables a forecast with large geographical coverage and high temporal resolution without data gaps.
In the last years, cloud motion vectors have gained significantly in importance within the scope of PV-forecasts and recently their relevance is also recognized for short-term forecasts of wind energy.A dense vector field of cloud motion vectors is usually needed for energy meteorology applications, for example for the forecast of solar surface irradiance.Neuronal networks are therefore widely used in order to gain a dense field accompanied with a fast computing time from high resolution satellite images.Once a successful training has been performed, the execution of the code is very fast.Voyant et al. [9] provides a review and overview of neuronal network methods applied within the forecast of solar surface radiation.A disadvantage of neuronal networks is their black box character.A neuronal network is, strictly speaking, only valid for the training framework, as only the behaviour of the training data sets can be reproduced by them.Application to other regions, periods or satellite instruments requires typically extensive retraining.Further, the black box character hampers a deeper understanding of the involved physics and physical reasons for the occurring uncertainties.
Optical flow methods might be a good alternative for the estimation of cloud motion fields.However they are not mentioned neither in the review of photovoltaic power forecasting performed by Antonanzas et al. [10], nor by the review of very short PV-forecasting with cloud modelling by Barbieri et al. [11].Other leading experts, for example Raza et al. [12] or Wolff et al. [1], do not mention optical flow methods by OpenCV as an option or alternative.However, the optical flow of satellite images has been used for the Geometric Accuracy Investigations of SEVIRI High Resolution Visible (HRV) Level 1.5 Imagery [13].Further, Simonenko et al. discussed the optical flow method TV-L 1 concerning the interpolation between observed cloud images, in order to improve the temporal information about convection volcanic ash plumes for instance [14].However, neither the short-term forecast of the solar surface irradiance nor its application is addressed.This shows that there are only few works about the application of optical flow methods for the forecast of cloud motion fields from satellite imagery and hence practically no works on satellite based short-term forecast of solar surface irradiance.Yet, the authors are aware of only one publication in which a multiple-scale optical flow method is applied within the scope of satellite based solar irradiance forecasts [15].The respective method is based on that and developed within the storm detection and nowcasting system Cb-TRAM [16,17].Unfortunately the details of the method are not well described and the software is not available in open access, which limits the scientific benefit of the work.Further, they applied the cloud motion vectors to cloud optical thickness and effective radii and not to cloud albedo.For the correct retrieval of cloud optical depth (COD) and effective radius r eff accurate information of the surface albedo (SAL) and the atmospheric composition are needed.Furthermore, simplifications in the radiative transfer are typically applied within the retrieval of the cloud optical depth and r eff .These items induce uncertainties in the estimation of the solar surface irradiance which can be avoided if the satellite observable CAL is used, as it does not rely on accurate information of the surface albedo, atmospheric composition or any radiative transfer model simplification [5].
Thus, to the knowledge of the authors the usage of the recent TV-L 1 method on the effective cloud albedo is a novel approach within energy meteorology and especially for the forecast of solar surface irradiance.Furthermore, this work is one of the first in which two optical flow methods of the OpenCV library are evaluated and optimized for the use of effective cloud albedo.The authors believe that the combination of the effective cloud albedo with TV-L 1 and SPECMAGIC NOW [18] is a new powerful method for the short-term forecast of solar surface irradiance.

Optical Flow Method
The optical flow is a pattern of apparent motion of image objects between two sequential frames caused by either the movement of the object or by the camera [19].Three-dimensional motion of objects can be projected onto a two-dimensional plane by calculating the optical flow.Its result is a vector field where each vector is a displacement vector showing the movement of pixels from first frame to second [20].Once computed, the optical flow can be used for a wide range of tasks [21].It may be applied for general image processing or more precise for motion detection, object segmentation, motion compensated encoding and stereo disparity measurements.On top of that it can be used to reconstruct three-dimensional motion of visual sensors and surface structures [20].
The optical flow works on two major assumptions which are first of all that the pixel intensities of an object do not change between consecutive frames and second of all that neighbouring pixels have similar motion.The assumption of constant intensity I(x, t) between two consecutive frames is valid for all of the following methods.It can be expressed as follows [20][21][22]: where x represents the location and t the time.After that, I(x, t) can be developed with a Taylor series to get the following functional: where δx δt = v is the velocity.This equation alone is not sufficient for the estimation of the optical flow though, because we have two unknown variables, x and t, and only one equation.At this point, a lot of methods for the determination of the optical flow were found to solve the problem by adding various conditions.There are methods such as block-based methods, discrete optimization methods or differential methods [23].The latter offers some more methods for instance the Lucas-Kanade-method or the Horn-Schunck-method [24,25] which were the first to extend the problem with additional conditions.Both are based on partial derivatives of the image signal or the sought flow field and higher-order partial derivatives.The most common algorithms for meteorological purposes concerning optical flow are by Farnebäck [26] and the duality based approach of the method TV-L 1 [27,28].
The optical flow estimation by Farnebäck [26] is the older one of the two named above.It uses two frames to estimate the flow.In a first step the neighbourhood of both frames has to be approximated by quadratic polynomials.This can be efficiently done by using the polynomial expansion transform.In a second step, the displacement has to be estimated with the help of the coefficients of the polynomial expansion f (x) where A is a matrix, b is a vector and c is a scalar.These coefficients are calculated from a weighted least squares fit to the signal value in the neighbourhood.Typically the centre point has the highest weight while the surrounding weights decrease radially.Large displacements can cause large errors in the estimated motion because the general assumption of Farnebäck is that the local polynomials at the same coordinates in the two images are identical except for a displacement.This issue can be solved by a multi-scale-approach which means that the algorithm starts at a coarse scale to get a rough displacement estimation and continue with finer scales to obtain a more accurate estimate [26].
TV-L 1 is a variational method based on the method by Horn and Schunck [25], however using other data and smoothness terms [28].Like the Farnebäck method, TV-L 1 uses two frames to estimate the optical flow.The TV-L 1 method is based on the minimization of a functional containing a data term using the robust L 1 -norm in the data fidelity term and a regularization term using the total variation (TV) of the flow [27].The method is based on the minimization of the following image-based error criterion: In this formula I 0 and I 1 denote the two image frames and u : Ω → R 2 denotes the disparity map which should be found with this method.Further, the disparity map u is the minimizer of the above mentioned criterion (see Equation ( 5)).The term φ(I 0 (x) − I 1 (x + u(x))) describes the image data fidelity and ψ(u, ∇u, . . . ) represents the regularization term.Moreover, λ works as a weighting factor between the data fidelity and the regularization term.If one selects φ(x) = x 2 and ψ(∇u) = |∇u| 2 , the result will be the Horn-Schunck-method.As the algorithm by Farnebäck, the TV-L 1 method uses scale-space approaches and on top of that coarse-to-fine warping to provide solutions for optical flow estimations with large displacements.Because the algorithm contains discontinuities it is more robust against noise than the classical approach by Horn and Schunck [25].
In the numerical implementation of both algorithms there are several parameters to adjust the estimation of the optical flow to the sort of data one is working with.This can range from the number of scales for the multi-scale-approach to smoothing factors and the number of iterations.However, the amount of parameters differs as well as the parameters of the two algorithms themselves.The algorithm of Farnebäck uses 8 parameters while the TV-L 1 method by Zach et al. uses 11, when you count the settings for the inner and outer iterations separately [26,28].For further information the reader may refer to the website of OpenCV [2].
After the calculation of the motion vectors for the optical flow estimation these vectors need to be applied to a data field to achieve a forecast at all.First of all, the motion vectors need to be extrapolated to yield a short-term forecast.In other words, the calculated motion vectors point in the direction where the pixel was coming from.To extrapolate the motion of the clouds these vectors need to be inverted to continue the motion in the given direction from the past.Second of all, a data field to start with is required.When the optical flow is calculated between the pictures of t = n − 1 and t = n in the past, the motion vectors are applied to the data at t = n and the result will be at t = n + 1.This process can be repeated as often as desired until a certain forecast time is reached at which the NWP delivers better results than the nowcasting.

The Heliosat Method
The effective cloud albedo can be derived from geostationary satellites by using the observed reflections of visible channels without the need for any further information.The effective cloud albedo is also referred to as cloud index by other authors [29][30][31].The visible channel at 600 nm from the Spinning Enhanced Visible and Infrared Imager (SEVIRI) on board of Meteosat Second Generation (MSG) is used for the calculation of the effective cloud albedo.The data is provided by EUMETSAT as rectified images of digital counts, capturing the signal of the reflection of the Earth atmosphere and surface.The location of the geostationary satellites was over the equator at 0 • longitude with a field of view up to 80 • N/S and 80 • E/W respectively.An example of this view is illustrated in Figure 1 for a clear sky reflection and effective cloud albedo case.The effective cloud albedo can be defined as the normalized difference between the all sky and the clear sky reflection in the visible range observed by the satellite.1 minus the effective cloud albedo defines the cloud transmission for values of the albedo between 0 and 0.8.For effective cloud albedo values above 0.8 this relation will be modified in order to consider the saturation and absorption effects in optically thick clouds [5].Due to the fact, that different luminance conditions may vary because of the sun-earth distance and the solar zenith angle the effective cloud albedo has to be corrected.Furthermore, the dark offset of the instrument has to be subtracted from the satellite image counts.The observed reflections are therefore normalised by application of the following equation: Here, D is the observed digital count including the dark offset of the satellite instrument.D 0 is the dark offset, which is the baseline value of the instrument in the absence of irradiance and therefore has to be subtracted.The sun-earth distance variation is taken into account by the factor f .Finally, the cosine of the solar zenith angle corrects the different illumination conditions at the top of the atmosphere introduced by different solar altitudes.The effective cloud albedo can be derived from the normalised pixel reflection ρ, the clear sky reflection ρ cs and the maximal cloud reflection ρ max as follows: Here, ρ is the observed reflection for each pixel and time and ρ cs is the clear sky reflection, which is calculated according to an approach discussed in Müller et al. [18] within the scope of spectral clear sky reflectance.The maximum reflection ρ max is determined by the 95th percentile of all reflection values ρ at local noon in a target region.It is characterized by high frequency of cloud occurrence for each month.This way, changes in the satellite brightness sensitivity are accounted for.All reflection types were corrected in the same manner using Equation ( 6).
Only the observed reflections are needed to derive the effective cloud albedo with the application of Equation (7).As a result, the effective cloud albedo is completely defined by the satellite observation with only one broadband visible channel needed.Accuracy and limitations of the method are discussed in [5].
The aim of the application of the optical flow onto the effective cloud albedo is to obtain a short-term forecast of the solar surface irradiation.First of all, the optical flow of the effective cloud albedo is used to displace the clouds and after that the solar surface radiation can be calculated.The solar surface irradiance is retrieved using the well established Heliosat relation between the effective cloud albedo and the solar irradiance, which is based on the law of energy conservation [29,31].As a consequence the basic relation between the solar irradiance and the effective cloud albedo is predominantly a linear relation given in the following equation: Here, SSI is the solar surface irradiance, SSI cs is the clear sky surface irradiance, which is derived with the clear sky model SPECMAGIC [32] and CAL is the effective cloud albedo, also called cloud index n in former publications [33].For effective cloud albedo values above 0.8 the above equation is modified in order to consider the saturation and absorption effects in optically thick clouds.The modification of the equation for small and large values of the effective cloud albedo is based on ground measurements and is described in more detail in Hammer et al. [31].The forecast of solar surface irradiance relies on the accuracy of the forecasted effective cloud albedo as can bee seen in Equation (8).As the clear sky variables are not forecasted it is the only quantity of interest for the present forecast study and thus for the following results and discussion.

Verification
For the purpose of verifying the optical flow results we calculated the absolute difference of the optical flow estimate and the measured satellite image.The unit of this result is % as it is the unit of the effective cloud albedo as well.This was done for all investigated cases.On top of that, three different error measures were calculated on the bases of the absolute difference which are the bias, absolute bias and root mean square error.The exact calculation of the error measures can be found in the following Equations ( 9)-( 11

TV-L 1 versus Farnebäck
As mentioned in subsection 2.1 there are two prominent methods to estimate the optical flow for meteorological purposes.To decide whether the Farnebäck or the TV-L 1 method works better for the effective cloud albedo we calculated the absolute difference of the optical flow estimate and the measured satellite image as described in subsection 2.3 for both methods.The parameter settings for this comparison were done by eye which is sufficient because differences can get small when setting the parameter values.The absolute difference is shown in Figure 2 and the absolute bias for this case and for another 10 exemplary cases can be found in Table 1.First of all, the higher absolute difference of the optical flow calculated with the method by Farnebäck is obvious.Moreover, the absolute bias equals 5.43 % for the Farnebäck method and it is equal to 4.37 % for the TV-L 1 method.Also the RMSE is higher with the use of the Farnebäck method, namely 9.53 % with the Farnebäck method and 7.59 % with TV-L 1 .Even though, this is just one example, we can approve that the TV-L 1 method provides the better results for all examined cases and this is why we decided to use this method for the estimation of the optical flow with the effective cloud albedo.
There are 11 different parameters in the TV-L 1 method DualTVL1OpticalFlow from OpenCV which can be modified to influence the result of the optical flow estimation.Some parameters like as well as the inner and outer iterations can affect the speed of your algorithm.is a threshold for the accuracy which will be achieved after the given number of iterations and thus has an impact on the precision but also on the speed of your algorithm.For the in section 1 mentioned multi-scale-approach the method needs the number of scales and the fraction or scale step by which the area is divided.Other parameters have an influence on the shape of the clouds, for instance τ, λ and θ.And in our case two of the parameters were less sensitive to the data and these were γ and the warping so the setting of these parameters didn't affect the results much.

Parameter optimization
The first task when estimating the optical flow is to change the parameters of the algorithm in a way that the optical flow for the favoured sort of data is optimal.For the short-term forecast of solar surface irradiance we used the effective cloud albedo as variable for the calculation.To optimize the parameters of the optical flow algorithm, we calculated the bias, absolute bias and RMSE as error measures over the area of Europe (-10 -25 • N, 36 -60 • E) .The results of the calculation of the absolute bias for the forecast times from 15 to 120 minutes are shown in Figure 3.There are shown two examples for the cases of 30th of September and 4th of October 2017.The left image shows the bias for the case of a situation where a front passes Germany and convective clouds form behind it and in Italy and the right figure shows a case of stratiform precipitation.As one would have expected, the error of the forecast grows with increasing forecast time.However, this growth does not have to be linear.Instead, the error growth decreases with time which leads to a root-function-shaped graph.This shape can be more or less bent, as can be seen in Figure 3 in the left plot, but never completely linear.To choose the optimal value for each parameter of the TV-L 1 method, the values had to be changed several times and the optical flow estimate with the parameter value which corresponds to the lowest absolute bias is the most optimal one.However, this does not always has to be as easy as it sounds.For some cases the parameter values have to be changed for each forecast time.Because this would be difficult to declare for every single case, there should be a better solution.As the differences between a case with optimal and second best values are small, the solution was to calculate the integral of the function seen in Figure 3 with the composite trapezoidal rule to get the overall optimum for the whole forecast.This was done for all 21 cases with a variety of weather situations between August and October 2017 (see Table 5).Despite the fact, that different weather situations were analysed, the choice of the parameter values was quite clear and therefore it is valid for all short-term forecasts of the effective cloud albedo.And again, the differences of the bias values for optimal and second best parameter values were small, but not negligible.The results of this optimization can be seen in Table 2.A common approach for numerical models as well as nowcasts is to choose the parameter values in dependence of spacial scales.In this study we examined the choice of the parameter value again but for shorter time periods concerning smaller spatial scales.The result was that the values presented in Table 2 perfectly fit for all forecast times.As can be seen in Figure 3, the plots do not show two or more different regimes which proofs the good choice of the parameters already.In other words, the function of the absolute bias against the forecast time is continuous and linearly growing.Moreover, it does not show jumps or features which would differ a lot from the observed shape.The number of cases where this separated choice of parameter values would be successful was too little to be efficiently implemented in the algorithm.On top of that, the achieved effect would be very small.

120-minute-forecast
In the following subsection, we will present two examples of a 120-minute-forecast based on the optical flow of the effective cloud albedo.The utilized algorithm is DualTVL1OpticalFlow from OpenCV with the in subsection 3.2 listed parameter values.These two examples were already introduced in the previous subsection.
The first case (see Figure 4) shows a quasi stationary front over Central Europe with convective clouds behind it on the westerly site.The front extends from north to south and moves towards the east.More convective clouds can be seen over southern Italy and the surrounding ocean.
The forecast figure can be easily recognized by its inward moving edge (Figure 4 left).Due to the given data set to which the optical flow is added to, there is no new information after shifting these cloud pixels to its calculated position.In other words, in an optical flow estimation all boundary conditions are set to zero.Besides that, the shown figure depicts a 120-minute-forecast, so the clouds usually move further than in a shorter time period.Moreover, the forecast figure can be found by comparing the cloud top structure.Due to the fact, that the new formation or dissipation of clouds cannot be displayed in the optical flow estimate, the overall structure of the clouds is softer.In the measured image the structure of the cloud tops especially in the area of the front is more irregular and  shows lots of details (Figure 4 right).All in all, the position and spatial extend of the cloud formation in the forecast fits quite well to the observations.The second case shown here was a different weather situation.Over the northern part of the chosen area there can be seen stratiform clouds which extend over large parts of Central Europe (see Figure 5).Again, there can be seen convective clouds over southern Italy and the Balkans.Compared with the observations (Figure 5 right) the overall structure of the cloud area looks quite similar.Nevertheless, the extension to the south is larger in the forecast because over the area of Austria and Switzerland the clouds dissipated in the time period of 120 minutes.Furthermore, on the southern tip of Italy there formed a few new clouds which cannot be seen in the forecast.To verify these short-term forecast results of the effective cloud albedo we calculated the absolute difference between the forecast and the corresponding satellite image at the same time.The verification results can be seen in Figure 6 for the above discussed cases (see Figure 4 and Figure 5).
In the case of 30th of September, the verification confirms the above mentioned quality of the forecast.Higher deviations can be seen in places with formation of convective clouds for example over the Mediterranean Sea near the southern tip of Italy or in the area on the western side of the front.There, the absolute difference can reach values up to 80 % in small spatial areas.Moreover, the change of the cloud top structure of the front seems to cause trouble for the algorithm as well.This can be explained through the changing intensity of each pixel throughout the forecast time.The bias is small for the whole area with bias = 1.98 % after 120 minutes.The other error measures are higher (absolute bias = 12.99 %, RMSE = 18.16 %) than the bias which shows that positive and negative deviations cancel out in the short-term forecast which is generally the case.All error measures for this case can be found in Table 3.
As for the case of 30th of September, the bias for the 120 minute forecast (bias = 1.84 %) is quite low for the case of 4th of October 2017.Also, the other error measures are higher so that in this case the positive and negative deviations cancel out, too (see Table 4).Most of the clouds especially in the stratiform precipitation area were forecasted very well and the verification shows only small deviations.Areas with high absolute differences can be found north of the stratiform clouds and in the South.Both situations can be explained through the formation of new clouds and the changing intensity of cloud pixels.This behaviour violates the criterion for the optical flow estimation which states that the intensity of the pixels have to be constant over time.This is, however, not fulfilled in all of the cases.Therefore, convective clouds pose a problem for the optical flow estimation.

Discussion
The applications of optical flow estimates are diverse.As shown in subsection 3.3 the utilization of optical flow estimation for a short-term forecast of the effective cloud albedo and hence of the solar surface irradiance shows promising results.Validation results reported in recent review publications by Voyant et al. [9], Antonanzas et al. [10] or Barbieri et al. [11] or rather publications by other leading experts, for example Raza et al. [12], Wolff et al. [1] or Cros et al. [34] do not provide any hints that the  application of the widely used neuronal networks lead to a significant better accuracy for cloud motion vectors.For example in Cros et al. [34] the RMSE of the 30-minute forecast of the effective cloud albedo is about 30 % for a neuronal network state of the art approach and a phase correlation method.Thus, the discussed optical flow method might be among the best approaches for cloud motion vector estimation.Moreover, the big advantage of the TV-L 1 approach, which has been optimised for the effective cloud albedo in this work, is the free access and transparent documentation of the method.However, no matter which satellite based method for cloud motion vectors is used, the limit of a good short-term forecast compared to NWP is approximately between 120 and 240 minutes because the forecast is only based on the optical displacement of pixels.Without physical equations this method does not lead to good forecast results after a certain time threshold.Comparisons with the numerical weather prediction models will follow in the future to provide more detailed information about the time when the accuracy of the NWP matches that of TV-L 1 .Further, we plan to investigate the benefit of rapid scan imagery which are available every 5 instead of 15 minutes.A currently known problem of satellite imagery methods is the new formation or dissipation of clouds in the forecast which can be caused for example by convection.This is obvious from the verification results (see Figure 6).As mentioned above, one criterion of the optical flow is that the intensity of image pixels has to stay constant between two consecutive frames.Due to the fact, that convective clouds form very fast this is clearly not fulfilled.Nevertheless, these areas where convection can be found are very small in comparison to stratiform clouds or fronts and are rather negligible.A common approach for short-term forecasts is the separation into sub-scales.Convection is a fast small-scale progress while pressure systems with fronts can extend up to 1000 km and exist for days.To cover both regimes the optimization process can be done for the first 60 minutes and the second 60 or more minutes.This was already done for the mentioned 21 cases and all in all one could say that it did not improve the forecast.From 21 cases there were only 4 of them in which a separately done optimization would make sense.Besides, the differences between the optimal and the second best parameter value are in the range of hundredth and thus negligible.The implementation of such a parameter change would just be too costly.

Conclusions
As the use of renewable energies as a source of electricity increased, the demand for more precise forecasts for wind and solar irradiance over shorter time horizons grew as well.A short-term forecast of solar surface irradiance can be obtained by optical flow estimation of the effective cloud albedo.For this purpose, we used the TV-L 1 algorithm by Zach et al. available from OpenCV [2,28].As can be seen in Figure 2 and Table 1, the method by Zach et al. for the estimation of the optical flow delivers better results than the method by Farnebäck for the purpose of a short-term forecast of the effective cloud albedo.Due to a high temporal and spatial resolution of satellite measurements the short-term forecast of solar surface irradiance can be gained from 5 minutes up to 4 hours with a resolution of 0.05 • .The performed experiments show that the use of TV-L 1 for the estimation of the optical flow works well for the short-term forecast of the effective cloud albedo and thus for the solar surface irradiance.Overall the error measures are small for the examined cases although the formation and dissipation of clouds in general pose problems for the optical flow estimation (see Table 6).One of the major assumptions for the optical flow estimation is that the intensity of pixels remains similar between two consecutive frames.However, this is not fulfilled when new clouds occur or grow.This is the case because the newly formed top of the cloud consists of smaller droplets and thus the effective cloud albedo is higher because more light is reflected.Moreover there can form bigger clouds or clouds at other positions than there was between the first two frames.Nevertheless, these issues take place on small scales and do not influence the regional forecast too much.In general, the results show that the approach is very promising.

Figure 1 .
Figure 1.Example of the clear sky reflection ρ cs (left) and effective cloud albedo CAL (right) for a 11 UTC slot in June 2005.

Figure 2 .
Figure 2. Plots of the verification of the optical flow estimate with the method by Farnebäck (left) and the TV-L 1 method (right) for a 15-minute forecast.The area of Germany is marked by the red frame.

Figure 3 .
Figure 3. Plots of the absolute bias of the effective cloud albedo against the foreceast time for the cases of 2017-09-30 at 13:00 UTC (situation with convection behind a front over Germany) and 2017-10-04 at 12:00 UTC (stratiform situation).The unit of the absolute bias is %.

Figure 4 .
Figure 4.A short-term forecast for 120 minutes of the effective cloud albedo for 2017-09-30 at 15:00 UTC can be seen in the left figure.The satellite image by MSG with the effective cloud albedo depicted for comparison is shown in the right image.

Figure 5 .
Figure 5.A short-term forecast for 120 minutes of the effective cloud albedo for 2017-10-04 at 14:00 UTC can be seen in the left figure.The satellite image by MSG with the effective cloud albedo depicted for comparison is shown in the right image.

PreprintsFigure 6 .
Figure 6.Verification plots of the optical flow method for the cases of 2017-09-30 at 15:00 UTC and 2017-10-04 at 14:00 UTC.It is shown the absoulte difference between the effective cloud albedo from satellite imagery and the effective cloud albedo from the short-term forecast for 120 minutes in %.

Table 1 .
Calculated absolute bias and RMSE for both optical flow methods, the one by Farnebäck and with TV-L 1 .The error measures were calculated for a 15-minute forecast over the area of Europe.These are 10 exemplary cases because the superiority of TV-L 1 can be clearly seen already and these values are sufficient to see the diffenrence between the perfomrance of the methods.

Table 2 .
List of parameter settings for the TV-L 1 method in the DualTVL1OpticalFlow algorithm by OpenCV.

Table 3 .
Values of the bias, absolute bias and RMSE for the case of 2017-09-30.The basis for the calculation is the area of Europe.

Table 4 .
Values of the bias, absolute bias and RMSE for the case of 2017-10-04.The basis for the calculation is the area of Europe.

Table 6 .
Mean values of the bias, absolute bias and RMSE for all investigated cases.The basis for the calculation is the area of Europe.