Multiplicative Decomposition Model to Predict UK’s Long-Term Electricity Demand with Monthly and Hourly Resolution

Abstract

The UK electricity market is changing to adapt to Net Zero targets and respond to disruptions like the Russia–Ukraine war. This requires strategic planning to decide on the construction of new electricity generation plants for a resilient UK electricity grid. Such planning is based on forecasting the UK electricity demand long-term (from 1 year and beyond). In this paper, we propose a long-term predictive model by identifying the main components of the UK electricity demand, modelling each of these components, and combining them in a multiplicative manner to deliver a single long-term prediction. To the best of our knowledge, this study is the first to apply a multiplicative decomposition model for long-term predictions at both monthly and hourly resolutions, combining neural networks with Fourier analysis. This approach is extremely flexible and accurate, with a mean absolute percentage error of 4.16% and 8.62% in predicting the monthly and hourly electricity demand, respectively, from 2019 to 2021.

1. Introduction

1.1. The Pressure to Get It Right

The stability of the UK energy market is subject to external pressures from long-term factors, such as the Net Zero target from the Paris Agreement, which sets out policies for decarbonising all sectors of the UK economy by 2050 [1], and geopolitical events like the Russia–Ukraine war.

The electricity supply sector was the biggest contributor to UK greenhouse gas emissions until 2014 [2]. With a steady increase in renewable energy since 2010 (Figure 1), the electricity supply sector dropped to fourth position in 2022 for UK greenhouse gas emissions, behind the domestic transport, buildings and product uses, and industry sectors.

The UK could potentially meet its electricity needs through wind and solar sources alone [3]. However, due to the intermittent nature of these sources, this approach would require investment in large-scale energy storage facilities [4]. National Grid published a pragmatic analysis of Future Energy Scenarios [5] with a focus on reducing the overall electricity demand and transitioning to low-carbon energy generation sources. A less carbon-intensive and more resilient grid is a credible option and would allow the UK to meet its 2050 Net Zero target.

Accurately forecasting electricity demand is key to building the future electricity grid, as the UK needs to be more resilient to these external pressures.

The forecast can be carried out on different timescales. In this study, we focus on long-term predictions (from 1 year and beyond), which are critical for strategic planning, such as deciding on the construction of new electricity generation plants or large-scale grid improvements.

1.2. Initial Observations

The time series data of UK electricity demand is provided by the National Grid ESO (Electricity System Operator) [6] with a resolution of one data point per 30 min (called a settlement period) from 2009 onwards. Figure 2a shows the raw data between 2009 and 2024, featuring both a long-term downward trend and cyclical variations on different timescales. Figure 2b shows the data averaged over 1 year (with a stride of 1 year), making the long-term downward trend clearly visible. Figure 2c displays the data averaged over 1 month (with a stride of 1 month), showing a clear seasonal pattern with a maximum of demand in winter and a minimum of demand in summer. Figure 2d shows the data averaged over 1 week (with a stride of 1 week), emphasising the weekly pattern, with weekdays having higher demand than weekends. Finally, Figure 2e shows the average of the data over 24 h (with a stride of 24 h), highlighting two clear peaks around 12 pm and 6 pm for both weekdays and weekends.

1.3. Literature Review

Several different approaches have been applied to forecast electricity demand in the literature. Given the pronounced cyclical patterns in the demand time series, approaches using Fourier analysis have been widely used.

A study of the monthly electricity demand in Spain [7] isolated the long-term trend in the data from the cyclic components. The trend was modelled using a simple neural network, with historical data from the previous 12 months. Cyclic components were modelled by fitting linear coefficients to six sinusoidal functions, whose frequencies were identified via the Discrete Fourier Transform (DFT) analysis of the time series. The overall predictions of electricity demand were obtained by summing the predictions from both models.

A similar approach was applied to forecasting demand in a fashion company [8]. The main difference from [7] was that the model for the cyclic components was based on the inverse Fourier transform using the amplitude, frequency, and phase of the DFT components of the time series. The model optimisation consisted of minimising mean absolute percentage error (MAPE) (Appendix A.4) by changing the number of Fourier components included in the inverse Fourier transform sum.

Besides long-term trend and cyclic components, Li et al. [9] also considered the short-term fluctuations from the time series. An Adaptive Fourier Decomposition (AFD) method was used to break down the time series into its unique features, and the authors showed that modelling irregularities as a distinct component of the signal decomposition adds value to the overall analysis.

A study of the hourly electricity demand in Turkey [10] proposed a descriptive model, where the trend and cyclic components were combined in a multiplicative manner. This model also included weekly variations within a year and hourly variations within a week. A second-order polynomial fit was used to model the long-term trend, and the model did not include a component for short-term fluctuations. The results provide medium- and long-term simulations of electricity demand with an hourly resolution. Hourly resolution is typically used for short-term predictions (1 h–1 week) to support the daily operation of electricity generation and distribution systems, but a long-term hourly prediction can provide greater accuracy in strategic planning. The model was, however, not tested on unseen data and therefore, prediction accuracy could not be confirmed.

For the UK specifically, an early study proposed by Hunt et al. [11] presented an additive model of the trend, cyclic, and irregular components to forecast energy demand by sector for residential, manufacturing, and transportation between 1971 and 1997. Independent variables like energy price, income, and temperature were included in an Autoregressive Distributed Lag model to study their direct influence on energy demand. Their analysis emphasised the stochastic nature of the trend, with its pace and even direction altering over time, showing that approximation by a linear time trend is not appropriate.

Additionally, Bowerman et al. [12] analysed the applicability of the additive versus multiplicative models, concluding that if seasonal variations in the time series are not constant, then a multiplicative decomposition is preferred to the additive decomposition.

Furthermore, Iftikar et al. [13] proposed a day-ahead electricity demand forecasting model based on Nord Pool power market data that decomposes the time series into long-term, seasonal and stochastic components using autoregressive models. A similar model was proposed in [14] for the monthly forecast of electricity consumption in Pakistan. A different approach, solely based on a feed-forward neural network and transfer learning, was proposed in [15] to provide a short-term load forecasting of day-ahead electricity demand for 27 European countries.

Finally, Fields et al. [16] provided a long-term forecast of electricity demand in Sierra Leone for capacity-building purposes using a model proposed by the International Atomic Energy Agency that integrates energy intensities from different sub-sectors by energy and fuel type, together with key drivers, such as GDP growth, population trends, and efficiency improvements.

1.4. Our Proposed Approach

In this study, we propose a multiplicative decomposition model for UK electricity demand that accounts for the following components:

• Long-term downward trend, as shown in Figure 2b.
• Cyclic components: Seasonal, weekly, and daily trends, as shown in Figure 2c–e.
• Short-term fluctuations that are not taken into account in any of the components above.

We apply the proposed model to both the monthly and hourly electricity demand data. The cyclic components of the monthly data are modelled using the approach presented by Fumi et al. [8]. The non-cyclic components are modelled using a neural network that includes independent variables to improve prediction accuracy. The individual components are then multiplied to obtain an overall forecast for electricity demand. This approach is explainable and flexible, with separate models adapted to each component of the signal. This combined approach generates a long-term prediction of the electricity demand with hourly resolution, achieving an MAPE of 8.62% for the testing period 2019–2021. To the best of our knowledge, this is the first attempt to combine prediction methods in a multiplicative model to achieve genuine long-term forecasting at an hourly resolution, and it has not previously been applied to the UK electricity demand.

Table 1. Summary of the key methodologies that informed the approach presented in this paper.

Reference	Method	Input Variables	Output Variable
González-Romera et al. [7]	Additive decomposition model: - Fourier series for the fluctuations, - Neural network (multi-layer perceptron) for the trend	- 5-year historical data for the Fourier model - 12 months past values for the neural network	Monthly electric energy demand in Spain—one-month-ahead prediction
Fumi et al. [8]	Fourier series	4-year historical sales data	Weekly sales of a medium- to large-sized Italian fashion company
Filik et al. [10]	Multiplicative decomposition model: - Polynomial function for the trend, - Fourier series for the weekly variations, - 2D discrete cosine transform for the week template	- Entire dataset (1982–2007) for the polynomial function and Fourier series, - Average hourly shape of 52 weeks in the year 2002 for the week template	Hourly forecasting of long-term electric energy demand in Turkey—descriptive model
Our proposed approach	Multiplicative decomposition model: - Fourier series for the fluctuations, - Neural network (multi-layer perceptron) for the trend and irregularities components	- Historical data from 2009 to 2018 for the Fourier model - Real GDP, average temperature, percentage renewables, electricity CPI, housing efficiency, number of new builds, population growth + 13 months past values for the neural network	Hourly forecasting of long-term electric energy demand in the UK—true long-term predictions

summarises methodologies from the key papers that informed this study and contrasts them with our proposed approach.

In addition, we compare our results with a traditional stochastic time series model, namely a SARIMA model. We show in Appendix C that even though a SARIMA approach offers an equally comprehensive modelling of trend, cycle, and random components, our approach is superior in terms of the achieved accuracy, in particular when predicting electricity demand with hourly resolution.

As shown in Figure 3, we implemented a structured computational pipeline to ensure a systemic progression from raw data acquisition to validated results.

This paper is organised as follows: in Section 2, we describe the electricity demand data and other datasets used as input to the model, such as the average temperature in the UK and housing energy efficiency. In Section 3, we present a detailed description of the main model proposed in this study, and in Section 4, we show the main results. In Section 5, we discuss the results and main conclusions of this study. Comparison of our model predictions with the ones obtained with a SARIMA model has been detailed in Appendix C.

2. Selection of Variables and Data Description

2.1. Dependent Variable: Electricity Demand

2.1.1. Data Selection

We accessed the time series data of UK electricity demand from the National Grid ESO (Electricity System Operator) [6], which is provided with a resolution of one data point per 30 min (called a settlement period) from 2009 onwards.

The dataset contains several measures of electricity demand [6], including the following:

• The National Demand, defined as the sum of electricity generation monitored by the National Grid on the transmission network via metering.
• The Transmission System Demand (TSD), which is the National Demand plus the additional generation required to meet station load (the power required by the generators themselves to generate electricity), storage pumping (to operate pumps at hydropower plants), and interconnector exports (power exported out from the UK).
• Embedded Generation Demand: Most of the solar and wind generation is embedded into the distribution network and not metered by National Grid, and therefore, the embedded generation demand must be estimated. The National Grid maintains calibrated models to estimate solar and wind electricity generation using weather data and data collected from selected sites.

Embedded generation effectively lowers the demand required from the transmission network. The steady increase in renewable energy, as shown in Figure 1, results in an increasing gap between the curves TSD only and TSD + embedded generation in Figure 4. An accurate measure of the UK electricity demand should include both Transmission System Demand and embedded generation. Therefore, we considered the sum of Transmission System Demand plus embedded generation Demand as our dependent variable. Figure 4 shows the time series for National Demand, Transmission System Demand (TSD) and TSD plus embedded generation.

2.1.2. Data Pre-Processing

Dealing with Zero-Demand Periods

The Transmission System Demand dataset contains settlement periods of zero demand. Since the demand for electricity is strictly positive, it seems to be due to errors in the data capture process, as it affects 48 consecutive settlement periods at different points in time (see

Table 2. Tally of periods with zero demand.

Date	Number of Settlement Periods with Zero Demand
29 August 2009	46
30 August 2009	2
9 July 2010	46
10 July 2010	2
13 July 2010	46
14 July 2010	2
29 May 2012	46
30 May 2012	48
31 May 2012	48
1 June 2012	48
2 June 2012	48
3 June 2012	48
4 June 2012	2
11 June 2012	46
14 October 2012	1

) with a major anomaly in 2012, having a span of 290 consecutive periods of zero demand.

We selected the average as the interpolation method for those settlement periods of zero demand. We observed in Figure 2e that the electricity demand on weekdays significantly differs from that on weekends. Therefore, we chose to interpolate missing weekday values from the nearest available weekday values before and after, and following the same procedure for weekend values. Notice that only 0.18% of the total number of data points had zero demand. Hence, the specific choice of interpolation method does not affect the main results of this study.

For days with zero demand for either 46 or 48 settlement periods, we replaced the whole day demand with an average of the first matching working day/non-working day with non-zero demand right before and right after. For example, 11 June 2012 is a working day that has a total of 46 missing settlement periods. These missing values are interpolated by calculating the average of the settlement periods from 8 June 2012, since it is the prior working day, and the settlement periods from 12 June 2012, which is the next working day.

For days with zero demand for only one or two settlement periods, we replaced the demand for these periods with an average of the first period with non-zero demand right before and right after.

Dealing with Summer/Winter Time Change

The summer/winter time change artificially creates days with 46 periods at the end of March each year and 50 periods at the end of October.

To ensure that the time series is regularly sampled with all days having 48 settlement periods, we imputed periods 47 and 48 by the average of period 46 and period 1 of the next day. For the days with 50 settlement periods, we replaced the value for period 48 with an average of periods 48, 49 and 50.

Descriptive Statistics of Half-Hourly Electricity Demand Data

Following the treatment of missing values, the dataset of UK electricity demand contains 262,944 data points at 30 min intervals from 1 January 2009 to 31 December 2023. The electricity demand is reported in MW per half hour.

Although statistical analyses identified several data points as outliers (outliers represent 0.08% of the total number of data points using the Interquartile Range method), inspection of the data indicated that these values were consistent with the expected range of natural variation and were therefore retained in subsequent analyses. The statistical characteristics of the UK electricity demand data are summarised in

Table 3. Descriptive statistics of half-hourly electricity demand data.

Statistics	Electricity Demand (MW)
Max	60,672
75%	40,329
Median	34,776
25%	29,220
Min	18,356

.

2.2. Independent Variables

We considered a set of independent variables to model the downward trend shown in Figure 2b and short-term fluctuations. Many factors could influence the electricity demand, such as temperature, economic factors, adoption of energy-efficient technologies, and population growth. But identifying all the relevant explanatory variables is challenging because some of these factors may not be directly measurable, and also because the data sources for some other variables do not share the same sampling granularity or time span as the electricity demand dataset.

Table 4. List of potential independent variables.

Independent Variable	Data Resolution	Context
Bid price [17] (GBP per megawatt-hour)	Half-hourly, 27 March 2001– 13 June 2024: 406,999 data points	The UK energy market is a so-called deregulated market where the price of energy is governed by the laws of supply and demand. In addition to contracts for the supply of electricity agreed between producers of electricity and the National Grid, there is also a short-term bidding process to keep the grid in balance every 30 min. The bid price can be quite volatile and is eventually reflected in the price paid by the consumers. We selected this factor to explore whether the bid price influences consumers’ behaviour: using less electricity to minimise expenses. The bid price was broadly stable with low volatility until 2020, after which both prices and volatility rose sharply, peaking in 2022 before easing but remaining above historical norms. SBP represents the System Buy Price and SSB, the System Sell Price. Since Thu 5 November 2015, the SBP and SSP are identical at each period.
Consumer price index (CPI) for electricity [18] (GBP)	Yearly, 1970–2022: 53 data points	The Department for Energy Security & Net Zero produced a historical time series of the electricity component for the consumer price index (CPI) in real terms. This is a proxy for the change in the electricity net selling value to all consumers, indexed on the year 2010 and integrating inflation. The logic for selecting this factor is the same as for the bid price: the consumer may be influenced by the price of electricity. Bid price and CPI are, however, different measures. The CPI rose in the 1970s–1980s, fell steadily through the 1990s, then began recovering in the 2000s, and surged sharply after 2020 to record highs.
Percentage renewables [19] (Percentage of electricity produced)	Half-hourly, 1 January 2009– 29 May 2024: 270,123 data points	In terms of mode of operation, the wind and solar energy sources have the advantage that no additional energy is required to operate the plant. The station load mentioned earlier (Section 2.1) is reduced when a larger portion of electricity is sourced from renewable sources. In addition, a higher percentage of renewables in the electricity mix may raise awareness of a more mindful consumption of electricity and influence consumers’ behaviour. Based on these considerations and the fact that the percentage of renewables has been steadily increasing across the time span considered (see Figure 1), we selected this factor to explain the decreasing trend in electricity demand in an inverse relationship.
Average temperature [20] (Celsius Degrees)	Monthly, January 2009– December 2022: 168 data points	Climate change affects countries in different ways, and in the UK, the yearly average temperature has been increasing steadily from 8.9 degrees in 2010 to 9.8 degrees in 2021 (detailed calculations are available in the supporting coding notebook on independent variables). This may impact the amount of heating required during winter while not yet creating a significant need for cooling in the summer. Hence, it may contribute to an overall decrease in electricity demand. The average temperature in the UK has been weighted by the population density of London, Birmingham, Manchester, Leeds, and Glasgow. Additional information includes the following: List of the UK’s largest agglomerations [21]. List of London boroughs [22]. Population density per Local Authority [23]. Local Authority Districts [24].
Real GDP [25] (USD)	Yearly, 1993–2022: 30 data points	The Gross Domestic Product (GDP) is often used as an indicator to explain consumers’ behaviour, with an increasing GDP often correlated with an increasing electricity demand. In our case, given that the electricity demand is decreasing, an increase in GDP may indicate that consumers can make a choice for more energy-efficient products. We selected this factor as it is traditionally used in studies on electricity demand. It was increasing linearly until 2007, with a first anomaly showing a plateau in 2008, followed by a drop in 2009. The GDP resumed its increasing trend after 2009 until a second anomaly with a sharp drop in 2020. The term ‘real’ indicates that the GDP is inflation-adjusted.
Population growth rate [26] (Percentage per year)	Yearly, 2009–2023: 15 data points	While the UK population has been constantly increasing since 2009, a more interesting indicator to follow is the population growth rate. Since 2012 the UK population has been increasing less each year, with a steep drop in 2020–2021, recovering from 2022 onwards. We selected the population growth as an indicator of reduced demand on the housing market and, therefore, on the domestic energy demand. The growth rate is calculated as the successive differences in the population data.
Gas demand [27] (GW per hour)	Yearly, 1998–2023: 26 data points	According to the 2021 Census [28], mains gas central heating was by far the most common way that households heated their homes. Around three in four households (74%) in England and Wales said it was their only central heating source. Looking at gas demand will help understand whether the decrease in electricity demand is due to fuel switching. The UK gas demand is irregular with a broadly decreasing trend since 2001. The total gas demand includes transformation (electricity generation + heat generation), energy industry use, losses, and final consumption. Electricity generation from gas is already included in the electricity demand data; to avoid double accounting, we removed the electricity generation from the total gas demand.
Number of new builds [29] (no unit)	Yearly—financial year, 2009–2022: 14 data points	Through building regulations for new homes, the UK government aims to significantly reduce carbon emissions by focusing on improving heating, hot water systems, and reducing heat waste. These regulations have been increasingly ambitious. Part L Volume 1, released in 2021, focused on minimum levels of efficiency or performance in a building’s design, construction, and services. An uplift to Part L released in June 2022 mandated a further reduction of carbon emissions by at least 31% compared to previous regulations. These measures contribute to reducing the demand for electricity. The number of new builds has been increasing since 2009, with a period of sharper increase between 2013 and 2015.
Measure of housing energy efficiency [30] (no unit)	Yearly—5-year rolling, 2010–2020: 11 data points	Efficiency of existing housing is being improved, the same as for new builds. The Energy Company Obligation (ECO) [31] is a government energy efficiency scheme for low-income and vulnerable energy customers, which aims to reduce energy bills and carbon emissions. ECO measures, started in 2013, include cavity wall, solid wall and loft insulation, smart heating controls, and micro-generation such as air source heat pumps and photovoltaics. These measures contribute to reducing the demand for electricity. The Energy Performance Certificate (EPC) is a measure of the energy efficiency of a building, based on its features such as the building materials used, heating systems and insulation. The average EPC ratings were flat or declining until 2018, but have since improved steadily, peaking around 2022 before levelling off. Each data point is set to the middle of the 5-year rolling period.

lists a set of variables that could potentially be used to model the downward trend. Table 4 also explains why each variable has been considered, and it summarises the data resolution and source(s) of each dataset.

2.3. Selection of Independent Variables: Correlation Analysis

In order to select the independent variables to be used to predict the downward trend in electricity demand, we studied the correlation between those variables and electricity demand. The results obtained with Spearman’s correlation coefficient [32] are shown in

Table 5. Spearman’s correlation coefficient between each of the independent variables listed in Table 4 and the electricity demand downward trend (Figure 2b). The underlined numbers indicate statistically significant results with a significance level of 5%.

Independent Variables	Correlation with the Signal’s Yearly Trend
Bid price	0.61
Electricity CPI	−0.95
Percentage renewables	−0.95
Average temperature	−0.83
Real GDP	−0.80
Population growth	0.92
Gas demand	0.57
Number of new builds	−0.99
Housing efficiency	−0.72

.

All independent variables, excluding gas demand, are significantly correlated with the downward trend in the electricity demand. Percentage renewables, population growth, number of new builds and electricity CPI are highly correlated with a correlation coefficient greater than 0.9 in absolute numbers.

To understand how the independent variables considered are correlated to each other, we calculated Spearman’s correlation coefficient between all pairs of independent variables (

Table A1. Values of Spearman’s correlation coefficient between independent variables. The underlined numbers indicate statistically significant results with a significance level of 5%.

A—Bid price	1	-	-	-	-	-	-	-	-
B—Electricity CPI	−0.555	1	-	-	-	-	-	-	-
C—Percentage Renewables	−0.482	0.945	1	-	-	-	-	-	-
D—Average temperature	−0.527	0.809	0.845	1	-	-	-	-	-
E—Real GDP	−0.572	0.827	0.891	0.682	1	-	-	-	-
F—Population growth	0.555	−0.891	−0.891	−0.745	−0.745	1	-	-	-
G—Gas demand	0.545	0.6	−0.464	−0.655	−0.427	0.518	1	-	-
H—Number of new builds	−0.6	0.955	0.955	0.809	0.809	−0.936	−0.564	1	-
I—Housing efficiency	0.252	0.640	0.650	0.388	0.491	−0.827	−0.159	0.715	1
	A	B	C	D	E	F	G	H	I

in Appendix A.1). To visualise those correlations, we built a network diagram using the obtained values, in such a way that two variables are shown to be connected if their Spearman’s correlation coefficient is significant and larger than 0.7 in absolute value (see Figure 5). It provides a visualisation of groups of variables sharing many connections and variables sharing only a few connections.

Accordingly, we extracted the following groups of variables:

• Real GDP, average temperature, percentage renewables, and electricity CPI.
• Housing efficiency and number of new builds.
• Bid price.
• Population growth.
• Gas demand.

We considered those groupings of variables as input to the combined long-term and short-term variability components of the model of electricity demand (see Section 3.1.5) instead of variables considered one by one, so as to avoid the effect of confounding variables. The sensitivity study protocol is detailed in Appendix B.1 with the groups of variables listed in

Table A3. Groups of variables selected for sensitivity study.

	Group Name
	Variables 1	Variables 2	Variables 3	Variables 4	No Var
Percentage renewables	✓	✓	✓	✓
Population growth		✓	✓	✓
Real GDP	✓	✓	✓	✓
Number of new builds			✓	✓
Electricity consumer price index	✓	✓	✓	✓
Average temperature	✓	✓	✓	✓
Dwelling efficiency			✓	✓
Bid price				✓
Gas demand				✓

.

3. Multiplicative Decomposition Model

3.1. Prediction of Monthly Electricity Demand

Here we propose a multiplicative decomposition model to describe monthly electricity demand. The raw UK electricity demand data from National Grid ESO is provided with a time resolution of 30 min, as mentioned in Section 2.1, i.e., the time interval between two consecutive data points is equal to

$$∆t^r=30 \mathrm{m}\mathrm{i}\mathrm{n}$$

(1)

(one settlement period). We denote the corresponding time series as $Y_i^r$, where $i=1, \dots , N^r$, and $N^r$ is the total number of data points considered. From this time series, we construct the monthly electricity demand time series $Y_i^m$ by adding all the values $Y_i^r$ corresponding to a given month, i.e.,

$$Y_i^m=\sum _{j=1}^{N^s\left(i\right)}{Y}_j^r$$

(2)

where $N^s\left(i\right)$ is the total number of settlement periods in the ith calendar month, $i=1, \dots , N^m$, and with $N^m$ denoting the total number of months considered. The time interval between two consecutive data points $Y_i^m$ and $Y_{i+1}^m$ is

$$\mathsf{\Delta }{t}^m=1 \mathrm{m}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{h}.$$

(3)

Figure 6 presents the standard deviation of the monthly electricity demand time series across multiple years, revealing non-stationary behaviour. In such cases, a multiplicative decomposition model is more appropriate than an additive one, as it better captures the changing variability relative to the level of the series [12].

Therefore, we propose the following model to describe the monthly electricity demand:

$$Y_i^m=T_i^m\prod _{j=1}^{P^m}{C}_{j,i}^m{ I}_i^m i=1, \dots , N^m$$

(4)

Here, we assume that the monthly electricity demand signal can be factorised into a long-term trend component $T_i^m$, a number $P^m$ of cyclic components $C_{j,i}^m$, and a short-term variability component $I_i^m$.

To implement the monthly multiplicative decomposition model, we use the workflow detailed in following section with illustrations of the key steps. This is followed by detailed explanations of each step.

3.1.1. Workflow to Implement the Monthly Multiplicative Decomposition Model

• Calculate the trend by smoothing the signal with a Gaussian low-pass filter with a kernel of radius equal to 12 months, Figure 7. Divide the signal by its long-term trend to obtain the cyclical components (detrended signal). Divide the detrended signal by its average to obtain a scaled detrended signal with cycles centred around 1.

2.

Fit an inverse Fourier transform model to the scaled detrended signal by finding the number of Fourier components that minimises the error between the modelled and actual detrended signal. The result is the predicted cyclical component, Figure 8. Divide the signal by the predicted cyclical component to obtain the combined long-term trend and short-term variability component.

3.

Plot the Auto-Correlation Function (ACF), Figure 9, for the combined long-term and short-term variability component obtained in step 2 to check for visible patterns. If there are still cyclical components present, repeat steps 1 and 2 on the signal obtained in step 2; otherwise, proceed to step 4.

4.

Model the combined long-trend and short-term variability component with a neural network, Figure 10.

5.

Assemble the overall prediction, Figure 11, by multiplying the predicted cyclical components in step 2 with the predicted long-term trend and short-term variability components in step 4.

3.1.2. Detrending the Signal—Step 1 in Section 3.1.1

We applied a Gaussian low-pass filter, with a kernel of radius equal to 12 months, to $Y_i^m$ to obtain an estimate ${\tilde{T}}_i^m$ of the long-term trend of the signal. Several other methods to identify the long-term trend were tested and compared in Appendix A.2, including different values of the kernel radius for the Gaussian filter. The Gaussian low-pass filter with radius 12 achieved the best goodness of fit (see the results in Appendix A.2).

Once the long-term trend ${\tilde{T}}_i^m$ was estimated, we divided the signal $Y_i^m$ by it to obtain a detrended signal that allows for estimating the cyclical components in the next step.

3.1.3. Fourier Model of Cyclical Components—Step 2 in Section 3.1.1

We modelled the periodic patterns present in the detrended signal $Y_i^m/{\tilde{T}}_i^m$ by means of its Fourier analysis ([33], pp. 203–229).

The detrended signal $Y_i^m/{\tilde{T}}_i^m$ was first scaled by its average so that the cycles were centred around 1 and could be subsequently incorporated into the multiplicative decomposition model.

Secondly, the Discrete Fourier Transform (DFT) of the signal was calculated (see Appendix A.3), and then the Fourier components were sorted by decreasing order of amplitude, with the zero frequency kept at the first position. After that, a partial reconstruction of the signal using the inverse DFT was obtained by using only a subset of the Fourier components: the sorted Fourier components were incorporated one by one, and every time, the error between the actual detrended signal $Y_i^m/{\tilde{T}}_i^m$ and the partially reconstructed one was calculated using the mean absolute percentage error (MAPE) and the mean absolute scaled error (MASE) (defined in Appendix A.4). By plotting the MAPE against the number of Fourier components (see Section 4.1.2 for detailed results), we could visually identify a minimum, suggesting an optimised model. We made sure that at least two Fourier components were included in the reconstruction: the zero and the first non-zero frequency components. This procedure led to the first cyclic component of the model as follows:

$${\tilde{C}}_{1,i}^m=A_{\mathrm{1,1}}^m+\sum _{k=2}^{n_1^m}{A}_{1,k}^m\mathrm{c}\mathrm{o}\mathrm{s}(2\pi {\nu }_{1,k}^m\mathsf{\Delta }{t}^{m}i+{\varphi }_{1,k}^m)$$

(5)

where $A_{\mathrm{1,1}}^m$ is the amplitude for the zero frequency component; $A_{1,k}^m$, ${\nu }_{1,k}^m$, and ${\varphi }_{1,k}^m$ are the amplitude, frequency, and phase of the k-th component, respectively; and $n_1^m$ indicates the number of Fourier components for the first cyclic component of the model.

We observed that the modelled cyclical component may present a time shift compared to the signal (see Section 4.1.2 for detailed results). In order to correct the time shift, we set the number of Fourier components arbitrarily to 5 or 6, and we calculated the MAPE for each value of the time shift. We then selected the time lag that corresponded to the minimum MAPE and applied that time lag to the obtained cyclical component.

After that, we divided the original signal $Y_i^m$ by the estimated first cyclical component ${\tilde{C}}_{1,i}^m$ to obtain the combined long-term trend and short-term variability component.

3.1.4. Review Remaining Periodicities—Step 3 in Section 3.1.1

Before proceeding to the next step, we checked whether there were still any prominent cycles present. This is because it is possible that not all cyclical components are removed by the previous step due to the usage of a limited number of Fourier frequencies and the main frequencies masking smaller ones.

In order to do this, we took the signal divided by the estimated first cyclic component $Y_i^m/{\tilde{C}}_{1,i}^m$ from step 2 and divided it by its long-term trend ${\tilde{\mathcal{T}}}_i^m$, once again estimated by a Gaussian low-pass filter. We then plotted the Auto-Correlation Function (ACF) of $Y_i^m/\left({\tilde{C}}_{1,i}^m {\tilde{\mathcal{T}}}_i^m\right)$ and visually identified any periodicities. The major frequencies could then be identified using the Fourier spectrum. When periodic patterns were still visible in the ACF plot, we applied the same steps 1 and 2 detailed in this section to the signal $Y_i^m/{\tilde{C}}_{1,i}^m$ to obtain the second cyclic component as follows:

$${\tilde{C}}_{2,i}^m=A_{\mathrm{2,1}}^m+\sum _{k=2}^{n_2^m}{A}_{2,k}^m\mathrm{c}\mathrm{o}\mathrm{s}(2\pi {\nu }_{2,k}^m\mathsf{\Delta }{t}^{m}i+{\varphi }_{2,k}^m)$$

(6)

This process was repeated until no pattern was visible on the ACF plot for the detrended signal divided by all cyclic components $\left(Y_i^m/{\tilde{\mathbb{T}}}_i^m{\displaystyle {\prod }_{j=1}^{P^m}}{C}_{j,i}^m\right)$, where $P^m$ denotes the total number of cyclic components used.

3.1.5. Modelling the Combined Long-Term Trend and Short-Term Variability Component—Step 4 in Section 3.1.1

Once all cyclical components were estimated, we divided the original signal $Y_i^m$ by ${\displaystyle {\prod }_{j=1}^{P^m}}{\tilde{C}}_{j,i}^m$ to obtain the combined long-term trend $T_i^m$ and short-term $I_i^m$ variability component, or equivalently, the monthly electricity demand time series detrended from cyclical patterns,

$$M_i^m=Y_i^m/{\displaystyle {\prod }_{j=1}^{P^m}}{\tilde{C}}_{j,i}^m$$

(7)

This signal may be influenced by external factors, including fluctuations in seasonality and the economy. To predict the time evolution of $M_i^m$, we proposed a neural network model, which includes independent variables, such as average temperature over the UK, housing energy efficiency, and percentage of renewables used in the energy generation, see Table 4.

We gathered the value of $l$ independent variables $V_i^{m,1}$, $V_i^{m,2}$, …, $V_i^{m,l}$ at timepoint $i$ and the values of long-term trend and short-term variability component between timepoints $i-1$ and $i-w,$ for $w$ timesteps in the past, creating a sparse array of size $(l+1) \times (w+1)$ organised as shown in the following matrix.

$$\left(\begin{array}{cccc}{V}_i^{m,1}& 0& \dots & 0\\ V_i^{m,2}& 0& \dots & 0\\ ⋮& ⋮& \ddots & ⋮\\ V_i^{m,l}& 0& \dots & 0\\ 0& M_{i-1}^m& \dots & M_{i-w}^m\end{array}\right)$$

(8)

Note that all independent variables are resampled monthly to align with the sampling of the signal.

This array is then input into a neural network, which predicts the value $M_i^m$ of the combined long-term trend and short-term variability component at timepoint $i$. The underlying assumption is that by knowing the conditions at timepoint $i$ of the independent variables and the behaviour of the electricity demand over the previous $w$ months, we can predict the electricity demand at timepoint $i$. We first used actual data for the independent variables to build a descriptive model of electricity demand data (Section 4.1), and then discuss a modification of the model that only uses past electricity demand data to provide a predictive model of electricity demand (Section 4.2).

The selected neural network is a multi-layer perceptron (MLP) consisting of two hidden layers with 18 and 40 neurons each, including a normalisation layer and a dropout layer; see Figure 12. We settled for this network structure following a sensitivity study testing different network architectures (the results are detailed in Appendix B.2). The output of the MLP gives the estimate for $M_i^m$, denoted as ${\tilde{M}}_i^m$.

A sensitivity study was conducted on the inputs to determine the combination of independent variables and the window length for historical data that yields the best predictions (the results are detailed in Appendix B.1).

The optimised input includes a 13-month window for historical data and the following seven independent variables:

• Real GDP.
• Average temperature.
• Percentage renewables.
• Electricity CPI.
• Housing efficiency.
• Number of new builds.
• Population growth.

This leads to the 8 × 14 input matrix shown in Figure 12.

The sensitivity study showed that normalising both the independent variables and the long-term trend and short-term variability components of the signal improved prediction quality compared to other pre-processing techniques. This leads to defining two important constants: the mean and standard deviation for the combined long-term trend and short-term variability component over the training and validation period (defined in the following paragraph). The output from the neural network was then multiplied by the standard deviation, and the mean was added to obtain a prediction of the combined long-term trend and short-term variability component.

The data was divided into training, validation, and test sets, comprising approximately 62%, 15%, and 23% of the data, as follows:

• 2009–2016: Training.
• 2017–2018: Validation.
• 2019–2021: Testing.

To avoid extrapolation, the model excluded the years 2022–2023 because some independent variables were only available until December 2021 at the time of our studies.

The validation dataset was used to validate the neural network model of the combined long-term trend and short-term variability component. Such data is not required to validate the model for the cyclic components. Hence, the whole period covering training and validation was used to optimise the estimation of the cyclic components of the signal.

3.1.6. Overall Prediction of Electricity Demand—Step 5 in Section 3.1.1

Finally, we computed the overall prediction of monthly electricity demand by multiplying the modelled cyclical components with the combined predicted long-term trend and short-term variability component, i.e.,

$${\tilde{Y}}_i^m=\prod _{j=1}^{P^m}{{\tilde{C}}_{j,i}^m\tilde{M}}_i^m$$

(9)

3.2. Prediction of Hourly Electricity Demand

The hourly electricity demand prediction model is a refinement of the previously introduced decomposition model (Section 3.1), scaled up to weekly resolution and multiplied by the typical weekly pattern at hourly resolution. This requires the decomposition model to predict the average hourly demand for the predicted week, which, when multiplied by the typical weekly pattern at hourly resolution, results in predicted hourly demand.

We use the notation ${\overline{Y}}^w$ to represent the average hourly demand over one week. The typical weekly pattern is an additional cyclic component that enters the overall model as another factor. However, the estimation of the typical weekly pattern is based on averaging rather than Fourier analysis, unlike the other cyclic component introduced in steps 2–3 in Section 3.1.1. In this section, we introduced the modifications to the basic approach (Section 3.1) and the addition of the weekly pattern.

We first construct the total hourly electricity demand time series

$$Y_i^h=Y_{2\mathrm{i}-1}^r+Y_{2\mathrm{i}}^r$$

(10)

where $i=1, \dots , N^h$, and $N^h=N^r/2$ denotes the total number of hours considered. We constructed the time series for the average hourly demand for the week $j$ (or weekly electricity demand, for short) ${\overline{Y}}_j^w$ as follows:

$$\begin{gathered} {\overline{Y}}_1^w={\displaystyle \frac{1}{168}}\sum _{i=1}^{168}{Y}_i^h \\ {\overline{Y}}_2^w={\displaystyle \frac{1}{168}}\sum _{i=169}^{337}{Y}_i^h \end{gathered}$$

(11)

etc. (notice that one week has 168 h), and $j=1, \dots , N^w$, where $N^w=N^h/168$ is the total number of weeks included in our time series of electricity demand. Then, we modelled the weekly electricity demand as follows:

$${\overline{Y}}_j^w=T_j^w\prod _{k=1}^{P^w}{C}_{k,j}^w{I}_j^w$$

(12)

with $j=1, \dots , N^w$. Here, analogously to Section 3.1, we assumed that the weekly electricity demand signal can be factorised into a long-term trend component $T_j^w$, a number $P^w$ of cyclic components $C_{k,j}^w$, and a short-term variability component $I_j^w$.

As before, the long-term trend $T_j^w$ and short-term variability $I_j^w$ components are combined as $M_i^w$ and modelled using the neural network architecture described in Figure 12. For simplicity and to avoid extended run time for the neural network predictions, we resampled the signal $M_i^w$ monthly to obtain $M_i^m$ and re-use the neural network exactly as set up in Section 3.1. The long-term trend of our weekly electricity demand is adequately captured by this approach, and we observed that the short-term variabilities are smoothed a little but not significantly.

After we obtained a prediction for the weekly electricity demand ${\overline{Y}}_j^w$, we could predict the hourly electricity demand by multiplying ${\overline{Y}}_j^w$ for each specific week $j$ with the scaled (mean value of 1) hourly electricity demand over one week.

To obtain the scaled hourly electricity demand over a week, we chose the year 2013, as it was a typical year in terms of electricity demand (see discussion in Section 4.3). For this calculation and the decomposition model later, it is convenient to structure the hourly electricity demand $Y_i^h$ into a matrix with elements $Y_{i,j}^h$ where $i$ indicates the hour of the week, with $i=1, \dots , 168$, and $j$ indicates the week of the year, with $j=1, \dots , 52$. We then divided $Y_{i,j}^h$ by the corresponding average weekly electricity demand value ${\overline{Y}}_j^w$, thereby obtaining a rescaled time series of electricity demand with hourly resolution for 2013 with an average equal to 1, i.e.,

$$Z_{i,j}^h=Y_{i,j}^h/{\overline{Y}}_j^w$$

(13)

where $i=1,\dots ,168$ and $j=1,\dots ,52$.

Finally, we averaged $Z_{i,j}^h$ over all weeks in 2013, i.e.,

$${\overline{Z}}_i^h={\displaystyle \frac{1}{52}}\sum _{j=1}^{52}{Z}_{i,j}^h$$

(14)

with $i=1,\dots ,168.$

Hence, ${\overline{Z}}_i^h$ represents the scaled hourly electricity demand over one week for a typical year, which we called the weekly pattern. Lastly, we predicted the electricity demand with hourly resolution ${\tilde{Y}}_{i,j}^h$ as follows:

$${\tilde{Y}}_{i,j}^h={\overline{Z}}_i^h{\tilde{\overline{Y}}}_j^w$$

(15)

where ${\tilde{\overline{Y}}}_j^w$ is the multiplicative decomposition model estimate of the weekly electricity demand with $j=1,\dots , N^w$, where $N^w$ indicates the total number of weeks in the time series, and index $i=1,\dots , 168$ is the index for the 168 h within week $j$. Since both ${\overline{Z}}_i^h$ and ${\tilde{\overline{Y}}}_j^w$ time series need to have the same hourly resolution for the multiplication to work; we resampled the time series where required.

4. Results from the Multiplicative Decomposition Method

We first present the results obtained for the monthly electricity demand model (Section 4.1 and Section 4.2), and then we show the results for the hourly electricity demand model (Section 4.3 and Section 4.4). In Section 4.1 and Section 4.3, we present descriptive models for monthly and hourly electricity demand, respectively, that use independent variables as input to the long-trend and short-term variability components of the model. In Section 4.2 and Section 4.4, we present long-term predictive models that only use past electricity demand data as input by using a modified neural network that uses recursive inputs.

4.1. Descriptive Model with Independent Variables for Monthly Data

4.1.1. Detrending the Monthly Electricity Demand Data—Step 1 in Section 3.1.1

We divided the signal $Y_i^m$ by the estimated trend ${\tilde{T}}_i^m$ obtained with a Gaussian low-pass filter with kernel of size 12, and rescaled each year by its average. Figure 13 shows the scaled detrended signal.

4.1.2. Estimating the Cyclic Components—Steps 2–3 in Section 3.1.1

First Cyclic Component

We computed the Fourier spectrum (Figure 14) of the signal shown in Figure 13. The Fourier spectrum shows a fundamental frequency at $3.17\times {10}^{-8}$ Hz, which corresponds to a period of 12 months, and higher frequency harmonics at periods of 6, 4, 3, and 2.4 months. These observations are consistent with the peak frequencies identified in the monthly electricity demand for Spain [7].

Table 6. Top 10 Fourier components by amplitude for the scaled detrended signal.

Amplitude	Phase	Frequency	Period (Months)
1.00 × 100	0.000	0.00 × 100	-
7.30 × 10−3	0.053	3.17 × 10−8	12.00
1.30 × 10−3	1.150	1.58 × 10−7	2.40
8.70 × 10−4	0.542	6.33 × 10−8	6.00
6.20 × 10−4	2.810	9.50 × 10−8	4.00
5.60 × 10−4	0.617	1.27 × 10−7	3.00
3.90 × 10−4	−1.670	5.38 × 10−8	7.06
3.70 × 10−4	−1.610	1.36 × 10−7	2.79
3.30 × 10−4	−1.800	1.08 × 10−7	3.53
3.10 × 10−4	1.020	4.75 × 10−8	8.00

shows the first 10 peak frequencies sorted by amplitude, which are incorporated one by one into the cyclic model until the MAPE reaches a minimum.

When evaluating the cyclic model over the training and validation period, it shows a shift compared to the detrended, scaled signal (Figure 15).

To correct the shift, we calculated the MAPE between the modelled and the actual detrended and rescaled signal for different time lags, initially setting the number of Fourier components to five arbitrarily. The MAPE reached a minimum at a lag of 1 month, with the minima repeating periodically after 12 months, as shown in Figure 16. For simplicity, we chose the time lag as 1 month.

We could now optimise the model by setting the time lag to 1 month between the predictions and the signal and changing the number of Fourier components. Figure 17 shows that the error reaches a minimum when the model contains the six largest Fourier components from Table 6.

The first cyclic model (lag of 1 month and six Fourier components) produces an MAPE of 0.15% and MASE of 0.46% over the test period 2019–2021 (Figure 8).

Second Cyclic Component

The next step is to remove the first cyclic component from the signal, detrend the resulting signal, and check whether there are any prominent periodic patterns still present; see Figure 9. Hence, we repeated all the steps above to identify any residual cycles.

The peak frequencies detected by the second Fourier analysis are detailed in

Table 7. Top 10 Fourier components by amplitude for the scaled detrended signal after the first cyclical component is removed.

Amplitude	Phase	Frequency	Period (Months)
1.00 × 100	0.000	0.00 × 100	-
5.20 × 10−4	0.711	2.85 × 10−8	13.30
3.90 × 10−4	−1.700	5.38 × 10−8	7.06
3.90 × 10−4	−2.380	4.12 × 10−8	9.23
3.60 × 10−4	−1.600	1.36 × 10−7	2.79
3.60 × 10−4	0.005	3.48 × 10−8	10.90
3.20 × 10−4	−1.770	1.08 × 10−7	3.53
3.20 × 10−4	1.030	4.75 × 10−8	8.00
3.00 × 10−4	0.860	1.61 × 10−7	2.35
2.90 × 10−4	−1.610	6.97 × 10−8	5.45

, showing a cycle with a period of 13.3 months and shorter seasonal components. The top frequency here has an amplitude one order of magnitude lower than the top frequency detected in the first cyclic analysis.

When evaluating the second cyclic model over the training period, we found a shift of 13 months (Figure 18) compared to the detrended, scaled signal.

With a time lag of 13 months between the input signal and the target predictions, we optimised the model by varying the number of Fourier components. As shown in Figure 19, the prediction error reaches a local minimum when the model includes the first 5, 9, and 14 Fourier components listed in Table 7. A model with five components underfits the signal, while a better MAPE is obtained with nine components. Selecting 14 components overfits the signal, since it produces better results for the training period but worse for the testing period. Hence, we selected nine components for the second cyclic model.

The second cyclic model produces an MAPE of 0.096% and an MASE of 1.32% over the test period 2019–2021. Figure 20 shows that most of the cyclic behaviour in the original signal $Y_i^m$ has been captured by the first cyclic model. The second cyclic model only brings a marginal improvement.

4.1.3. Estimating the Combined Long-Term Trend and Short-Term Variability Component—Step 4 in Section 3.1.1

With both cyclical components removed from the signal, we then had the long-term trend and short-term variability components left, which we modelled using the neural network, as detailed in Section 3.1.5.

The neural network predicts the combined long-term trend and short-term variability component for the timepoint $i$, using as input the independent variable values at the same timepoint $i$ and historical data for electricity demand between $i-1$ and $i-w$. Note that our aim here is to build the model that best describes the electricity demand by using actual data of the independent variables to predict electricity demand one month ahead.

Figure 10 shows the fit of the neural network model to the scaled signal detrended from cyclical components over the training and validation periods, 2009–2018. The training parameters are summarised in

Table A7. Summary of the training parameters of the neural network.

	Short-Term Predictive Model, Monthly and Hourly Resolution	Long-Term Predictive Model, Monthly Resolution	Long-Term Predictive Model, Hourly Resolution
Learning rate	0.001	0.001	0.001
Batch size	61	64	64
Number of epochs	10,000	7112	7591
Method	“ADAM”, “Beta1”→0.9, “Beta2”→0.999, “Epsilon”→1/100,000, “GradientClipping”→None, “L2Regularization”→None, “LearningRate”→Automatic, “LearningRateSchedule”→None, “WeightClipping”→None

in Appendix B.3. Due to the nature of the MAPE calculation, with the error between the predicted and actual values divided by the actual value, it shows large errors when the values are close to zero. Here the MAPE is 210.87% for the neural network predictions over the test period 2019–2021. The MASE of 0.83% is, however, more indicative of the goodness of fit. We included the convergence curve of the loss function obtained for both the training and the validation sets in Figure A7 in Appendix B.3.

4.1.4. Overall Model—Step 5 in Section 3.1.1

Figure 21 shows the MAPE values for the final predictions of the multiplicative decomposition model over the training (2008–2016), validation (2017–2018), and testing period (2019–2021). An MAPE of 5.29% for 2020 is an excellent result given the anomaly introduced in the electricity demand by the COVID-19 pandemic.

Note that the MAPE of each component of the overall model, as seen in Section 4.1.2 and Section 4.1.3, do not translate to the MAPE for the overall model. The bulk of the overall prediction is carried by the first cyclic component and the model for the long-term trend and short-term variability components. The second cyclic component improved marginally the overall predictions.

The model tends to overpredict the signal, as shown by the scatter plot in Figure 22. This would result in oversizing the electricity network, which is a desired outcome: a deficit of electricity results in blackouts, while a surplus of electricity could be stored or exported.

The residuals are considered random (auto-correlation test = 0.75 > 0.05, the null hypothesis that the data are uncorrelated is not rejected at the 5 percent level based on the Ljung–Box test), confirming that the model captures all the key features of the signal.

4.2. Long-Term Predictive Model Without Independent Variables for Monthly Data

The model for the combined long-term trend and short-term variability component in Section 4.1 is limited to one-month-ahead prediction. However, it is well optimised, achieving an MAPE of 4.37% over the testing period 2019–2021. In this section, we leverage the optimised one-month-ahead prediction model by applying a recursive approach to generate long-term forecasts. In this scenario, the independent variables would require their own long-term predictive models, rather than relying on the actual observed data used in Section 4.1. While developing such models is beyond the scope of this study, we highlight the potential benefits of this approach and discuss it further in Section 5.

4.2.1. Short-Term Predictive Model, Monthly Resolution

Without the independent variables, the input to the neural network becomes a one-dimensional array

$$\begin{array}{ccc}{(M}_{i-1}^m& \dots & M_{i-w}^m\end{array})$$

(16)

The architecture of the neural network remains identical to Figure 12, with the difference that now the input is an array of size {11}, as is the normalisation layer. As shown in the sensitivity study (see Appendix B.1), the optimal window size $w$ for historical data, when no independent variables are included, is 11 months.

4.2.2. Long-Term Predictive Model 1, Monthly Resolution

We introduced a recursive function, illustrated in Figure 23, that appends the output from the neural network to its input, creating an array of size 12. Then, the function drops the first element of the array to maintain an array size of 11, which is input into the neural network to predict the next step. Over the long term, the input to predict the long-term trend and short-term variability components at time step $i$ includes only the outputs from previous time steps between $i-1$ and $i-w$.

The training parameters for this long-term predictive model 1 are summarised in Table A7 in Appendix B.3. We included the convergence curve of the loss function obtained for both the training and the validation sets in Figure A8 in Appendix B.3.

4.2.3. Long-Term Predictive Model 2, Monthly Resolution

We tested a modified neural network (Figure 24), which includes an additional normalisation layer and a scaling layer before the output layer. Figure 25 shows the recursive predictions using the previous neural network (Figure 12) with an input array size of 11 and the modified neural network (Figure 24).

To assess the difference between the different neural network models, we needed to consider the overall model.

Table 8. MAPE for the overall model predicting monthly electricity demand using different models for the long-term trend and short-term variability components.

	Overall Training Period 2009–2016	Overall Validation Period 2017–2018	Overall Testing Period 2019–2021	2019	2020	2021
Short-term predictive model, monthly resolution	1.17	1.63	3.52	2.49	5.35	2.71
Long-term predictive model 1, monthly resolution	1.17	1.63	4.62	2.29	6.51	5.06
Long-term predictive model 2, monthly resolution	1.39	1.66	4.16	2.15	6.84	3.48

shows that using the long-term predictive model 2, described in Figure 24, improves the long-term electricity demand predictions compared to the long-term predictive model 1, described in Figure 12. The results with the short-term predictive model, i.e., no recursion, are also shown for reference.

4.3. Descriptive Model with Independent Variables for Hourly Data

In this section, we applied the same process as in Section 4.1 to hourly data, i.e., including actual independent variables without recursion.

The weekly time series ${\overline{Y}}_j^w$ with $j=1,\dots , N^w$ was processed through the method described in Section 3.2. The trend was extracted via the Gaussian low-pass filter with a kernel of radius 52 (weeks).

Table 9. Cyclic models optimisation parameters for hourly data.

	Time Lag	Number of Fourier Components
Cyclic component model 1	13	12
Cyclic component model 2	5	2

summarises the characteristics of the first and second cyclic component models.

As mentioned in Section 3.2, we chose 2013 as a typical year to obtain the scaled hourly electricity demand over a week. Figure 26 shows that the hourly pattern of electricity over a week, scaled to be centred around 1, is stable between the years 2009 and 2023. All years are within ±5% of the pattern for 2013.

The third cyclic component model is simply the weekly pattern for 2013 repeated over, as shown in Figure 27.

The predictions for the first two cyclic components have a weekly resolution, the weekly pattern has a resolution of hours, and finally, the predictions for the combined long-term trend and short-term variability component have a monthly resolution. All time series were resampled hourly to be able to perform the multiplication at each timepoint.

Figure 28 shows the predictions from the proposed model compared to the hourly total of the UK electricity demand data, with Figure 29 showing a zoom on one week, Monday to Sunday, in August 2019. The MAPE value for the overall model was 7.5% for the testing period 2009–2021.

The scatter plot (see Figure 30) of predictions versus actual values shows a slight tendency of the model to overpredict the electricity demand.

4.4. Long-Term Predictive Model Without Independent Variables for Hourly Data

In order to derive true long-term predictions for the hourly electricity demand, we remove the independent variables from the input to the neural network and apply the recursive function, as described in Figure 23.

As shown in

Table 10. MAPE for the overall model predicting hourly electricity demand using different models for the long-term trend and short-term variability components.

	Overall Training Period 2009–2016	Overall Validation Period 2017–2018	Overall Testing Period 2019–2021	2019	2020	2021
Short-term predictive model, hourly resolution	6.27	6.51	7.51	7.22	13.36	14.20
Long-term predictive model 1, hourly resolution	6.27	6.51	8.62	7.32	13.76	14.51
Long-term predictive model 2, hourly resolution	6.29	6.50	9.79	7.60	14.29	14.96

, the original neural network, described in Figure 12, gives marginally better predictions than the modified neural network, Figure 24, introduced for the monthly electricity demand long-term predictions. The training parameters for this long-term predictive model are summarised in Table A7 in Appendix B.3. We included the convergence curve of the loss function obtained for both the training and the validation sets in Figure A10 in Appendix B.3.

5. Discussion

In this paper, we have proposed a novel multiplicative decomposition model to predict the long-term UK electricity demand at both monthly and hourly resolution. The model assumes that electricity demand can be described as the product of cyclic components, including yearly, weekly, and daily cycles, and a long-term trend and short-term variability component. We model the cyclic components based on the Fourier analysis of the detrended signal, and we apply a neural network model to describe the combined long-term trend and short-term variability component.

We have successfully implemented an approach similar to the model of cyclic components in fashion demand [8]. We have also efficiently applied a neural network-based prediction for the combined long-term trend and short-term variability components, a notably simpler approach to the implementation of the Autoregressive Distributed Lag model used by Hunt et al. [11].

Our model achieves highly accurate results for long-term predictions with an MAPE of 4.16% for the period 2019–2021 with a monthly resolution, and an MAPE of 8.62% for the same period with an hourly resolution. Moreover, our model is highly flexible and accurate: when increasing the forecasting period from monthly to hourly, the model’s resolution increases by approximately 730× while the model error only increases by 2.1×.

Based on our review, other papers focus on descriptive modelling rather than long-term prediction. As such, our comparison is restricted to the descriptive component of our approach (Section 4.1), which yielded an MAPE of 3.47% over a 3-year horizon. This may be compared with the Spanish electricity demand model [7], which reported an MAPE of 1.74% over 5-year predictions.

We compared our proposed model with a traditional SARIMA model, which performs worse, achieving long-term predictions on monthly data with an MAPE of 7.11% for the period 2019–2021 (Figure A13 in Appendix C.2). The results from the SARIMA model predictions for hourly data are substantially poorer, obtaining an MAPE of 42.57% for the same period. Figure A15 in Appendix C.3 shows the SARIMA model predictions for hourly data, which are unable to capture the dynamics of hourly electricity demand.

The novelty of our model lies in the provision of a long-term prediction of electricity demand at hourly resolution, achieving highly accurate results. However, our model also presents some limitations. It is noteworthy that the long-term predictions did not include any exogenous variables. Modelled data for those exogenous variables could be included to improve the model’s performance, as we showed that including those in the descriptive model leads to a 1.4% improvement in MAPE for the monthly data.

Models for the exogenous variables could also consider different scenarios to study the impact of different factors on electricity demand, such as an increase in housing energy efficiency following a government policy, different renewable energy penetration rates, or edge-case scenarios in UK temperatures due to climate change.

Our proposed model could be further improved by specifically modelling the disruption in electricity demand caused by external major events, such as the COVID-19 pandemic. An intervention model with a pulse function or step function that estimates the impact of lockdowns on the electricity demand could be integrated as input to the neural network. In fact, observation of hourly demand data during the Christmas holidays allowed us to estimate a drop of 18% in electricity demand due to business closures, as detailed in Appendix D.

Finally, applying the model to different segments of electricity demand, such as by region or by industry, or to adjacent domains such as natural gas demand forecasting, could provide further insights into its adaptability and broader applicability.

Overall, this paper offers a promising approach to improving National Grid’s ability to predict UK electricity demand with greater resolution than previously achieved. The model could contribute to ongoing efforts to plan and shape the future electricity grid, particularly in supporting decisions around future electricity generation capacity.