Renewable-Aware Geographical Load Balancing Using Option Pricing for Energy Cost Minimization in Data Centers

Abstract

It is becoming increasingly difficult to properly control the power consumption of widely dispersed data centers. Energy consumption is high because of the need to run these data centers (DCs) that handle incoming user requests. The rising cost of electricity at the data center is a contemporary problem for cloud service providers (CSPs). Recent studies show that geo-distributed data centers may share the load and save money using variable power prices and pricing derivatives in the wholesale electricity market. In this study, we evaluate the problem of reducing energy expenditures in geographically dispersed data centers while accounting for variable system dynamics, power price fluctuations, and renewable energy sources. We present a renewable energy-based load balancing employing an option pricing (RLB-Option) online algorithm based on a greedy approach for interactive task allocation to reduce energy costs. The basic idea of RLB-Option is to process incoming user requests using available renewable energy sources. In contrast, in the case of unprocessed user requests, the workload will be processed using brown energy or call option contract at each timeslot. We formulate the energy cost minimization in geo-distributed DCs as an optimization problem considering geographical load balancing, renewable energy, and an option pricing contract from the derivative market while satisfying the set of constraints. We prove that the RLB-Option can reduce the energy cost of the DCs close to that of the optimal offline algorithm with future information. Compared to standard workload allocation methods, RLB-Option shows considerable cost savings in experimental evaluations based on real-world data.

1. Introduction

Cloud computing has emerged as a popular means of delivering many web-based services, including IoT, data storage and processing, audio and video distribution, etc. [1]. Numerous data centers located in various locations around the globe are used by every cloud service provider to handle user demands. These data centers (DCs) use much power, which is a significant factor in the data center’s operating expenses [2]. Cloud service providers (CSPs) face enormous challenges because of the high monthly cost of DC electricity usage. In 2020, there were around 8 million DCs running, according to the U.S. Department of Energy [3]. The amount of energy used by these data centers to handle the workload is equivalent to 2% of world power consumption, or 416.2 terawatt-hours [3]. By 2022, it is projected to account for 7% of the world’s energy consumption, which would be a steep rise [4]. Therefore, CSPs are under significant pressure and play an essential role in helping data centers reduce their power usage and related expenses.

Energy efficiency in geographically dispersed data centers is a topic of extensive study. Geographical load balancing (GLB) is a standard solution discussed in this research work [5,6,7,8]. Utilizing the regional differences in dynamic pricing of power, GLB distributes incoming user requests across geographically scattered DCs to reduce energy expenses [2]. Another way to reduce data center running costs is to employ renewable energy using ESDs to power data centers [9,10,11,12]. However, due to battery capacity limitations, completely powering data centers is problematic [13]. To solve this problem, CSPs primarily rely on brown energy and batteries as a backup during power failure [14].

Recent research has advocated for techniques to distribute the cost of power over many locations over time [3,15,16,17]. These include installing energy storage systems, opportunistic optimization [18,19,20,21,22], demand response strategies, and buying energy policies [23]. These new methods and control strategies [24,25,26,27] have encouraged cloud service providers to enter the electricity derivative markets. The value of a derivative depends on that of some underlying asset. Options, swaps, forwards, and futures are all utilized in the market to hedge against the risk of fluctuating power prices [5,6]. With options, the purchaser has no obligations and has the right to practice it on or before the expiration date, giving them more flexibility than other derivatives [5].

We significantly contribute to the literature by studying how CSPs with numerous data centers in different locations may best distribute their workloads and save energy. In contrast to other publications, we present the data center’s energy cost minimization problem as an optimization problem considering the power call option, renewable energy sources, and the presence of energy storage devices (ESDs). To begin deciding whether or not to purchase the electricity call option, we apply the Black–Scholes Model (BSM) to a real scenario to ascertain the option’s value. Based on the maximal battery capacity charged by renewable energy, the lowest brown energy rate, and market option pricing, we suggest an optimum online workload distribution technique (RLB-Option). We also conduct extensive tests to evaluate the performance of RLB-Option utilizing on-site renewable energy, workload traces, and hourly electricity prices.

The paper’s remaining sections are structured as follows: in Section 2, we present the literature review. Section 3 and Section 4 examine the incoming workloads and formulate the research problem of lowering power costs through renewable sources, GLB, and an option pricing contract as an optimization problem. This optimization problem is addressed in Section 5. In Section 6, we assess the proposed algorithm against the benchmark by analyzing real-world data from the options in the derivative market, incoming workload, and electricity prices. Results show that the proposed algorithm is more effective. The concluding remarks and suggestions for further study are presented in Section 7.

2. Related Work

We structure the related work based on the following factors:

2.1. GLB and Power Management

Researchers, academics, and businesses have focused heavily over the past decade on energy management approaches to reduce the overall expenses of the data center [13,14,28,29,30,31,32,33,34]. However, prediction of data centre load [35,36,37,38,39] is mandatory before power management. The first studied the energy consumption in large-scale networks [34]. They considered the incessant energy prices seen in the realistic market to formulate the problem. The researchers created a price optimizer [40,41] to track and report regional pricing differences for power to maximize profit in the current distribution system. Taking into account the wholesale electricity market, Ref. [42] investigated increasing power concerns to reduce data center electricity costs without compromising service quality. The authors used integer programming to frame the minimization issue and used Brenner’s fast polynomial-time method to solve it. Workload distribution among geographically dispersed DCs was the topic of [43], which included a framework that considered fluctuating energy costs. They suggested an online workload distribution strategy to handle the renewable energy supply, the risk of time variability, and customer demand in the market. Additionally, Ref. [44] brought attention to this issue and considered a scenario in which service providers would purchase power from irregular energy markets and GLB in tandem. Energy supply companies and consumers engage in an auction system to establish rates. To reduce power costs in the wholesale market, Ref. [43] thought of using existing on-site renewable energy and energy storage capacities in data centers. The researchers use the Lyapunov optimization technique to formulate the problem. They develop an online algorithm to balance storage capacity and cost savings in the energy sector. Ref. [45] investigated the issues in the traditional energy market with geographical workload distribution, particularly concerning market clearing and bidding. The authors use a market clearing methodology to design an online algorithm to solve the optimization problem.

2.2. GLB and Geo-Distributed Data Centers

The GLB has been the subject of many studies in cloud data centers [16,17,28,29,30,31,46]. Most research has been on determining the best approach to load balancing regardless of time, which has led to formulating an optimization problem that can be handled using various optimization methods. By illustration, Ref. [47] considered interactive and indivisible workload and modelled geographical load balancing as a cost minimization problem. The authors suggest an online algorithm called GreenGLB to distribute incoming workload based on factors such as power costs, the number of available servers, and the data center’s environmental impact. Green energy and fluctuating power costs were other topics covered by [23]. To reduce the cost of the DC, the researchers design an algorithm, namely DGLB, for workload balancing. To investigate the interplay between power costs, service delays, and bandwidth expenses, Ref. [48] built a cost minimization model. The researchers designed a short-term prediction mechanism (SPM) for the distribution of workload to cut down on DC running costs.

2.3. Renewable Energy and Storage

Recent years have seen a rise in the number of studies looking at how geographically dispersed data centers might use renewable energy and batteries to save their running expenses [7,8,15,16,29,44,49]. In addition, electric vehicles (EV) are utilized to cater this issue [50,51,52,53]. When there is a power outage, cloud providers often employ backup batteries to continue handling incoming workload [29]. Not only that but renewable energy sources are often used to recharge the batteries. Instead, we think of batteries as the primary power source for handling user demands, and we will charge them using either derivative (i.e., options) or the lowest rates of brown energy. Since batteries have the limitations, they must be included in the GLB optimization problem formulation. Second, the suggested method in this research does not need information about the future to function, but GLB-based algorithms need [17].

3. Problem Setting

This section outlines our approach to the interactive workload allocation problem and our system model—moreover,

Table 1. Set of essential notations.

Notation	Description
$t\in \left\{1,T\right\}$	Discrete-time index
$i\in \left\{1,N\right\}$	Cloud data center index
$W\left(t\right)$	The total incoming workload
$w_i\left(t\right)$	Total assigned workload
$D^{max}$	Maximum limit of delay
$d_i\left(t\right)$	Average delay
$d_i^Q\left(t\right)$	Queuing delay
$R_i^{max}$	The maximum limit of renewable energy
$R_i\left(t\right)$	Renewable energy at a data center
$q_i\left(t\right)$	Electricity price
$S_i^{max}$	Total servers at a data center
$S_i^{ac}\left(t\right)$	Active servers
$S_i^{in}\left(t\right)$	Inactive servers
${\mu }_i$	Service rate
$P_i^{IT}\left(t\right)$	IT equipment power usage
$P_i\left(t\right)$	Total power consumption at a center
$P_i^{ac}\left(t\right)$	Active server’s power usage
$P_i^{in}\left(t\right)$	Inactive server’s power usage
$P_i^{max}\left(t\right)$	Maximum power consumption
$B_i\left(t\right)$	Power from brown energy
$O_i\left(t\right)$	Power from call option contract
$C_i\left(t\right)$	Energy cost
$C\left(t\right)$	Total Cost of electricity
$X_i$	Strike price
$r$	Interest rate
$\tau$	Option contract expiry time
$\sigma$	Future price volatility
$N(d_1)$	Call option probability change
$N(d_2)$	Spot prices probability

summarizes the set of essential notations and their definitions used in this research work.

3.1. Problem Formulation

Every geo-distributed data center i has hundreds of servers, either homogeneous or heterogeneous. There are $N$ such data centers. We take homogenous servers into account in our model. Each data center is powered by on-site renewable energy sources and brown energy from the real-time and options markets. Cloud service providers purchase energy from wholesale and derivative markets to power the DC. To reduce the overall expected operating cost of the geo-distributed DCs, the service providers strive to optimize GLB via option pricing in the derivative market. We examine a discrete-time model in which a global-LB receives the user requests $W\left(t\right)$ for each timeslot $t$. (See Figure 1). The global-LB functions as a workload router to reduce the data center’s energy costs. It decides which DC $i$ to choose online based on the option pricing, cheapest rates of brown energy, and highest battery power. After choosing a DC $i$, the incoming workload $w_i\left(t\right)$ is sent to the local load balancer (local-LB). Each DC distributes the allotted workload to the appropriate server by the servers’ current usage level.

3.2. The Model of Incoming Workload

The global load balancer receives every incoming workload at the beginning of a time interval $t$. Workloads may be classified into two broad categories: batch and interactive [31]. The study takes into account a DC’s interactive and non-splitable workload. Let $w_i\left(t\right)$ denote the work done in time slot $t$ at data center $i$:

$${\displaystyle \sum }_{i=1}^N{\lambda }_i{w}_i\left(t\right)=W\left(t\right)\hspace{1em}\hspace{1em}\forall t\in \left[1,T\right]$$

(1)

$${\displaystyle \sum }_{i=1}^N{\lambda }_i=1where{\lambda }_i\in \left[0,1\right]\forall t\in \left[1,T\right]$$

(2)

To indicate that incoming work cannot be divided, we utilize the Boolean expression λi ∈ {0, 1}. It is clear from (1) and (2) that the $W\left(t\right)$ may only be distributed to one data center. Every data center often houses hundreds of homogeneous servers to handle the incoming workload. However, the total active server $S_i^{ac}\left(t\right)$ cannot exceed its total limit $S_i^{max}$. Thus, we have:

$$S_i^{ac}\left(t\right)\le S_i^{max}\hspace{1em}\forall i\in \left[1,N\right]$$

(3)

3.3. The Model of Quality of Service (Delay)

The cost of delay arises due to waiting time during user request processing; this time includes $d_i^Q\left(t\right)$ (queuing time) and $L_i\left(t\right)$ (network latency) from the global-LB to the data center $i$ during the time $t$ [17]. We assume that network latency is constant. We simulate the waiting time in the queue using MM1 and examine the $d_i^Q\left(t\right)$ (average waiting time) at each data center $i$ [29,54]. Moreover, we assume that every single server is up and actively handling user requests. In addition, we use the assumption that all data center servers operate at the same service rate ${\mu }_i$ [28]. Each data center has a maximum allowable delay to guarantee a certain level of service (quality of service). As a result, we can write down as an expression:

$$d_i\left(t\right)=L_i\left(t\right)+\frac{1}{{\mu }_i{S}_i^{ac}\left(t\right)-w_i\left(t\right)}$$

Data center $i$ per-request delay cost during time $t$ is

$$d_i\left(t\right)=\beta \left[L_i\left(t\right)+\frac{1}{{\mu }_i{S}_i^{ac}\left(t\right)-w_i\left(t\right)}\right]$$

$\beta$ is a conversion factor translates the typical user request wait time into monetary terms [47]. Here, we consider $\beta =1$ to guarantee the QoS, we must adhere to the following restriction:

$$d_i\left(t\right)\le D^{max}\forall i\in \left[1,N\right]$$

(4)

3.4. The Model of Power Utilization

The entire running cost of a geo-distributed DC includes not only the delay cost but also the cost of power. The data center’s energy consumption is devoted to cooling infrastructure and IT equipment. Each data center in a cloud industry uses much power for its cooling systems, which are essential for keeping the servers at an optimal temperature [2]. Power Usage Effectiveness (PUE), a measure of the energy efficiency of a cooling system, is taken into account. The PUE of a data center is the ratio of its IT equipment’s power consumption to its total power consumption [55]. Standard PUE in the cloud computing business is, on average, 1 to 2 [31]. Therefore, we define the model for the data center $i$ IT equipment’s power consumption throughout the time $t$ through:

$$P_i^{IT}\left(t\right)=S_i^{ac}\left(t\right)\left[P_i^{in}\left(t\right)+P_i^{ac}\left(t\right)U_i\left(t\right)\right]$$

We measure the average utilization of servers as $U_i\left(t\right)=\frac{w_i\left(t\right)}{S_i^{ac}\left(t\right){\mu }_i}$ at time $t$ in DC $i$. Hence, the overall power utilization in DC during the time $t$ is computed via

$$P_i\left(t\right)=PU{E}_i\left(t\right)·S_i^{ac}\left(t\right)\left[P_i^{in}\left(t\right)+P_i^{ac}\left(t\right)\frac{w_i\left(t\right)}{S_i^{ac}\left(t\right){\mu }_i}\right]$$

Moreover, the total electricity cost of all the data centers is computed as follows:

$$C\left(t\right)={\displaystyle \sum }_{i=1}^N{q}_i\left(t\right)·P_i\left(t\right)$$

3.5. The Model of Renewable Energy

On-site renewable power generation (such as wind or solar) and brown energy are two regularly used sources to power data centers [33,56]. In our study, we assumed on-site renewable energy production using solar panels. At the discrete-time $t$ in DC $i$, the production of green energy is represented by $R_i\left(t\right)$:

$$0\le R_i\left(t\right)\le R_i^{max}\left(t\right)$$

(5)

4. GLB Optimization Problem

This paper reflects on geographical load balancing as an optimization problem. We classify the optimization problem into two sub-problems.

4.1. Problem-I: Calculate the Value of Electricity Call Option (V)

In problem-I, at time slot $t$, a cloud service provider calculates $V$ to decide whether or not to purchase the call option. The possibility of buying energy in the derivatives market relies on the value of $V$. With a strike price of $X_i$ the option holder is entitled to use $OMWh$ of power at time $t$ throughout the designated month. Executing the call option $C_{option}$ will cost $OMh{X}_i$ in total. Taking into account that there is 30 $\mathrm{d}$ in a month and 24 $\mathrm{h}$ in a day, the monthly power cost would be $C_{option}=720·O{X}_i$. The overall cost of power delivered to the data center is $C_{spot}={\displaystyle \sum }_{t=1}^{Mh}E\left(q_i\left(t\right)\right)$ if the $V$ is not executed owing to the cheapest electricity rates in an open market. This research examines the European electricity call option, which is only implemented when the underlying market is “in-the-money” (ITM). Consequently, the payoff of the $V$ is calculated through the following equation:

$$\pi =max\left(O{\displaystyle \sum }_{t=1}^{Mh}E\left(q_i\left(t\right)\right)-OMh{X}_i,0\right)$$

For the electricity call option, the preceding equation sets an upper bound on the exercise price as

$$O{\displaystyle \sum }_{t=1}^{Mh}E\left(q_i\left(t\right)\right)-OMh{X}_i\ge 0$$

The Black–Scholes Model (BSM) provides an estimate for the value of an electricity call option [24] if it is acquired at timeslot $t$, as follows:

$$V=q_i\left(t\right)·N\left(d_1\right)-\frac{X_i}{e^{-r\tau }}·N\left(d_2\right)$$

(6)

subject to;

$$d_1=ln\frac{q_i\left(t\right)}{X_i}+\left(r+\frac{\sigma }{2}\right)\tau$$

(7)

$$d_2=d_1-\sigma \sqrt{\tau }$$

(8)

4.2. Problem-II: Minimization of Energy Cost

The geographical load balancing optimization problem aims to reduce the overall costs in data center $i$ across the time interval $t\in \left[1,T\right]$ by selecting DC $i$ based on $d_i\left(t\right)$ and $P_i\left(t\right)$. To do this, we define the following GLB optimization problem:

$$min{\displaystyle \sum }_{t=1}^T{\displaystyle \sum }_{i=1}^N{\lambda }_i\left[C_i\left(t\right)+d_i\left(t\right)\right]$$

(9)

subject to

$${\displaystyle \sum }_{i=1}^N{\lambda }_i{w}_i\left(t\right)=W\left(t\right)\hspace{1em}\hspace{1em}\forall t\in \left[1,T\right]$$

(10)

$${\displaystyle \sum }_{i=1}^N{\lambda }_i=1,{\lambda }_i\in \left[0,1\right]\hspace{1em}\forall t\in \left[1,T\right]$$

(11)

$$0\le S_i^{ac}\left(\mathrm{t}\right)\le S_i^{max}\hspace{1em}\hspace{1em}\forall i\in \left[1,N\right]$$

(12)

$$0\le d_i\left(t\right)\le D^{max}\hspace{1em}\hspace{1em}\forall i\in \left[1,N\right]$$

(13)

$$0\le P_i\left(t\right)\le P_i^{max}\hspace{1em}\hspace{1em}\forall i\in \left[1,N\right]$$

(14)

$$0\le R_i\left(t\right)\le R_i^{max}\left(t\right)\hspace{1em}\forall i\in \left[1,N\right]$$

(15)

Allocating $W\left(t\right)$ at each time $t$ to the single data center is guaranteed by constraints (10) and (11); having no overprovisioned servers in data center $i$ is indicated by constraint (12). The average response time is tracked by constraint (13), and the total power taken from the energy market is not negative and does not exceed a particular threshold value indicated by constraint (14). Finally, the constraints (15) ensure that renewable energy cannot be negative and does not exceed the upper limit.

5. Proposed Solution

This section offers the answer to the optimization problem described in Section 4. First, we use the BSM to solve problem-I. Be aware that GLB judgments affect the call option’s value. We provide an online approach to reduce the energy usage and related costs at time $t$ of DC $i$ to address problem-II (optimal workload distribution).

5.1. Problem-I: Calculate V

We calculate the value of the electricity call option for discrete-time $t=0$ and choose whether to purchase a call option or energy in the open market (see Algorithm 1, Part-A). Using the parameters $q_i\left(t\right)$, $X_i,r$, $\tau$, $\sigma$, (see Table 1 for explanation), we apply BSM to obtain the premium value of call option $V$:

$$V=q_i\left(t\right)·N\left(d_1\right)-\frac{X_i}{e^{-r\tau }}·N\left(d_2\right)\hspace{1em}\hspace{1em}\forall t\in \left[1,T\right]$$

Algorithm 1: RLB-Option

Part A:  Determine the value of$\mathit{V}$

$1:\hspace{1em}\mathrm{At}\mathrm{time}t=0$$,\mathrm{observe}{q}_i\left(t\right)$$,X_i$$,\mathrm{and}{V}_p$

$2:\hspace{1em}\mathrm{Calculate}r,\tau ,d_1,d_2,N\left(d_1\right),\mathrm{and}N\left(d_2\right)\mathrm{to}\mathrm{solve}V$.

$3:\hspace{1em}V=q_i\left(t\right)·N\left(d_1\right)-\frac{X_i}{e^{-r\tau }}·N\left(d_2\right)$.

4:  Subject to constraints (7) and (8)

$5:\hspace{1em}\mathrm{if}V\ge V_p$then purchase the call option for electricity end if

Part B:  Energy Cost Minimization—Solve the Optimization Problem

$6:\hspace{1em}\mathrm{Read}W\left(t\right),q_i\left(t\right),$$\forall i\in \left[1,N\right]$

$7:\hspace{1em}{i}^{*}=min\left(q_i\left(t\right)\right)$

$8:\hspace{1em}{B}_i\left(t\right)={\eta }^d·B_i\left(t-1\right)$$\hspace{1em}\forall i\in \left[1,N\right]$

I. Process $W\left(t\right)$ using energy storage devices

9:  if   $B_{i^{*}}\left(t\right)\ge {\delta }_{i^{*}}$$\mathit{then}CP=min\left\{P_{i^{*}}\left(t\right),B_{i^{*}}\left(t\right)-{\delta }_{i^{*}}\right\}$

10:   $B_{i^{*}}\left(t\right)=B_{i^{*}}\left(t\right)-CP$

11:   $P_{i^{*}}\left(t\right)=P_{i^{*}}\left(t\right)-CP$ end if

II. Process $W\left(t\right)$ using option pricing

12:  if   $\left\{q_{i^{*}}\left(t\right)>X_{i^{*}}\&V\ge V_p\right\}$ then $CP=min\left\{P_{i^{*}}\left(t\right),O_{i^{*}}\left(t\right)\right\}$

13:   $O_{i^{*}}\left(t\right)=O_{i^{*}}\left(t\right)-CP$ & $P_{i^{*}}\left(t\right)=P_{i^{*}}\left(t\right)-CP$

14:   $B_{i^{*}}^{up}\left(t\right)=min\left\{B_{i^{*}}^{max},B_{i^{*}}\left(t\right)+{\eta }^c·O_{i^{*}}\left(t\right)\right\}$

15:   ${\Delta }_{i^{*}}^B\left(t\right)=B_{i^{*}}^{up}\left(t\right)-B_{i^{*}}\left(t\right)$

16:   $B_{i^{*}}^{ave}=\frac{\left(B_{i^{*}}^{ave}·B_{i^{*}}\left(t\right)\right)+\left({\Delta }_{i^{*}}^B\left(t\right)·X_{i^{*}}\right)}{B_{i^{*}}^{up}\left(t\right)}$

17:   $B_{i^{*}}\left(t\right)=B_{i^{*}}^{up}\left(t\right)$ end if

III. Process $W\left(t\right)$ using the lowest rate of brown energy

18:   $CP=P_{i^{*}}\left(t\right)$ & $P_{i^{*}}\left(t\right)=0$

IV. ESDs charging with renewable energy sources

19: for $i=1toN$ do

20:  if ${\tau }_{i^{*}}^{out}\left(t\right)\ge {\alpha }_{out}$ OR $q_i\left(t\right)\le {\theta }_i$ then

21:     $RU=B_i^{max}-B_i\left(t\right)$

22:     $B_i^{up}\left(t\right)=min\left\{B_i^{max},B_i\left(t\right)+{\eta }^c·RU\right\}$

23:  else if $B_i\left(t\right)\le {\delta }_{i^{*}}$ then

24:     $RU={\delta }_{i^{*}}-B_i\left(t\right)$

25:     $B_i^{up}\left(t\right)=\left\{B_i\left(t\right)+{\eta }^c·RU\right\}$ end if

26:     ${\Delta }_i^B\left(t\right)=B_i^{up}\left(t\right)-B_i\left(t\right)$

27:     $B_i^{ave}=\frac{\left(B_i^{ave}·B_i\left(t\right)\right)+\left({\Delta }_i^B\left(t\right)·q_i\left(t\right)\right)}{B_i^{up}\left(t\right)}$

28:     $B_i\left(t\right)=B_i^{up}\left(t\right)$ end for

In Algorithm 1 (Line 1–4), we begin by solving for $V$ at time $t=0$, then compare the value against $V_p$. In the derivative market, an option contract for a certain quantity of power is signed with the provider if $V$ is larger or equal to $V_p$ (Line 5). We execute the electricity call option to acquire the necessary megawatts at a certain strike price in a specific timeslot $t$ whenever the $q_i\left(t\right)$ is higher than $X_i$.

This study considers a 24-h window in which an option can be exercised. The time when an option can be exercised is denoted by $t_j$, $j=1,2,...,T$. The decision-making processes of CSPs in the call option of electricity and real-time markets are depicted in Figure 1.

5.2. Problem-II: Energy Cost Minimization—Solve the Optimization Problem

To mitigate the overall costs of the geo-distributed DCs, part B of Algorithm 1 defines the pseudo-code of the global load balancer’s renewable aware load balancing with option pricing (RLB-Option) policy as it processes the incoming workload. The information is gathered in line (6) for each geo-distributed data center. $B_i\left(t\right)$ is the amount of power currently in ESDs, and $i^{*}$ is the data center with the least power costs on the spot market (Lines 7 and 8).

Part-B of RLB-Option has four sections. In section-I, $W\left(t\right)$ will be processed using renewable-aware ESDs (Lines 9 to 11). ${\delta }_{i^{*}}$ denotes the least level of power in ESDs ($20\%$). If $B_{i^{*}}\left(t\right)\ge {\delta }_{i^{*}}$ is satisfied, then the workload will be processed using renewable aware ESDs. The required amount of energy for the processing of incoming workload (CP) is calculated, and assign the workload to $i^{*}$ (Line 9). Moreover, in Lines 10 and 11, $B_{i^{*}}\left(t\right)$ and $P_i\left(t\right)$ are updated for $i^{*}$ during time $t.$ On the other hand, if the batteries do not have enough level to process the user requests or in case of the residual unexecuted user requests, then $W\left(t\right)$ will be executed either using the minimum price value of the brown power source (Line 18) or option pricing (Lines 12–17). To get the threshold value ${\theta }_{i^{*}}$ for DC $i$ at time $t$, we utilize the following formula to calculate the statistical threshold value (STV) [1]:

$${\theta }_{i^{*}}=antilog\left[avg\left(log{q}_i\left(t\right)\right)+1.282·std\left(log{q}_i\left(t\right)\right)\right]\hspace{1em}\forall t^{\prime }\in \left[1,t\right],\forall i\in \left[1,N\right]$$

The value of current workload processing is determined, and the user requests (if settings are not met in section-I) or residual unexecuted user requests are allocated to the data center $i^{*}$ using the call option for electricity. Finally (section-II), $O_i$ and $P_i\left(t\right)$ are adjusted under how the task at lines (13) was processed. On lines (14–16), $B_{i^{*}}^{up}\left(t\right)$, ${\Delta }_{i^{*}}^B\left(t\right)$, and $B_{i^{*}}^{ave}$ are computed. The RLB-Option will not execute the call option if the $q_i\left(t\right)$ is less than $X_{i^{*}}$. Instead (section-III), the incoming user requests will be allocated to DC $i^{*}$ based on brown energy’s cheapest costs (Line 18).

Additionally (section-IV), in line 21, RU to charge the energy storage devices is computed, and the level of batteries is updated as necessary if the outside temperature of the data center is higher than the threshold value ${\theta }_{i^{*}}$ (21 °C) or the spot electricity prices are lower than the threshold value ${\theta }_{i^{*}}$ (Line 20). Otherwise, RU is computed, and the batteries are updated appropriately if the current battery level falls below the minimal level (Lines 24–26). Lines 27 and 28 updates the $B_{i^{*}}^{ave}$ after calculating ${\Delta }_{i^{*}}^B\left(t\right)$ and the mean power storage cost.

6. Numerical Evaluation

Here, we assess the efficacy of our proposed algorithm RLB-Option using energy prices, call options for electricity, on-site renewable energy, and Wikipedia’s incoming user requests intending to reduce the overall costs of the geo-distributed DCs.

6.1. Experimental Setup

The experimental settings that we employed for this research work are presented in this section. The evaluation procedure considers a time horizon of one hour and timeslots with a length of one month ($T=720$ h).

In order to forecast future electricity prices at the start of each timeslot, we utilize the Auto-Regressive Integrated Moving Average (ARIMA) model. Electricity price forecasting can be done in several ways, including artificial neural networks and regression analysis. However, earlier works like [57,58] have already employed the ARIMA model with good results. It is important to remember that the value of call option V is calculated by factoring in the anticipated power price. Moreover, we utilized MATLAB for the simulation environment. Despite the availability of other simulation tools (such as CloudSim) [36], we chose MATLAB because of its freedom in terms of programming the various features as per our needs.

6.1.1. Description of Geo-Distributed Data Centers

We take into account three ($n=3$) geo-distributed DCs. The data centers are presumably situated in Ontario, Canada, Utica, New York, and Illinois, respectively. Every data center has $S_i^{max}=\mathrm{15,000}$ maximum servers, and it is assumed that the service rate ${\mu }_i$ to execute $W\left(t\right)$ for a single server in each data center is 1 [47] for experiments. One server will use $P_i^{ac}\left(t\right)=120$ Watts and $P_i^{in}\left(t\right)=60$ Watts of electricity during each time slot when it is active and idle, respectively [13]. Each DC $i$ has a power usage effectiveness set to 1.20 [45]. The maximum network latency is 10 milliseconds, while the queuing delay is 1 millisecond [29]. As a result, $D^{max}=11$ millisecond is the maximum allowable delay (including network and queueing delays). It is assumed that $\beta$ is equal to 1 [49]. Additionally, we presumed that there is a single global-LB and that all data centers have uniform configurations.

6.1.2. Description of Incoming Workload

Over 720 h of timeslots, with interactive workloads of geo-distributed data centers from 18 September 2007, to 19 October 2007, we employ Wikipedia user request traces [31]. The patterns of hourly user requests are shown in Figure 2. The user request traces contain one job per line. Every line consists of:

• URL;
• Start and finish time of every user request;
• A flag that indicates whether the workload information in a database has been updated or not;
• A counter used to sort the user requests.

6.1.3. Description of Energy Prices

Since we assume that the price of renewable energy is negligible, only the grid electricity price is taken into account. Grid energy prices are dynamically adjusted in response to when electricity is used [59,60]. From 18 September 2018, to 17 October 2018, we utilize the hourly energy prices $q_i\left(t\right)$—in dollar MWh for three geo-dispersed data centers. The wholesale energy market provides these hourly rates, which are available online [14]. Dynamic hourly energy prices are depicted in Figure 3. Due to high levels of renewable energy output and low power consumption, we found that electricity costs in Ontario, Canada are negative at some times of the day.

6.1.4. Baseline Algorithms

To confirm the effectiveness, we contrast RLB-option performance with the following three workload allocation techniques suggested by previous research studies [1,2,17].

i.

Baseline Algorithm-I (BA1) [17]: In this approach, the authors considered energy storage devices powered by brown energy, option pricing, and dynamic energy prices to process the incoming user request. Brown energy from the grid is the primary energy source in BA1. However, neither thermal storage nor renewable energy are considered in this workload allocation strategy.

ii.

Baseline Algorithm-II (BA2) [2]: This strategy ignores option pricing in favour of deploying energy storage devices and the least expensive time-varying power costs to fulfil incoming user demands. In BA2, workloads are sent to the closest geo-distributed data center for immediate processing. Many businesses already use this strategy, prioritizing meeting incoming workloads as quickly as possible over saving money on energy costs or using renewable resources.

iii.

Baseline Algorithm-III (BA3) [1]: In this method, GLB is used exclusively to prioritize incoming user requests, with consideration given only to the time-varying call option. The strategy does not account for energy storage or fluctuating electricity costs in the real-time market, instead relying solely on options from the derivatives market to power data centers.

6.2. Numerical Results

The following is the structure that will be used to arrange the assessment of RLB-Option based on the traces of the incoming user requests, renewable energy, brown energy, and option pricing:

6.2.1. Energy Cost Minimization Using Renewable Aware Load Balancing

Take into account dynamic energy derivatives (i.e., call option) which have been ignored in most prior research works aimed at minimizing energy costs. The average energy cost in a geographically distributed data center for RLB-Option and baseline algorithms is shown in Figure 4. BA3 produces maximum power cost since it does not employ ESDs (i.e., batteries) and dynamic energy costs in the real-time market, instead relying solely on options from the derivatives market to power data centers for workload distribution. The energy cost is much higher in BA2 since it does not account for the use option pricing in favour of deploying energy storage devices and relies totally on brown energy to fulfil incoming user demands. The energy expenses for BA1 are lower than those of BA2 and BA3. The effectiveness stems from an algorithm that processes incoming user requests using battery power, option pricing, and the least expensive electricity available on the spot market to process the incoming user requests.

When compared to the benchmark algorithms, RLB-Option yields the best results. RLB-Option considers renewable energy (i.e., solar), ESDs, option pricing, and the cheapest power rates to dynamically lower energy expenses during task allocation. We found that there are times when the average energy cost is negative because power rates in some areas are negative (see Figure 4).

Table 2. RLB-Option improvement over baseline algorithms.

Comparison Factor	RLB-Option Improvement Over
	BA1	BA2	BA3
Energy Cost	22%	39%	57%

summarizes and compares RLB-Option’s performance against BA1, BA2, and BA3. According to the entries, RLB-Option has increased efficiency by 22% compared to BA1, 39% to BA2, and 57% to BA3.

6.2.2. Minimizing Average Delay Cost

The average delay for the incoming workload is shown in Figure 5. Keep in mind that, in our experiments, the maximum delay was set at 11 ms. Response times are consistently less than the upper limit, indicating that the RLB-Option has attained the highest level of service quality during workload allocation in geo-distributed data centers.

6.2.3. Trade-Off between Delay and Cost

This section focuses on the trade-off between average delay time and overall operational expenses of the geo-distributed DCs in RLB-Option. We select a different value of $V$ and track the cost of the DCs and user request response time in RLB-Option. Figure 6 depicts the result. The figure demonstrates that the analytical performance results are supported by the fact that increasing the parameter $V$ results in a lower total operating cost for RLB-Option at the expense of a somewhat longer time processing the workload. By increasing the value of $V$, RLB-Option significantly decreases the cost of the DC. In contrast, it could delay the processing of user requests until the price of electricity is low, or sufficient green energy is available, resulting in a long queuing delay time.

6.2.4. Impact of ESD Cost

We fix parameters $V$ and $S_i^{max}$ for all data centers and test RLB-Option under various values of the loss of ESDs ${\beta }_i$ to determine the influence of ESDs cost on the operating cost savings. The outcome is depicted in Figure 7. We can see that the operating cost savings decrease as the ESDs cost factor rises. RLB-Option does not use the ESD at all when ${\beta }_i$ is high. However, even in this scenario, there are still cost savings compared to BA1 due to the GLB and interactive workload distribution.

7. Concluding Remarks and Future Work

A key challenge for cloud computing is using a significant amount of energy in geographically dispersed data centers to process user requests. In this research work, we investigated the electricity cost minimization problem for geographically dispersed data centers considering renewable energy, fluctuating power prices, and call options for electricity under variable data center system dynamics. To address this problem, we suggested a provable-efficient online algorithm, RLB-Option, to allocate incoming user requests among multiple geographically distributed data centers. RLB-Option considers renewable energy sources, brown energy, and electricity call option contracts to minimize the energy cost of the data centers. The basic idea of RLB-Option is fourfold: (i) The incoming user requests will be processed using renewable aware ESDs. (ii) If the batteries do not have a sufficient level to process the user requests, then user requests will be processed using the electricity call option in the derivative market. (iii) The workload will be processed using the lowest rate of brown energy. (iv) Finally, renewable energy sources (i.e., solar panels) will be used to charge the ESDs. Experimental results showed that data center energy expenditures were significantly reduced when RLB-Option was applied. Our future work is twofold. First, we will focus on the batch workload and bandwidth cost, which were not accounted for in our formulation and significantly impacted the running costs of geographically dispersed data centers. Secondly, we plan to extend our work to deal with energy-efficient workload distribution using data deduplication and the string-matching technique.