2.1. Computational Offloading
Computational offloading can be implemented offline or online. In offline implementation, all system information needed to make the offloading decision is previously known and is based on historical or predictive knowledge, such as the computational capabilities of fog nodes, the total number of IoT devices, and their total workload (number of requests). This is applied during the system design stage. In online deployment, the computational offloading decision takes place at run-time and considers the current system status and process characteristics, such as the current waiting time and the current available computational resources, without prior knowledge of system inputs considered in the offline deployment. Several studies investigate computational offloading in offline deployment [8
]. In [19
], Wang et al. investigated the optimized offloading problem to minimize task completion time given tolerable delay and energy constraints. The optimization problem was formulated as a mixed integer nonlinear programming problem that jointly optimizes the local computation capability for IoT devices, the computing resource allocation of fog nodes and the offloading decision. Wang et al. [19
] decomposed it into two independent sub problems to find the optimal amount of workload that should be processed locally at IoT devices and at fog nodes. A hybrid genetic-simulated annealing algorithm was developed to optimize the offloading decision. Tang et al. [18
] aimed to increase the total number of executed tasks on IoT devices and fog nodes under deadline and energy constraints. The authors in [18
] considered this as a decentralized, partially observable offloading optimization problem in which end users are partially aware of their local system status, including the current number of remaining tasks, the current battery power, and the nearest available fog node. Such parameters are used to assess if tasks should be processed locally or offloaded to the nearest fog node. Their approach enables IoT devices to make an appropriate decision based on its locally observed system.
Liu et al. [11
] addressed a multi-objective optimization offloading problem in a fog environment with the aim of minimizing execution delay, energy consumed at mobile devices, and offloading payment cost for using fog/cloud resources. The multi-objective problem was formulated into a single problem using scalarization method [11
]. The proposed solution found the optimal offloading probability that accomplishes the stated objectives. Mukherjee et al. [20
] designed an offloading technique focusing on jointly optimizing the computing and communication resources at fog systems to reduce end-to-end latency. Their technique considers the trade-off between transmission delay and task execution delay when making the offloading decision, in which a fog node can seek additional computational resources from either one of its neighbors, or the cloud data center, to reduce task execution delay at the expense of the transmission delay. The optimization problem was transformed into convex quadratically constraint quadratic programming and solved using CVX toolbar, which is a MATLAB-based modelling system for convex optimization. Their simulation results demonstrated that their proposed solution offers minimal end-to-end latency in comparison to executing all tasks at end-user devices and executing all tasks at the primary fog nodes.
Zhu et al. [13
] proposed a task offloading policy based on execution time and energy consumption. This approach helps mobile devices to make an appropriate decision on whether to process their tasks locally or offload them to a fog node, or the cloud. During the decision-making procedure, mobile devices calculate both the execution time and the energy consumed when executing the task on the local device and compare this with the execution time and the energy consumed when offloading and receiving the processed task on a fog node; the energy consumed when executing the tasks on fog nodes are not considered. Based on this comparison, the IoT device makes a decision with the least cost (execution time plus energy consumption). Comparing their scheme to Random, no offloading, and only offloading when considering only execution time, their simulation results showed an optimization of the execution time of tasks and energy consumption of mobile devices. Mukherjee et al. [21
] formulated the offloading problem as an optimization problem with the goal to minimize the total system cost, which is the sum of the total delay of end-users’ tasks and the total energy consumed at end-users’ devices due to local processing of tasks and uploading tasks to the fog environment for processing. Under delay and energy constraint, the optimization problem was transformed into a quadratically constraint quadratic programming problem and solved by semidefinite relaxation method. Within a heterogeneous environment where fog nodes have different computational resources, the proposed solution enables the optimal amount of workload to be identified that should be processed at end-user devices, primary and neighboring fog nodes, and cloud servers. The decision on when to offload depends entirely on the availability of computational resources. The authors stated that having higher computational resources at fog nodes helps to reduce the system cost. Increasing number of end-users leads to greater congestion at fog nodes, leading to fog nodes preferring to send their workload to the cloud server for processing rather than other neighboring fog nodes.
Chen and Hao [14
] studied offloading problem in dense software-defined networks, formulating this as a mixed-integer nonlinear problem that is decomposed into: (i) deciding whether the task is processed locally at the end-user device or offloaded to the edge device; (ii) determining the computational resources that are dedicated to each task. Chen and Hao [14
] developed an efficient software-defined task offloading scheme to solve these sub-problems. The results of their proposed scheme demonstrated the superiority of their approach at decreasing end user device energy consumption and overall task execution latency. In IoT-Fog-Cloud architecture, Sun et al. [22
] presented the “ETCORA” algorithm, which consists of two parts. The first part aims to find the optimal offloading decision based on minimizing time and energy, and the second part optimizes resource allocation in terms of transmission power allocation. Their proposed solution helps to minimize energy consumption and completion time of tasks compared to other schemes. Zhao et al. [12
] investigated the computational offloading problem in the context of radio access networks to reduce the weighted sum of total offloading latency plus total energy consumption. To improve the offloading decision and enhance the allocation of computation and radio resources, the authors formulated the problem as a non-linear, non-convex joint optimization problem. Their proposed solution was more effective than mobile cloud computing (MCC), which processes all end-user tasks on a cloud server, and mobile edge computing (MEC), which processes all end-user tasks in the edge computing system. The reason their approach was more effective compared to MCC and MEC is that it made use of a combination of available resources at the cloud and fog nodes, compared to cloud only
as in MCC, and edge only as in MEC.
The hybrid-computational offloading optimization problem was investigated by Meng et al. [23
], where two types of models were considered; namely cloud computational offloading and fog computational offloading. The authors aimed to minimize the consumption of energy caused by transmitting and processing tasks at mobile terminals, fog, and cloud servers under deadline constraints. Meng et al. [23
] introduced a new concept called computation energy efficiency
that is defined as “the number of computation tasks that are offloaded by consuming a unit of energy”, to solve the optimization problem. Based on the proposed solution that considers offloading tasks to fog and cloud servers for execution, simulation results show the effectiveness of the solution compared to only offloading tasks to either cloud only or fog only resources. Xiao and Krunz [8
] proposed a workload scheduling method that ensures user response time is minimized under available power constraints. In their study, the energy spent while processing tasks was ignored and only the energy consumed for offloading each unit of received workload was considered. Cooperation between fog nodes to offload workload by an agreement between neighboring nodes, the workload arrival rates, and the workload processing capabilities determines the amount of offloading carried out. Their experimental results indicated that the average response time decreased due to allowing cooperation between fog nodes. Additionally, a crucial trade-off between the fog node’s power efficiency and the average response time was observed. Xiao and Krunz [8
] proposed that the response time of end-user tasks should be set to its highest tolerable point to optimize energy consumption at fog computing systems. This enables most of the tasks to be processed at end-user devices, avoiding any offloading.
Regarding online deployment of computational offloading, few studies have addressed this, such as [15
]. Yousefpour et al. [16
] suggested a delay-minimization approach to reduce overall service delay. In their approach, the estimated queueing delay, which is utilized as the offloading threshold, determines whether a fog node processes its incoming task(s), or offloads these to one of its neighbors or the cloud server. If the offloading threshold has been reached, then the best neighboring fog node in its domain is selected to offload its upcoming tasks. The best neighboring fog node is chosen based on having the minimum total of propagation delay and queuing delay. Compared to other models, their results achieved the minimum average service delay. Yin et al. [24
] determined where to process end user tasks into task scheduling and resource allocation problems, where tasks are either processed locally at end-user devices or offloaded to fog nodes or cloud servers. In an intelligent manufacturing environment, the authors introduced fog computing and utilized the concept of the container within the fog system, intending to reduce overall delay and optimize the number of concurrent tasks for the fog node. In their online model, generated tasks by end-users are transmitted to the request evaluator, which is located at a fog node that decides whether to accept or reject the task based on its deadline requirement. If the task is accepted, then the task is transmitted to the task scheduler, which determines whether the task is processed at fog nodes or cloud servers based on the available resources and the execution time of this task, which involves computation and transmission time. Finally, the resource manager is responsible for reallocating the required resources to process the task at fog nodes. Experimental results showed the effectiveness of their approach compared to other benchmarks.
Al-Khafajiy et al. [15
] proposed an offloading mechanism that allows fog-to-fog collaboration in heterogeneous fog systems, intending to minimize overall service latency. Their mechanism utilizes a FRAMES load balancing scheme that aims to detect congested fog devices, determine the amount of workload located at fog devices’ queues that require offloading, based on their deadline requirement, and finally select the best fog node that provides the minimal service latency for the selected workload. They evaluated their proposed mechanism using a simulation. Their numerical results indicated the effectiveness of their proposed model in terms of minimizing overall latency in comparison with different algorithms. In a fog-cloud computing system, Gao et al. [17
] investigated the issue of dynamic computational offloading and resource allocation. In order to reduce energy consumption and delay while having a stable queueing status, the authors formulated the problem as a stochastic network optimization problem. They provided a predictive approach to computational offloading and resource allocation that depended on the trade-off between delay and energy use. Their approach implied that a delay reduction can be induced by increasing the allocation of computational resources at fog nodes; however, because of the processing of more tasks, energy consumption increases, and vice versa. Compared to other systems, the authors showed the importance of their method. Mukherjee et al. [25
] developed a scheduling strategy that managed to fulfil the deadline constraint of end-user tasks, taking into account computational resources. The deadline constraint of a given task and the availability of a neighbor, in their scheduling policy, help to decide on whether to place a given task in the fog node queue, e.g., in its high-priority queue or low-priority queue, or offload it to one of its neighboring fog nodes. Their findings illustrated the efficacy of their suggested strategy as opposed to the no offloading and random schemes. Table 1
presents a summary of relevant articles concerning computational offloading at fog computing systems and the forms in which these systems execute.
2.2. Dynamic Server Energy Management
Dynamic Server Energy Management has been used in the wireless local area network and the cloud, and it has proven to be efficient in terms of improving power quality. Although up to the time of our study, this has not yet been implemented in the fog area. In WLANs, the energy efficiency was enhanced by placing access points (APs) in sleep mode or turning them off. In [27
], Marsan and Meo observed that in a community of APs, getting one AP in each community to control the system and service the incoming clients when all others are turned off will minimize energy consumption by up to 40 percent. Furthermore, an additional 60% of consumed energy can be saved if all APs are turned off, especially during idle periods, e.g., at night. Li et al. [28
] suggested an energy-saving method for state transformations in which APs are not only turned on and off based on consumer requirements, but there is also an intermediary stage that aims to reduce the frequency of switching. The authors stated that increasing the switching frequency will shorten AP’s service life. In addition to that, the intermediary stage will help to avoid latency and energy overhead caused by switching on APs.
It was suggested that servers could be periodically switched off [29
] or placed into sleep mode [31
] in cloud computing systems to conserve energy resources. In [29
], the authors examined the issue of the placement of virtual machines (VMs) to save resources concerning energy and yet retain QoS. When underutilized data centers are detected, all VMs are migrated to other active data centers, and these underutilized data centers are placed in sleep mode according to [31
] or shutdown as per [29
]. This is intended to reduce the consumption of energy at cloud computing systems and is called ‘VM consolidation’. Numerous VM migration approaches were suggested to assess which virtual machines can be migrated from overloaded data centers. Moreover, in order to satisfy the QoS specifications of the system, a switched-off data center may also be activated to handle the migrated VMs. According to Mahadevamangalam [31
], the energy demand for an idle data center is ~70% percent of the energy generated by a fully utilized data center. Thus, by switching off idle-mode data centers, up to 70% of the energy consumed can be saved in the cloud system.
2.3. Comparison of the State-of-the-Art
provides a summary of related work in computational offloading in fog computing systems, highlighting the architecture model, e.g., IoT-Fog means that end-user tasks are processed locally at IoT devices or offloaded and processed at fog nodes, use of fog cooperation, communication, the stated objectives of the work, and evaluation tools arranged by offline or on-line offloading decisions. Offline deployment helps to predict the best output for the system at its design stage; and online deployment mimics various scenarios in real-world environments, involving uncertainty and unpredictable events, and helps the system to produce a better outcome. However, most of the literature is focused on offline deployment. Additionally, the problem of computational offloading is usually investigated with the aim of minimizing an overall delay in the system; managing system resources is sometimes included, especially minimizing energy consumption of IoT/end user devices.
Managing resources in the system is much easier within offline deployment than online deployment, especially when all the system data is known in advance. In offline deployment, most attention has been given to addressing the energy consumed at IoT devices compared to fog devices. Additionally, when considering energy consumed at fog nodes, often the trade-off between delay and energy has been investigated.
In this work, we consider online deployment of computational offloading and the potential for minimizing energy consumption at fog nodes (compared to energy consumption of networks or cloud servers). Computational offloading and resource management at fog environments has received limited attention so far. When considering computational offloading, existing efforts utilize a fixed threshold that determines when to start offloading; in the current work, a dynamic threshold is investigated to address its impact on the system.