This section summarises the research articles identified in the survey. Articles are presented by layer (cloud, fog, and edge) and within each layer in ascending order based on their year of publication.
4.1. Cloud Computing
Malawski et al. (2015) [41
] examine resource scheduling and provisioning for a scientific workflow ensemble on an Infrastructure as a Service (IaaS) cloud. The goal of the study was to maximise the number of prioritised workflows that can be completed given budget and deadline constraints. Malawski et al. develop three algorithms to solve this problem—(1) a dynamic provisioning dynamic scheduling (DPDS) algorithm, (2) a workflow-aware DPDS (WA-DPDS) algorithm that extends DPDS by introducing a workflow admission procedure, and (3) a static provisioning static scheduling (SPSS) algorithm that creates a provisioning and scheduling plan before running any workflow tasks. This enables SPSS to start only those workflows that it knows can be completed given the deadline and budget constraints, and eliminates any waste that may be allowed by the dynamic algorithms. The algorithms are evaluated using CloudSim on a range of synthetic workflows, which were generated based on statistics from real scientific applications. The results of the simulation studies indicate that the two algorithms that take into account the structure of the workflow and task run-time estimates, WA-DPDS and SPSS, yield better results than the simple priority-based scheduling strategy (DPDS), which makes provisioning decisions based purely on resource utilisation.
Arianyan et al. (2015) [44
] explore consolidation in virtualised cloud data centers to tackle the problem of energy consumption and related increases in carbon emissions. They propose Enhanced Optimisation (EO) policy as a novel resource management procedure in cloud data center and novel multi-criteria algorithms for both phases of resource allocation, and determination of under-loaded physical machines (PMs) in cloud data centers. More precisely, they propose TPSA (TOPSIS (Technique of Order Preference Similarity to the Ideal Solution) Power and SLA-aware Allocation) as a novel policy for resource allocation as well as MDL (Migration Delay), AC (Available Capacity), and TACND (TOPSIS Available Capacity, Number of VMs, and Migration Delay) for determination of under-loaded PMs. Arianyan et al. used CloudSim to test and evaluate the proposed policies and algorithms using synthetic workloads. They compare the simulation results using different parameters. Performance is measured by comparing energy consumption, number of VM migrations and SLA violations.
Kecskemeti (2015) [35
] propose a new IaaS simulation framework called DISSECT-CF for simulating discrete event-based energy consumption for clouds and federations of clouds. It is designed for easy extensability, energy evaluation, and rapid evaluation of multiple scheduling and IaaS internal behaviours. The simulator has five main components: Event System (time reference), Unified Resource Sharing (resource consumption, resource scheduler), Energy Modelling (power state, consumption model), Infrastructure Simulation (VMs, PM, Network node), and Infrastructure Management (VM scheduling, PM scheduling). DISSECT-CF offers two major benefits: (i) a unified resource-sharing model, and (ii) a more complete IaaS stack simulation including, for example, VM image repositories, storage, and in-data-centre networking. DISSECT-CF is analysed by first validating it against the behaviour of a small-scale infrastructure cloud at the University of Innsbruck. The system simulated behaviour matched real-life experiments with negligible error (in terms of application execution time, larger-scale network transfers, and energy consumption). For larger-scale experiments, DISSECT-CF was validated by comparing it to CloudSim [6
] and GroudSim [71
] using both real-world and synthetic traces. The comparison results reveal that DISSECT-CF is scalable and produces more accurate results.
Bux et al. (2015) [43
] propose an extension of CloudSim, DynamicCloudSim, for evaluating resource allocation and scheduling strategies for cloud computing. The extended features of DynamicCloudSim include the ability to model instability in the cloud including homogeneity and dynamic changes of performance at run-time, as well as failures during task execution. DynamicCloudSim allows the simulation of different kinds of tasks (e.g., CPU, I/O, bandwidth-bound, etc.) on VMs with different performance characteristics. To inject greater verisimilitude, DynamicCloudSim supports heterogeneity, randomisation of individual performance of a VM, dynamic changes at runtime due to external loads, and fault tolerant approaches (through the introduction of straggler VMs and failures during task execution). In their paper, Bux et al. assess the impact of instability on the performance of four schedulers (Round Robin, Heterogeneous Earliest Finishing Time (HEFT), Greedy Queue and Longest Approximate Time to End (LATE)) using DynamicCloudSIm and compare results with results from real cloud infrastructure (Amazon EC2).
Magalhaes et al. (2015) [45
] propose an extension of CloudSim in the form of a web application model to capture the behavioural workload patterns of different user profiles, and to support analysis and simulation of resource utilisation in cloud environments. The workload patterns are modelled in the form of statistical distributions, thus the patterns fluctuate based on realistic parameters to represent dynamic environments. The simulation results suggest that web application modelling can produce data to accurately represent different user profiles. A validation model is provided through graphical and analytical methods to show that the simulator effectively represents the observed patterns.
Higashino et al. (2016) [27
] propose an extension to CloudSim called CEPSim for modelling and simulating both complex event processing (CEP) and stream processing (SP) in the cloud. It can be used to study the scalability and performance of CEP queries and to easily compare the effects of different query processing strategies. The simulator can model different CEP systems by transforming user queries into a directed acyclic graph (DAG). The simulator supports resource placement and scheduling algorithms. CEPSim is highly customisable and can be used to analyse the performance and scalability of user-defined queries and to evaluate the effects of various query processing strategies. The CEPsim simulation results are compared to real CEP/SP system results. Experimental results suggest that CEPSim can simulate existing systems in large Big Data scenarios with accuracy and precision.
Castro et al. (2016) [47
] first evaluate the impact of RAM and CPU usage on energy consumption in a cloud data center; then, they propose two joint CPU-RAM approaches for dynamic VM placement in cloud data centers. The former is concerned with the detection of under-loaded servers, and the latter concerned with VM placement. The proposed approaches are tested in CloudSim and the results show that they can reduce energy consumption and SLA violations. However, the approach only examines consolidation on the data center servers and not in the network and storage devices. The technique is computationally heavy.
Da Silva et al. (2016) [46
] propose a topology-aware VM placement algorithm. The goal of the proposed algorithm is to place a set of VMs in data centers in order to minimise the areas of a hierarchical data center network for a set of VM placements, so that fewer switches are needed to serve the network flows. This should lead to consolidated network flows from the VMs. In order to test their algorithm, Da Silva et al. extend CloudSim by adding a data center network topology (fat-tree topology) and an energy consumption model. Simulation results show that the proposed algorithm prevents the formation of network bottlenecks therefore accepting more requests of allocation of VMs, without compromising energy efficiency. The energy consumption of servers and switches are taken into account, and these are switched off whenever idle. However, the simulation addresses only one network topology and considers just one traffic pattern.
Samimi et al. (2016) [42
] propose a new model called Combinatorial Double Auction Resource Allocation (CDARA) to address biased market-based resource allocation by the cloud provider. CDARA focuses on the principle of equitable fairness to both providers and buyers. The scenario of multiple users and multiple cloud providers was simulated using four entities (user, broker, cloud provider, and cloud marketplace). CDARA has seven phases: (1) advertising and resource discovery where the provider proposes his services (resources), the user proposes his requirements, and the Cloud Information Service (CIS) creates the list of resources that matches with the user’s requirements; (2) creating the auction and the bundle; (3) informing the end user of an auction; (4) determining the winner; (5) resource allocation; (6) pricing model; and (7) task allocation and payment. Samimi et al. use CloudSim to evaluate the performance and efficiency of CDARA based on an economic perspective and intensive compatibility. The simulation results show the effectiveness of the CDARA model. In CDARA, the winners are determined by a mediator; determining an optimal winner that maximises total profit for the cloud providers would seem to remain an open challenge.
Cai et al. (2017) [48
] propose Delay-based Dynamic Scheduling (DDS) algorithm for dynamic resource provisioning and scheduling to minimise resource renting costs while meeting workflow deadlines. New VMs are dynamically rented by the DDS according to the practical execution state and estimated task execution times, to fulfil the workflow deadline. Bag-based deadline division and delay scheduling strategies consider the bag structure to decrease the total renting cost. Cai et al. use a simulator called ElasticSim, an extension of CloudSim, to evaluate the performance of their proposed algorithm. The simulation results suggest that the dynamic algorithm decreases the resource renting cost while guaranteeing the workflow deadline compared to existing algorithms.
Lin et al. (2017) [50
] propose an extension for CloudSim, MultiRE-CloudSim. They add a multi-resource scheduling and power consumption model which allows for a more accurate evaluation of power consumption in dynamic multi-resource scheduling for cloud computing. MultiRE-CloudSim offers the following capabilities to CloudSim: (1) the ability to change the configuration of host and power models of resources easily, and test the effect of the algorithm under different parameters; (2) simulation of tasks that demand multiple types of resources and define different resource allocation algorithms with fine-grained evaluation; (3) seamlessly switch between static load and dynamical load experiments which makes it able to simulate more scenarios; (4) power simulation of multi-resource scenarios which is more accurate when compared with single resource CPU power simulation. Lin et al. evaluate the performance of their proposed simulator by comparing the results of the simulation with different parameters such as task makespan, power consumption, and SLA violation.
Arianyan et al. (2017) [51
] examine the key objectives in cloud ecosystems i.e., energy consumption, SLA violation, and number of VM migrations. They consider the importance of different system criteria including CPU, RAM, and network bandwidth in the decision-making process. Resource management is addressed by proposing novel Dynamic Voltage and Frequency Scaling (DVFS)-aware algorithms to solve the four sub-problems of an online consolidation problem as well as a novel DVFS governor. The four consolidation sub-problems are (1) determination of overloaded PMs, (2) determination of under-loaded PMs, (3) selection of VMs that should be migrated from overloaded PMs, and (4) placement of migrating VMs on PMs. They propose novel fuzzy multi criteria and an objective resource allocation algorithm that enables resource administrators to weight different criteria in the resource management solution by importance using fuzzy weights. CloudSim is used to evaluate the performance of the proposed algorithms. Experimental results suggest that the combination of the proposed resource management policies leads to a notable reduction in both energy consumption and SLA violations compared to benchmarks.
Ranjbari et al. (2018) [52
] not only evaluate CPU usage but propose an algorithm using learning automate overload detection (LAOD) to predict CPU usage of PMs based on user resource usage history in order to improve resource utilisation and reduce energy consumption. The technique is tested using CloudSim and compared to benchmark algorithms. The results suggest that the approach outperforms comparators in reducing data center energy consumption. In addition, this study illustrates the benefit of using learning algorithms to predict resource usage for efficient resource management in the cloud.
Mishra et al. (2018) [56
] propose Energy-aware Task-based Virtual Machine Consolidation (ETVMC), a new task-based VM placement algorithm for mapping tasks to a VM, and a VM to a PM, in the cloud. The goal of ETVMC is to efficiently allocate tasks to VMs and then VMs to hosts so the allocation minimises energy consumption, makespan and task rejection rate. The proposed approach is evaluated and tested using CloudSim, and the simulation results demonstrate the effectiveness of the proposed algorithm compared to FCFS (First-Come First-Served), Round-Robin and EERACC (Energy Efficient Resource Allocation in Cloud Computing).
Gawali et al. (2018) [53
] propose an heuristic approach for optimising task scheduling and resource allocation in the cloud. The heuristic is a combination of four methods: (i) the modified analytic hierarchy process (MAHP), (ii) bandwidth aware divisible scheduling (BATS) + BAR (Balance-Reduce) optimisation, (iii) longest expected processing time preemption (LEPT), and (iv) divide-and-conquer methods. The MAHP is used to process the task before its allocation in the cloud while BATS + BAR optimisation is used to allocate the resources for that given task. The optimisation takes into consideration the bandwidth and cloud resources as constraints. The LEPT is used to preempt resource-intensive tasks. The divide-and-conquer method is applied to aggregate the results after task preemption (scheduling and allocation). Gawali et al. use CloudSim to test and evaluate their proposed approaches. Experimental results suggest that the proposed approach outperforms the existing BATS algorithm in terms of turnaround time and response time metrics.
Filelis-Papadopoulos et al. (2018) [22
] propose a highly parallel MPI- and OpenMP-based simulation framework for hyperscale cloud computing, the CloudLightning Simulator. The simulation implements a sequential search and deploy algorithm that iterates through available resources to deploy VMs and tasks. The validation of the framework is based on number of tasks deployed, task execution time and resource utilisation parameters. Results suggest that the proposed simulator can cater for extremely large numbers of cloud nodes, the execution of extremely large numbers of tasks, and can support CPU, memory and network over-commitment.
Kumar et al. (2018) [57
] propose a resource allocation model for processing applications efficiently in the cloud, PSO-COGENT. PSO-COGENT uses the meta-heuristic based particle swarm optimisation (PSO) approach to allocate the task to resources in an optimal way. PSO-COGENT optimises execution time, cost, and reduces energy consumption in cloud data centers, considering deadline as a constraint. The problem is formulated as a multi-objective scheduling problem in the form of a mathematical model and defined objective and fitness functions. Kumar et al. use CloudSim to evaluate the performance of their proposed algorithm. Simulation results show PSO-COGENT reduces execution time, execution cost, task rejection ratio, energy consumption, and increases the throughput in comparison to benchmark algorithms i.e., POS, Honey Bee and Min-Min.
Al-Mansoori et al. (2018) [55
] propose a dynamic resource optimisation methodology for SDN-enabled Big Data Stream Processing (BDSP) in public clouds. Their algorithm, named SDN Base VM Mapping for Resource Optimisation (SBVMMRO), uses CPU, memory and storage metrics to allocate workload. They also consider BDSP workloads including networking and computation. Al-Mansoori et al. use an extension of CloudSim, CloudSimSDN, to validate their methodology by comparing delay times and the number of servers needed to process different types of BDSP workloads. Experimental results reveal the utility of an SDN-based approach for cloud resource optimisation for BDSP by decoupling data forwarding and network controlling.
Filelis-Papadopoulos et al. (2018) [37
] examine the performance of a novel dynamic cloud resource allocation framework based on the principles of self-organisation and self-management (SOSM). The evaluation is performed using a warehouse scale model of a cloud data center and a parallel time-advancing loop-based simulation framework. Simulation experiments take into account resource heterogeneity including hardware accelerators such as Graphics Processing Units (GPU), Many Integrated Cores (MIC) and Field Programmable Gate Arrays (FPGA). The SOSM framework evaluation is conducted by analysing resource utilisation parameters such as CPU, memory, network and energy consumption. Results suggest that the proposed approach enables more accurate decisions on the placement of VMs over resources compared to the traditional centralised management approach.
Fernández-Cerero et al. (2018) [58
] propose SCORE, a hybrid discrete-event multi-agent scheduling and resource utilisation simulator to reduce energy consumption in cloud computing environments. SCORE comprises two parts: (a) an energy-aware independent batch scheduler; and (b) a set of energy-efficiency policies for the hibernation of idle VMs. Four scheduling policies have been proposed for the control of energy consumption and the makespan during the assignation of tasks to VMs. The proposed scheduler takes into account the security demands of each task and trust levels of VMs that are computing those tasks. Additionally, the proposed simulator enables the compute of energy consumption for the whole system including the energy spent on performing security operations. Fernández-Cerero et al. test and validate their proposed simulator using different policies and parameters. Results suggest a reduction of up to 45% of cloud system energy consumption costs.
Madni et al. (2019) [68
] propose a Multi-objective Cuckoo Search Optimisation (MOCSO) algorithm for dealing with the resource scheduling problem in cloud computing. The algorithm reduces costs for cloud users and enhances performance by minimising task execution time and maximising resource utilisation. Improved resource utilisation enables increased revenue for cloud providers as more user requests can be processed with existing hardware. The MOCSO algorithm is evaluated using CloudSim and its performance compared against other multi-objective algorithms i.e., ant colony, genetic, multiple order model migration and particle swarm. Simulation results suggest that the proposed MOCSO algorithm performs better than the existing algorithms, and that it balances multiple objectives for expected time to completion and expected cost to completion matrices for resource scheduling in the IaaS cloud computing environment.
Zakarya et al. (2019) [64
] extend CloudSim to support the modelling of heterogeneous workloads and resources such as CPU in the cloud. The extended features also support scheduling policies, VM migration and energy consumption. Energy consumption is optimised by solving the problem of heap memory, by removing the VM object when finished and forcing the Java garbage collector to clear memory. Zakarya et al. also use several performance benchmarks from Amazon EC2, relate them to Google workload and feed them into their extended CloudSim to model variations in workload run-times. Simulation results are first compared to small-scale real measurements (obtained on the University of Leeds test-bed) and then compared to the results of simulations with DISSECT-CF. Zakarya et al. statistically validate the proposed allocation and migration policies with 29 different datasets from the Google production cluster. Simulation results suggest that the accuracy and precision of this extended version of CloudSim reaches 98.63%. Results suggest that proposed policies can reduce energy consumption by up to 3.66%.
Priya et al. (2019) [65
] propose a Fuzzy-based Multidimensional Resource Scheduling and Queuing Network (F-MRSQN) in cloud data centers. F-MRSQN includes three stages to be performed between the users and the servers in a cloud environment. Stage 1 obtains incoming requests from the cloud users; Stage 2 is online fuzzy-based multidimensional resource scheduling by the resource manager; and Stage 3 performs load optimisation using Multidimensional Queuing Network. The problem was modelled as a multi-objective optimisation problem that optimises the resource scheduling and load balancing while minimising the resource utilisation and processing time in the cloud environment. Simulation analysis suggests that F-MRSQN achieves better performance in terms of average success rate, resource scheduling efficiency and response time. It also suggests that F-MRSQN improves resource scheduling efficiency by 7% and reduces response time by 35.5% when compared to the benchmarks.
Moghaddam et al. (2019) [63
] propose an extension of CloudSim by embedding new functionalities such as dealing with anomaly detection and elasticity of the resources (VMs) in the cloud environment. The proposed Anomaly and Cause Aware auto-Scaling (ACAS) framework consists of three main modules, monitoring, a data analyser (to detect anomalies), and a resource auto-scaler, which exploits two types of the resource adjustment policies, horizontal and vertical scaling. Horizontal policies address resource configuration strategies that change the number of active VMs in the system, whereas vertical policies are defined at finer grains of control (Elastic VMs) and adjust the number of allocated resources based on the new demands of the VM. An extensive experimental evaluation of the proposed system is done under various loads. Results demonstrate the ability of ACAS to maintain better quality CPU and response time performance compared to existing resource management solutions.
4.2. Fog Computing
Gupta et al. (2017) [28
] propose an extension to CloudSim, iFogSim, which is capable of modelling fog computing and IoT scenarios. It enables simulation of resource management and application scheduling policies across edge and cloud resources under different scenarios and conditions. Additionally, iFogSim allows application designers to validate the design of their application with regards to cost, network use, and latency. Resource management is the core component of the iFogSim architecture and consists of components that coherently manage resources of the Fog device layer in such a way that application-level QoS constraints are met and resource wastage is minimised. The paper also presents a simple IoT simulation recipe and two case studies to demonstrate how one can model an IoT environment and plug in and compare resource management policies.
Sood (2018) [54
] propose a social network analysis (SNA) technique to detect resource deadlock at the fog layer. SNA is a rule-based approach for priority resource allocation. If the deadlock is detected, SNA tries to find free resources from jobs already allocated, and if the deadlock is not solved, the job is sent to the cloud. The allocation is based on rules and priorities. Sood uses CloudSim to evaluate the performance of their proposed technique and examine resource utilisation and request processing metrics. Simulation results suggest that SNA provides an effective solution to detect and solve deadlock in the fog and enables efficient resource utilisation in both the cloud and fog layers. Notwithstanding this, the study does not compare SNA to other state-of-the-art techniques.
Mahmoud et al. (2018) [60
] propose an energy-aware allocation policy for placing application modules in fog devices. It is an energy-efficient strategy that allocates the incoming tasks to Fog devices based on the remaining CPU capacity and energy consumption. The aim of the proposed allocation strategy is to increase energy efficiency at fog devices by allocating application tasks based on improved Round Robin and DVFS algorithms. Mahmoud et al. use iFogSim to evaluate the performance of the proposed approach. Remote patient monitoring was used as the use to evaluate the energy efficiency of the proposed placement strategy. The performance of the proposed strategy is evaluated by comparing it with the default allocation and cloud-only policies. Simulation results suggest that the proposed solution is more energy-efficient with approximately 2.72% in energy savings compared to cloud-only policies and approximately 1.6% in energy savings compared to the fog-default.
Qayyum et al. (2018) [61
] propose FogNetSim++, a novel simulation framework that provides detailed configuration options to simulate large fog network scenarios. The framework is based on OMNet++ and incorporates customised mobility models, fog node scheduling algorithms, and handover management mechanisms. FogNetSim++ can simulate network traffic at a packet level including packet drop, delay, network errors, handovers, latency, and re-transmissions. To evaluate the performance of the simulator, a communication network scenario is simulated based on a smart traffic management system. The simulation analysis demonstrates the scalability and effectiveness of the FogNetSim++ in terms of CPU and memory usage. The network parameters, such as execution delay, packet error rate, handovers, and latency are also benchmarked to demonstrate the FogNetSIM++’s ability to simulate network functionality.
Naranjo et al. (2018) [62
] present a study of a popular applications used in the Fog/IoT paradigm, followed by an architectural design for resource management using a container middleware layer. The proposed architecture allows dynamic scaling of compute and network resources to meet the demands of IoT and meet required QoS. In addition, the proposed management architecture is extended by a bin packing heuristic that works with scaling options to jointly reduce energy consumption. The approach is evaluated using two scenarios in iFogSim, a static and a mobile application, and compared against Modified Best-Fit-Decreasing (MBFD) and Maximum Density Consolidation (MDC) algorithms. Results suggest that the average energy consumption of the proposed heuristic is 20.36% and 33.84% less than benchmark algorithms.
AbdElhalim et al. (2019) [70
] use iFogSim to explore the latency of user nodes offloading tasks in a hyper-dense fog-to-cloud computing network. AbdElhalim et al. introduce adaptive low latency networks, irrespective of the number of user nodes, based on game theory. Game theory was applied to reduce energy consumption on the fog nodes as well as maintaining an acceptable latency. They also develop automated distributed fog computing for computational offloading using the theory of minority games. Resource allocation was performed in a distributed manner for better network management. Simulation results suggest that the proposed algorithm achieves the user satisfaction latency deadline as well as the required QoE. Moreover, it guarantees an adaptive equilibrium level of the Fog-to-Cloud computing system, which is suitable for heterogeneous wireless networks.
Talaat et al. (2019) [67
] propose an Effective Load Balancing Strategy (ELBS) for fog computing environments using data from a medical sensor application use case. Their solution includes a Priority Assigning Strategy (PAS), a Data Scheduling Algorithm (DSA), an External Data Requesting Algorithm (EDRA), a Server Requesting Algorithm (SRA), and a Probabilistic Neural Network based Matching Algorithm (PMA). PAS is used to classify incoming tasks based on their priority. DSA is used to locate the data on a local file system for task execution, and EDRA used to fetch data from remote file system if it is not present at the task processing location. The SRA algorithm allocates resources to process tasks by liaising between the Fog Master Server and Master Server Manager. Finally, PMA is used to match tasks with the server in a master priority queue. The proposed approach efficiency is demonstrated by modelling the system and medical sensor data use-case in iFogSim. Simulation results suggest that ELBS outperforms benchmarks achieving the lowest average turnaround time and failure rate. Accordingly, ELBS may be a suitable strategy for load balancing in the fog as it supports reliable execution for real-time applications.
Guerrero et al. (2019) [69
] focus on resource provisioning in fog environments. They propose a lightweight decentralised service placement policy for performance optimisation in fog computing in order to optimise service placement and so reduce latency and network usage. iFogSim is used for evaluation and results compared to the existing (Edgewards) placement policy, which support the efficiency claims of the proposed approach.