Simulating Fog and Edge Computing Scenarios: an Overview and Research Challenges

: The fourth industrial revolution heralds a paradigm shift in how people, processes, things, 1 data and networks communicate and connect with each other. Conventional computing infrastructures 2 are struggling to satisfy dramatic growth in demand from a deluge of connected heterogeneous end 3 points located at the edge of networks while, at the same time, meeting quality of service levels. The 4 complexity of computing at the edge makes it increasingly difﬁcult for infrastructure providers to plan 5 for and provision resources to meet this demand. While simulation frameworks are used extensively in

. Cloud, fog and edge domains (based on Mahmud et al. [8]) Fog and edge computing provide new and significant architecture design challenges for all of those 42 involved in the 4IR/IoT chain of service provision. Fog and edge computing use cases vary by the extent 43 and degree of (i) contextual location awareness and low latency, (ii) geographic distribution, (iii) scale and 44 coordination of end-point networks, (iv) heterogeneity, interoperability and functionality of end points, (v) 45 real-time vs batch processing, (vi) mobility of end points, and (vii) interplay between the edge, the fog 46 and the cloud layers [4]. These factors determine the extent to which quality of service levels can be met, 47 performance bottlenecks avoided, energy consumption optimised, under-utilization reduced, and costs 48 managed efficiently and effectively [9]. However, the complexity and scale of these use cases are orders of 49 Version June 10, 2020 submitted to Future Internet 3 of 15 magnitude greater than conventional enterprise and cloud computing scenarios. Successful deployment 50 of 4IR/IoT use cases requires optimal placement of computation and storage along the C2T continuum on 51 a case by case basis and new approaches to test resource placement and management strategies at scale. 52 Like cloud computing, researchers and developers seeking to test resource placement and 53 management strategies for fog and edge computing face a number of significant challenges. Firstly, 54 commercial service providers typically do not give the necessary infrastructure access or control to third 55 parties [10]. Secondly, establishing a test bed with a high degree of verisimilitude is both complex, costly, 56 resource and time-intensive. Thirdly, from a research perspective, the use of commercial third party 57 services and proprietary test beds limit the extent to which experiments can be validated and results 58 reproduced.

59
Cloud computing researchers have attempted to overcome these issues through the application 60 of a range of modelling techniques. For example, Petri Nets [11],Markov Chains [12], Fault Tree (FT), 61 Reliability Graphs (RG) and Reliability Block Diagrams (RBD) [13] have all been used as analytical 62 modelling techniques in distributed system research. A Petri Net is a mathematical modelling language that 63 is commonly used to describe dynamic and parallel system behaviour in order to analyse the performance 64 and availability metrics of systems capable of being clearly described using mathematical definitions [14]. 65 In Markov chains, system models are defined as a sequence of stochastic events that can be used for 66 estimating system behaviour using complex probability distributions. The increasing scale, complexity 67 and heterogeneity of C2T systems renders the use of traditional mathematical modelling techniques 68 difficult to utilise, however simulation is increasingly being adopted as a suitable approach due to its 69 ability to model systems with such characteristics. 70 Given this, the use of simulation frameworks [15,16] has come to greater prominence in cloud 71 computing research. Service configuration and resource placement and management strategies can be 72 simulated prior to infrastructure deployment, performance can be optimised, technical and commercial 73 hypotheses tested, and research results validated and reproduced in a low cost, low risk and often 74 time-sensitive manner [10]. Research on fog and edge computing is still at a relatively early stage of 75 development. Unsurprisingly, research on simulation frameworks to support fog and edge computing 76 and the multitude of use cases that 4IR and IoT represent is lagging.

77
While there are a number of research surveys and articles exploring the challenges of fog computing 78 (e.g. [17,18]), edge computing (e.g. [19,20]), simulation frameworks for cloud computing (e.g. [15,16]) and 79 IoT data analytics (e.g. [21]), there is a paucity of publications addressing the challenges in modelling and 80 simulating fog and edge computing specifically. This paper complements existing works on fog and edge 81 computing by presenting the challenges and design considerations for simulation frameworks of fog and 82 edge infrastructures. Our aim is to support both computing and simulation researchers entering this field 83 in determining the requirements of the simulation platform that they would need to develop or employ to 84 evaluate their technical proposals.

85
The remainder of this paper is organised as follows. The next section discusses challenges of fog and 86 edge computing modelling and simulation. This is followed by a review of existing simulation frameworks 87 for fog and edge computing and a gap analysis against support for four key criteria -(a) infrastructure 88 level modelling, (b) application-level modelling, (c) resource management modelling, (d) mobility, and 89 (e) scalability. Our analysis suggest significant gaps in relation to requirements to model mobility and 90 scalability, after which the article concludes.  were not designed to cater for the processing, storage and data generated by billions of distributed end 97 points operating in often dynamic environments with intermittent network connectivity. Unsurprisingly, 98 service providers have faced unprecedented challenges planning and managing for greater demands 99 while meeting minimum service levels. Fog computing has emerged to complement cloud computing.

100
As discussed earlier, fog computing is located between smart end-devices at the edge of networks and 101 traditional cloud or data centres [4]. It plays an important role in reducing network congestion and 102 facilitating location awareness, mobility support, real-time interactions, scalability and interoperability 103 [8,22]. In contrast, edge computing, in its purest sense, defined by the exclusion of cloud and fog, and 104 is limited to a small number of peripheral network layers [4]. Both fog and edge computing present 105 significant challenges for service providers and researchers including application architecture design and 106 deployment, infrastructure and network management, mobility, resource management, and scalability.   IoT services are used to offload data processing and storage from sensors to edge locations selectively 121 pushing some of the data up the network stack to the cloud. VNFs are chains of network functions that 122 handle mobile network protocol traffic (e.g. LTE stack) or provide network traffic filtering and routing 123 functions, such as enterprise, firewall and VPN services. Law [23] suggests "a simulation model should always 124 be developed for a particular set of objectives. In fact, a model that is valid for one objective may not be for another.".

125
Modelling all of the applications deployed within a fog/edge network can be beneficial for infrastructure 126 providers but constructing a simulation solution that can efficiently handle a set of such broad objectives is 127 a challenge and needs careful consideration. [25]. However, in order to build meaningful system behaviour models the monitoring data has to undergo 149 additional processing to extract behaviour trends of workload and application resource demands. Such a 150 process brings big data management and processing challenges into play that require further development 151 within the scope of the simulation domain. effectively requires a number of key decisions, not least how the data will be collected, where and how data 172 will be processed (edge, fog or cloud), and how often the data should be sent to the cloud for long-term 173 storage or further analysis. There are two competing pressures informing these decisions -utilisation of 174 infrastructure and end user quality of service.

175
Complex Event Processing (CEP) systems are increasingly cited for processing and analysing high 176 volumes of data and detecting events of interest when they occur [28,29]. However, the CEP task can 177 be time-consuming, and commonly fog or edge devices present computational and storage capacity 178 limitations, when compared to cloud capacity and capability. As such, caching is used widely to bring 179 storage functionality to network edges with lower latency, less excessive bandwidth consumption, and 180 reduced streaming times [19,30,31]. This is particularly the case for content distribution use cases such as 181 IP video, forecast to account for a significant portion of all IP traffic in the coming years [32]. Wang et al. 182 Version June 10, 2020 submitted to Future Internet 6 of 15 [19] summarise the main challenges in content caching in edge networks as caching placement, content 183 popularity, caching policies and algorithms, and mobility awareness.

184
Understanding data load generation and its propagation through a given system is a worthwhile 185 approach for deciding on (optimal) resource placements. Data load prediction has been presented as a 186 solution for proactive system remediation [33]. In this case, historical (big) data stored at cloud and live 187 data collected in the fog and edge devices are used to feed models and predict important metrics, such   The simulation approach has proven to be a worthwhile endeavour in testing resource allocation and 217 management in cloud. In their study, Stier et al. [45] presents direct integration between the simulation 218 and optimisation framework which was implemented in order to test available resource management 219 algorithms that can be directly used within a real system. In addition simulation can also be used as part using Hazelcast and Infinispan in memory distributed data stores [52]. As seen in Table 1 parallel DES 254 execution is not widely adopted in newly released edge simulation tools limiting their application range.

290
According to Dastjerdi and Buyya [59], in order to enable real-time analytics in fog and edge 291 computing at the software-level, we must be concerned about different resource management and 292 scheduling techniques including resource distribution, load balancing, migration, and consolidation.

293
At the physical layer, fog and edge systems have many additional requirements that need to be addressed, test network protocols, especially in the initial design stage. They were not designed with fog and edge 304 computing environments in mind; as such, they are outside the scope of this paper. We redirect the reader 305 to a detailed survey by [61] for further information on these simulators.

306
While there are a wide range of simulators for cloud computing, there are relatively few that can be 307 used to simulate fog and edge computing scenarios. Next, we briefly describe a selection of prominent 308 simulators used for fog and edge modelling and compare them in qualitative terms. EdgeCloudSim is relatively user-friendly providing a mechanism to obtain the configuration of devices 340 and applications from the XML files instead of defining them programmatically. IOTSim was designed to 341 simulate edge computing environments where large data volumes are sent to a big data processing system 342 by the IoT application [66]. As such, it adds a storage and the big data processing layer in to CloudSim.

343
In the storage layer, the network and storage delays are simulated for IoT applications. The big data 344 processing layer simulates MapReduce to support the batch-oriented data processing paradigm. Both 345 EdgeCloudSim and IOTsim inherit the same scalability and DES limitations as iFogSim.    In summary, despite an increase interest on fog and edge computing, research on suitable simulation 386 frameworks to support the requirements of this domain is lagging (see Table 1). Most of the existing 387 simulation tools, albeit a small number, place a greater emphasis on fog computing. They have significant 388 limitations in scalability and mobility support. All the existing simulators use DES at their core and the 389 dependence on CloudSim for three of the simulators places an additional limitation to them particularly 390 in terms of scalability. Therefore, there is an urgent need for simulation tools with greater coverage of 391 characteristics of fog and edge computing. 392 Ficco et al. [72] argue that purely simulated environments and real testbeds are not sufficiently 393 representative of real world scenarios and/or are unacceptably expensive. As such, they suggest that