Moving to the Edge-Cloud-of-Things: Recent Advances and Future Research Directions

Abstract

Cloud computing has significantly enhanced the growth of the Internet of Things (IoT) by ensuring and supporting the Quality of Service (QoS) of IoT applications. However, cloud services are still far from IoT devices. Notably, the transmission of IoT data experiences network issues, such as high latency. In this case, the cloud platforms cannot satisfy the IoT applications that require real-time response. Yet, the location of cloud services is one of the challenges encountered in the evolution of the IoT paradigm. Recently, edge cloud computing has been proposed to bring cloud services closer to the IoT end-users, becoming a promising paradigm whose pitfalls and challenges are not yet well understood. This paper aims at presenting the leading-edge computing concerning the movement of services from centralized cloud platforms to decentralized platforms, and examines the issues and challenges introduced by these highly distributed environments, to support engineers and researchers who might benefit from this transition.

1. Introduction

The Internet of Things (IoT) has pervaded our daily life by making things interconnected through the Internet smarter, distributed and more autonomous. Consequently, the development of IoT applications has gained attention from various domains [1]. For example, IoT technologies offer extraordinary opportunities to interconnect public transport systems, where sensors are integrated to enable transportation things (i.e., cars and traffic lights) to communicate with each other and build an intelligent transportation system. Furthermore, the shared information is used to obtain statistical data on traffic, relating to aspects such as vehicle monitoring, vehicle maintenance, traffic managing systems for autonomous vehicles, or vehicle parking managers. Also, the available IoT data is exploited to study the behavior of drivers, for instance, to prevent drink-driving accidents, or ensure pedestrian-safe smart crossing systems. Therefore, the integration of IoT technologies in the transportation domain has facilitated the lifestyle of drivers. Thus, the IoT paradigm has improved many aspects of our lives by providing many opportunities to develop smart solutions with more intelligent and prediction-oriented capabilities.

Several technologies are used to ensure the proper functioning of IoT devices [2], such as cloud computing that provides many benefits to IoT devices, including high-performance computing, storage infrastructure, and processing and analysis of IoT data in real-time based on the current context of IoT sensors. As a result, cloud computing is considered as a revolutionary paradigm that enables IoT devices to be robust, smart and self-configuring. Simultaneously, cloud providers take the benefits of the evolutional growth of IoT devices by building further services for IoT applications that can facilitate the communication between IoT objects that are interconnected and controlled through the cloud infrastructure. As a result, a new paradigm called Cloud-of-Things (CoT) has appeared to describe this fusion between cloud computing and IoT technologies [3].

Over time, computing is a service that has become an integral part of all major areas of human life, such as healthcare, commerce, education, agriculture, or smart cities. Before the appearance of the cloud paradigm, computing services first led to centralized servers before progressing towards distributed computing. Subsequently, computing services have been optimized by backing the centralized model with the cloud paradigm that allows users to consume unlimited services in any place and at any time based on a pay-as-you-go concept. Accordingly, cloud computing has played a major role in supporting ubiquitous IoT applications. However, there is a powerful concept that imposes restrictions on the distributed environment, which is edge computing. In fact, cloud technologies face some accessibility challenges when providing services to end-users. For example, mobile clients can move among different places, yet require cloud services with minimum cost and short response time. In this context, however, new obstacles, such as the unstable connection between cloud nodes and mobile devices that could prevent cloud providers from achieving optimal performance scores, are created and need to be faced.

To explore how well edge computing has been accepted by the scientific community, we collected 16 surveys published between 2015 and 2018 that focus on formally describing the edge paradigm and its related advantages, challenges, and issues. After reviewing the content of the papers, we selected the main observation within each survey and its related primary theme (such as security). Then, we classified the collected keywords in

Table 1. Related surveys on edge computing.

Reference	Topics	Background Described	Challenges and Opportunities Discussed
[5]	Edge technologies	Overview of concepts, applications and benefits of edge computing in different areas.	Programmability, naming, data abstraction, service management, privacy, security and optimization metrics.
[6]	Edge technologies	Description of existing solutions for distributed data stream processing and the techniques to manage resource elasticity features of clouds.	Development and deployment of stream processing applications under heterogeneous distributed architectures.
[7]	Communication networks	Overview of benefits of edge computing technologies in future 5G mobile networks.	Wireless backhauling, security, management and orchestration of services.
[8]	Mobile edge computing	Review on computation/communication components of the typical existing solutions in mobile edge computing (MEC).	Mobility management, deployment, privacy, security, and standardization of MEC systems in terms of 5G.
[9]	Mobile edge computing	Overview of concepts and applications of mobile edge computing.	Computational offloading, infrastructures, security and privacy mechanisms, resource optimization, and pricing in MEC.
[10]	Security	Overview of the edge approaches (like fog computing, mobile cloud computing and mobile edge computing) and the integration of security mechanisms in edge paradigms.	Security challenges in edge paradigms like trust management, Identity and authentication, Intrusion detection systems, protocol and network security.
[11]	Security	Overview of security and privacy issues of fog computing.	Security challenges of fogging like access control, intrusion detection systems, secure and private data computation.
[12]	Fogging	Overview of concepts and applications of fog computing.	Network issues, quality of service (QoS), accounting, billing, monitoring, computation offloading, programming model, resource management, security and privacy in fog computing.
[13]	Communication networks	Overview of concepts and network applications of fog computing.	SDN communication, incentives, standardization, heterogeneity, resource management, radio access networks, and offloading in fog computing.
[14]	Fogging	Overview of concepts and applications of fog computing and classification of fog systems according to five criteria: heterogeneity, QoS management, scalability, mobility and federation.	Classification of current architectural and algorithmic challenges in fogging according to five criteria: heterogeneity, QoS management, scalability, mobility and federation.
[15]	Edge technologies	Overview of multi-access edge architectures and orchestration deployment options.	Orchestration, programmability, QoE, resiliency, continuity, mobility, monetization, and security of MEC services.
[16]	Communication networks	Overview of 5G and fog computing technologies.	Ubiquity of devices, connectivity, network management, privacy and security challenges in fog computing.
[17]	Fogging	Overview of fog computing as an intermediate layer between IoT devices and cloud datacentres.	Context-aware resource, service provisioning, multi-tenant Support, power management, distributed application deployment, pricing, programming languages, simulation tools, sustainable and reliable services in fogging.
[18]	Big data	Overview of fog computing in terms of Big Data.	Programming models, data management, security, privacy, and trust for fog applications.
[19]	Mobile edge computing	Overview of concept mobile edge computing and computation offloading to MEC.	Computation offloading challenges in MEC: offloading decision, allocation of computing resources, mobility management and distribution, management of MEC resources, and traffic paradigm imposed by coexistence of offloaded data and conventional data.
[20]	Edge technologies	Overview of edge computing motivation and edge technologies (i.e., cloudlet).	Resource management and allocation, scalability of resource provisioning strategies, data abstraction, fault tolerance and quality of service, security and privacy in the edge computing.

. Accordingly, we found that the surveys present an exhaustive state-of-the-art overview that focuses on discussing the big research directions in resource management (75%), security paradigms (68%), algorithmic aspects (44%), and network technologies (50%) in the context of edge computing. The review also shows that the edge paradigm is still in its early stages and the scientific community needs more time to understand how to use its benefits (such as resource allocation) to offer feasible services for end-user clients [4]. Moreover, the surveys (Table 1) only focus on the advantages of edge computing without addressing emerging challenges, such as the choice of the best strategies that IoT users should follow to obtain best fitting services. Notably, there are three selection service options offered to the end-users: pure cloud services (the edge service is a middle layer between the cloud and the end user); edge services (extenders of classical cloud services); and, coordinated fog-to-cloud services (a collaborative model to facilitate matching between cloud and fog services).

In this work, we cover unaddressed problems and remaining challenges for the reviewed issues, and discuss new ways to deal with these. We conduct a comprehensive passage from the center to the edge to describe how the consumption of services has changed as a result of this new era of computing, and the emerging challenges related to the Edge-Cloud-of-Things paradigm. Furthermore, we explain how this paradigm is likely to support the progress of IoT applications. The contributions of this study can be summarized as follows:

• Present Cloud, Edge and IoT concepts in a comprehensive way for readers from different areas.
• Highlight the advancement of the Edge-Cloud-of-Things paradigm and its support to the current innovation in various domains (i.e., healthcare).
• Provide a summary of the most common technologies related to Edge Computing.
• Conduct a case study to demonstrate the examination of the impact of location based IoT services in Edge environment.
• Discuss the most recent issues and future trends that need to be addressed for the development of sustainable IoT applications and services under the Edge-Cloud-of-Things paradigm.

This paper is structured as follows. Section 2 provides a brief overview of edge technologies and the benefits of their integration with IoT paradigms. Section 3 provides an understanding of edge infrastructures with some related projects and previously established studies. Section 4 points out the effects of edge technologies on IoT applications and centralized cloud services. Section 5 conducts a case study to demonstrate one of the key challenges identified during literature review. Finally, Section 6 summarizes further challenges that we might expect to face along the advancement of edge technologies, and Section 7 concludes the paper.

2. Review of Edge Paradigms

In this section, we present an overview of current paradigms within the domain of edge computing.

2.1. Fog Computing

Edge computing covers a wide range of specializations, such as fog computing [17] and mobile edge computing [21]. Furthermore, as these technologies are closely associated with IoT applications, the satisfaction of the end-clients is achieved. For example, the fog is a decentralized computing paradigm that extends cloud services closer to IoT devices by acting as an intermediate layer between the traditional cloud and IoT end-users. However, mobile edge computing focuses more than fogging on solving the delay-sensitive and context-aware applications for the proximity of mobile subscribers by dealing with the characteristics of wireless networks (e.g., base stations), while running cloud computing capabilities at the edge of the network. That means the distance is an essential parameter in edge computing. On the other hand, in 2017, Nist [22] defined fog computing as follows: “Fog computing is a horizontal, physical or virtual resource paradigm that resides between smart end-devices and traditional cloud or data centers. This paradigm supports vertically-isolated, latency-sensitive applications by providing ubiquitous, scalable, layered, federated, and distributed computing, storage, and network connectivity”. Moreover, according to Nist [22], fog computing is not a synonym of edge computing, but it is a part of the edge.

2.2. Cloudlet

Under the branches of the edge computing, several technologies have included the concept of mobile cloud computing [23], which involves delegating or offloading mobile tasks remotely to the centralized cloud platforms. Cloudlet (also called micro-cloud) [24] is one of the decentralized architectures that have been proposed to augment the computation capabilities of IoT devices (notably, mobile devices) by reducing communication delay, saving energy, and providing software instantiated in real-time on nearby computing resources. The micro-cloud is deployed in public places, such as coffee shops, and immediately offers customized services to the local clients using virtual machine technologies. On the other hand, cloudlet could act as a middle layer, located between the centralized cloud platforms and the IoT applications. As a result, cloudlet facilitates the management of requests and returns the responses to the end-clients in real-time. MobiScud [25] is an example of a network architecture inspired by the cloudlet model to integrate cloud services into mobile networks. Despite the success of cloudlet platforms in solving communication delay, the mobility of users remains the biggest challenge faced by cloudlets. For this reason, mobile cloudlet platforms [26] have been suggested to enable clients to access services from any mobile device, such as smartphones and mounted computers on trams or metros. Thus, mobile cloudlets could further minimize the application response time and reduce the energy consumption and cost of network resource usage by moving among different sites and delivering direct services to other users within their proximity.

2.3. Ad-Hoc Clouds

Ad-hoc cloud computing is a revolutionary paradigm concerning the delivery of computing services over the network, with the benefits of efficient cloud resource utilization and improved manageability. The idea of building a sophisticated ad-hoc cloud computing environment has been discussed in [27] to facilitate distributed collaboration. Similarly, in [28], the authors proposed a simple, robust, efficient, scalable, and heterogeneous multi-tenant database architecture for the ad-hoc cloud to allow various organizations to collaborate and create a common cloud platform without harming their existence or profitability. Likewise, in [29], the advantages of the Big data paradigm (e.g., classification concept) have been exploited to predict desktop uptime for job allocation in the ad-hoc cloud. Another work in [30] studied the management of computation offloading for vehicular ad-hoc clouds and focused on implementing cloud services between vehicles, interconnecting with wireless ad-hoc networks.

It has been shown that the ad-hoc cloud strategy, which considers multiple attributes (e.g., computation capacity), outperformed the traditional cloud servers, with a higher task completion rate, higher exploitation of various IoT devices [31], and shorter completion time. To save these dynamic and distributed characteristics of the ad-hoc cloud, in [32], the authors introduced an autonomic and secure ad-hoc cloud computing system to overcome the issues regarding service management, security, and privacy of ad-hoc clouds in both static and mobile modes. Similarly, in [33], the authors proposed a node collusion method for the isolation and protection of sinkhole nodes from any false routing information, which could lure other nodes to use its service. However, the involvement of the salient features of the mobile devices (such as mobility) in ad-hoc clouds has influenced the combination formed by machine-to-machine or machine-to-cloud communications [34,35]. Consequently, the characteristics of ad-hoc clouds (e.g., heterogeneity of the devices) still imposes greater challenges in terms of resource management, routing, security, and privacy.

2.4. Mobile Edge Computing

Due to the utilization of a massive number of mobile devices in our daily life, the researchers have proposed building cloud infrastructure by integrating many mobile devices that could act simultaneously as customers and providers of services. In [36], an IoT-Based computational framework for healthcare has been proposed to monitor human activities involving physical effort. For demonstrating the usability of IoT biomedical sensors to deliver advanced applications to the end-users, the authors monitored footballers’ heart rates during a football match, highlighting the benefits of wearable devices in the mobile environment, as this kind of IoT devices supports several communication standards for processing IoT data in real-time (such as Bluetooth and WLAN). Thus, the exploitation of mobile devices further enhances the idea of connecting nearby computing resources in edge computing. Consequently, the cooperating mobile devices could satisfy the requirements of IoT applications (like low latency) better than the traditionally centralized clouds. In this context, several mobile cloud platforms have been proposed in academia to produce the most promising edge results, such as GEMCloud [37], NativeBOINC project [38], Hyrax [39], and MobiCOP [40]. However, energy is one of the challenges faced while utilizing mobile devices as an infrastructure of computing resources, because mobile devices use their energy to offer the services. Another issue is the number of participants in the constructive infrastructure, as there is a significant number of mobile devices on which large computational tasks are being completed. So, the question is how to guarantee the cooperativity and sustainability of mobile cloud devices in order to achieve an optimal computing performance.

3. Moving Cloud to Edge Computing

In this section, we focus on gathering the recent quality approaches reflecting the fast growth of the edge paradigm. Notably, we describe the benefits brought about by deploying edge technologies as extenders of cloud services, middle layers for the access to cloud services, and collaborators for cloud platforms. For that reason, we selected papers through searching academic databases and well-known publishers. Furthermore, we used specific keywords characterizing edge computing, such as fog computing, fogging, fog-to-cloud, mobile edge, cloudlet, mobile cloudlet, vehicular ad hoc network based on edge, and fog to fog. We limited the search to the up-to-date papers over the last five years, i.e., since 2013. We then synchronized the paper selection according to the commonly agreed on papers, which resulted in 39 references in total. After collecting the search results, each paper underwent a relevance check, during which its relevance to edge technologies was verified. The results are summarized in two tables. In

Table 2. An overview of edge computing implementations.

Reference	Description
[41]	mF2C is a European project that aims to design, implement and validate novel management for a coordinated fog and cloud computing systems
[42]	Google’s Go language used to implement fog architecture for IoT applications
[43]	Cloud and fog environments are used to develop application for smart healthcare environments that is being developed as part of OpSIT-Project-Germany
[44,45]	Nippon Telegraph and Telephone Corporation (NTT) developed an edge accelerated web platform to reduce the response time for cloud applications
[46]	Implementation of fog node to manage heterogeneous smart objects in complex IoT scenarios
[47]	Implementation of a wearable cognitive assistance based on Google Glass and cloudlet
[48]	Development of the CloudAware framework based on mobile edge features to design elastic mobile applications
[49]	Cisco Kinetic platform provides fog processing module to process IoT Data
[50]	Vortex platform allows to connect each vortex edge device to an IoT device to construct global shared fog domain
[51]	The Nokia Multi-access Edge Computing (MEC) platform for processing mobile network applications
[52]	Cisco IOx platform offers edge features to develop IoT applications
[53]	FogSim simulator allows the manipulation of edge features
[54]	ParaDrop is an edge computing platform for wireless gateways and applications
[55]	Mobile Fog is a programming model for large scale IoT applications
[56]	Implementation of a cloudlet mesh architecture for securing mobile clouds from network attacks
[57]	WiCloud is a mobile edge computing platform with OpenStack for enhancing location-awareness and managing inter-mobile-edge communication
[58]	HomeCloud framework integrated NFV (Network Function Virtualization) and SDN (Software Defined Networking) for orchestration and application delivery in the Edge-cloud
[59]	Implementation of the MQTT broker functionalities at Fog computing node based on SDN for orchestration of IoT devices
[60]	Implementation of cloudlet based on SDN for patient monitoring
[61]	A converged wireless access architecture based on edge servers for heterogeneous cloud access networks
[62]	UbiFlow framework for management and configuration of heterogeneous IoT networks
[63]	FSDN is a Vehicular Adhoc Network (VANET) based on fog features for supporting connectivity, mobility, scalability and flexibility of VANETs
[64]	OpenFog is a consortium that provides fogging features
[65]	F2C (Fog-to-Cloud) computing is a hierarchical layered architecture relying on a coordinated and distributed management of both fog and cloud platforms
[66]	F2F (Fog-to-Fog) architecture allows fogs to communicate with each other

, we give a global overview of edge computing implementations. A summary of the different edge computing approaches and research works is described in

Table 3. An overview of edge computing approaches and research works.

References	Date of Publication	Description	Collaborator with Cloud Architectures	Extender of Centralized Cloud Platforms	Middle Layer	Main Research Domains Discussed	Advantages of Edge Solutions	Drawbacks of Edge Solutions
[67]	2016	Fog computing architecture supporting software-defined embedded system equipped with the resources of clients to compute and store data.	No	Yes	No	Resource management	Balance the workload on a client device and computation servers. Place task images on storage servers. Balance the I/O interrupt requests among the storage servers.	Limited fog resources (like memory) cause the pervasiveness of the execution environment.
[68]	2013	Edge servers in fog computing architecture are used to improve web-site performance.	No	Yes	No	Network communication	Use context awareness available at the fog nodes to adapt with the current users’ conditions. Optimize the rendering of web-site. Save the bandwidth. Reduce latency for content delivery.	The context of the other edge technologies does not be addressed.
[43]	2015	Fogging is proposed as an intermediate role at the edge to develop a smart healthcare application that is being developed as part of OpSIT-Project -Germany.	No	No	Yes	Network communication	Provide low latency, mobility support, location awareness, and security for the end-users. Improve the quality of health services.	Real world implementation and experimental evaluation of the services are missed.
[69]	2015	Fog computing is proposed to manage the IoT resources and allow cloud platforms to act according to the current situations of clients (i.e., pricing).	No	No	Yes	Resource management	Use fog nodes to provide Efficient scheduling, allocation, pricing, security, and management of cloud resources. Predict the necessary cloud resources for processing users’ tasks.	The heterogeneity of services and the impact of mobility at the edge are not addressed.
[44,45]	2013	Nippon Telegraph and Telephone Corporation (NTT) developed an edge accelerated web platform to reduce the response time for cloud applications.	No	No	Yes	Network communication	Improve response time of cloud applications like intelligent transport control systems. Support the transmission of huge IoT data.	The catching data within the proposed platform is not discussed.
[70]	2017	Multi Fog nodes based on a Hierarchical Game Approach proposed to interact with multiple cloud provider to reduce delay and cost for the clients.	No	No	Yes	Resource management	Solve the virtualized resource allocation problem. Offer the optimal amount of virtualized resources based on users’ requests. Achieve a stable resource allocation solution.	It focused on reducing delay without considering how to ensure task completion given a certain delay constraint. It ignored the high concurrency of fog servers.
[71]	2016	Fog computing proposed to improve the QoS by using the context of multi-media IoT devices.	No	Yes	No	Resource management	Estimate the cloud resources necessary for processing requests according to the behavior and historical record of users. Minimize cloud resources underutilization. Enhance the quality of services.	It does not discuss how the other edge paradigms are investigated to improve the quality of services.
[72]	2015	Fogging based system model proposed to solve the pairing problem with the same domain of IoT nodes.	No	Yes	No	Resource management Network communication	Reduce cost by using IoT resources that share their own available resources between each other in Fog platforms. Augment the local fog capabilities by exploiting the available IoT resources. Exploit the ability of IoT devices to use various networks technologies like WiFi and Bluetooth.	It does not describe how fog servers could collaborate with the other edge paradigms like cloudlets.
[73]	2015	Fog computing based on distributed Dataflow programming mode suggested to develop IoT applications according to the perspective of the fogging.	Yes	No	No	Programming model	Offer a generic IoT development model that integrate easily with edge and cloud platforms. Allow IoT devices to exploit easily the advantages of edge paradigms such as reducing cost and time of execution.	a distributed discovery and communications infrastructure are required to facilitate the communication between IoT devices that span multiple networks.
[74]	2015	Fog Computing Based UV (ultraviolet radiation) measurement for mobile phones.	No	Yes	No	Programming model	Use personal mobile phones within the fogging to measure UV.	The cooperation of mobile phones and fog platforms are not described. How to save the energy of mobile phones is not discussed.
[75]	2014	Fog Computing based smart gateway proposed to enhance the communication between cloud and IoT devices.	No	No	Yes	Cloud service management	Provide an efficient mechanism for pre-processing the data before sending to the cloud. Create quick cloud service provisioning. Reduce delay of sensitive IoT applications.	The heterogeneity of IoT, edge, and cloud paradigms is not addressed.
[76]	2015	multiple cloudlets used in Fog computing to enhance the migration of virtual machines to cloud platforms.	No	No	Yes	Resource management Network communication	Exploit the properties of fogging to support the capability of mobile devices like providing low latency. Support virtual machine migration between mobile devices and clouds.	The other edge paradigms are not discussed.
[77]	2016	A second fog layer is proposed to enhance the communication of cloud and fog infrastructures.	Yes	Yes	Yes	Coordinated management of fog-to-cloud systems	Support the communication between clouds and IoT devices. Minimize the necessity of demanding further Cloud resources. Minimize delay inter-fog node communication. Minimize cloud access delay.	The other edge paradigms are not investigated. The heterogeneity of IoT, edge, and cloud paradigms is not addressed.
[65]	2016	A new strategy proposed to coordinate the different cloud and fog resources.	Yes	Yes	Yes	Coordinated management of fog-to-cloud systems	Emerge fog and cloud services. Provide a highly distributed and dynamic edge-cloud services such as computing and storage. Bring together heterogeneous and commodity edge devices. Deploy different smart value-added services.	The other edge paradigms, security, network traffic control, and middleware for managing the resources are not addressed.
[78]	2015	A new solution proposed to enhance the cooperation between the fog and the cloud.	Yes	Yes	Yes	Coordinated management of fog-to-cloud systems	Save communication bandwidth. Reduce transmission latency. Improve significantly the performance of cloud services.	Security, scheduling policies of resources, and selection of multiple cloud services are not addressed. The other edge paradigms are neglected.
[79]	2017	Deployment of application modules in fog-cloud infrastructure to manage the IoT resources in the network infrastructure.	Yes	Yes	Yes	Coordinated management of fog-to-cloud systems	Provide real-time IoT data processing. Reduce transmission latency. Improve the QoS for IoT devices.	Security, scheduling policies of resources, and selection of multiple cloud services are not addressed. The other edge paradigms are neglected.
[46]	2015	A fog node for managing heterogeneous smart objects in complex IoT scenarios.	No	Yes	No	Resource management Network communication	Support the interaction of the IoT application with fog nodes. Support multiple network access at the edge computing. Enhance the networks capabilities.	The impact of mobility at the edge, security, and other edge technologies are not addressed.
[42]	2016	Google’s Go language used to implement fog architecture for IoT applications.	No	Yes	No	Programming model	Provide interoperability, real-time interaction with cloud services. Reduce transmission latency. Develop suitable application for fog and IoT platforms.	Limited fog resources cannot handle large amounts of requests such as lack of memory usage. Issue with parallel computational of IoT tasks. The other edge paradigms are not investigated.
[47]	2014	Development of a wearable cognitive assistance based on Google Glass and cloudlet.	No	Yes	No	Programming model	Exploit different edge paradigms and their advantages like cloudlet. Attain the low end-to-end latencies.	Limited edge resources like limited battery capacity and limited processing capability of wearable devices. Security and multiple network technologies are not investigated.
[48]	2015	Development of the CloudAware framework based on mobile edge features to design elastic mobile applications.	No	Yes	No	Resource management	Reduce transmission latency. Speed up and perform geo-distributed IoT data. Save energy and bandwidth. Support offloading operation at the edge computing.	Selection of edge services, caching concepts, security features, and multiple network technologies are not addressed.
[55]	2013	Mobile Fog is a programming model for large scale IoT applications.	No	Yes	No	Programming model	Support low latency and scalability requirements of diverse IoT applications.	Finding enough mobile fog resources to support the distributed geospatial IoT applications.
[56]	2015	Implementation of a cloudlet mesh architecture for securing mobile clouds from network attacks.	No	No	Yes	Security Network communication	Safeguard mobile cloud services from intrusions and network attacks.	Mobile edge paradigm (like mobile cloudlet) is not discussed.
[57]	2016	WiCloud is a mobile edge computing platform with OpenStack for enhancing location-awareness and managing inter-mobile-edge communication.	No	Yes	No	Network communication	Provide cloud services at the edge of the mobile network. Save energy and bandwidth. Allow IoT application to access to real-time network information directly.	Multi-service selection, multi network access, security features are not discussed.
[58]	2016	HomeCloud framework integrated NFV (Network Function Virtualization) and SDN (Software Defined Networking) for orchestration and application delivery in the Edge cloud.	No	No	Yes	Network communication Programming model	Offer easy interaction with cloud services and IoT devices. Provide a new suitable application delivery mechanism based on the advantages of edge computing. Enhance the internal communications among cloud, edge, and IoT platforms.	Multi- interaction with cloud services and networks, security, usability, portability and flexibility in mobile environment of HomeCloud framework are not addressed.
[59]	2016	Implementation of the MQTT broker functionalities at Fog computing node based on SDN for orchestration of IoT devices.	No	Yes	No	Network communication Programming model	Perform the MQTT broker functionalities for delivering messages in a reliable manner to the end-host. Offer flexible and manageable development platform for the network designers. Enhance the basic IoT functionalities and analytics.	Scheduling policies, catching mechanism, integration of the MQTT broker functionalities with other edge paradigms are not addressed.
[60]	2016	Implementation of cloudlet based on SDN for patient monitoring.	No	No	Yes	Network communication	Enable patients to access healthcare services at competitive costs. Provide a faster and more efficient healthcare feedback to patients.	Mobility concept, advantages of the mobile edge paradigms, security and privacy information of the data patients are not addressed.
[60]	2015	A converged wireless access architecture based on edge servers for heterogeneous cloud access networks.	No	No	Yes	Network communication Programming model	Costs of networking resource will be lower for the end-clients.	Implementation of various wireless protocols and criteria used for selecting network services are not discussed.
[61]	2015	UbiFlow framework for management and configuration of heterogeneous IoT networks.	No	Yes	No	Mobility Management resource Network communication	Optimize the selection of access points in multi-networks. Satisfy IoT flow requests. Guarantee network performance for each IoT domain. Achieve scalable mobility management and robust flow scheduling in IoT multi-networks. Keep up with the changing traffic volumes provided by SDN in a flexible manner.	Network virtualization, northbound interface standardization, synchronization, capabilities of SDN in term of software are not discussed.
[63]	2015	FSDN is a Vehicular Ad-hoc Network (VANET) based on fog features for supporting connectivity, mobility, scalability and flexibility of VANETs.	no	yes	no	Management resource Network communication	Exploit resource manager and Fog orchestration to support surveillance services. Resolve the main challenges in VANETs which are: poor connectivity, less scalability, less flexibility, and less intelligence. Augment Vehicle-to-Vehicle (V2V), Vehicle-to-Infrastructure (V2I), Vehicle-to-Base Station communications and SDN centralized control. Optimize resources utility and reduce latency by exploiting the advantages of fog computing.	Investigation of SDN characteristics, development of new protocols at SDN controller, backup mechanisms, optimization of the resource utilization and service hosting, migration, and replication are not discussed.
[66]	2017	Fog-to-Fog architecture allows fogs to communicate with each other, and it reduces the overall end-to-end delay for F2C.	Yes	Yes	Yes	Coordinated management of fog-to-cloud systems	Help IoT devices to meet their hard-constraints. Support the transmission of IoT data to the fog. Enhance and upgrade the Fog-to-Cloud (F2C) communication. Increase the probability of IoT jobs to meet their delay-related requirements. Consolidate the fog nodes into one virtual fog with higher availability, better storage and processing power.	Communication and synchronization between fogs or between the cloud and the fog layer, catching mechanism, selection of services within F2C, Investigation of other edge technologies (like mobile cloudlet) are not discussed.

. Finally, in Section 3.1, we present the key observations aimed at identifying the current primary research axes in edge computing.

3.1. Summary of Key Observations

From our review, we found that the management of resources includes 70% of the selected papers. The network communication papers represent 50%, and the programming model works account for 27%. Thus, the management of services is a fairly mature topic that attracts many researchers. Moreover, edge technologies contribute to managing different services within IoT [72], cloud [75], network [46], fog-to-fog [66], and fog-to-cloud [79] platforms. However, due to the characteristics of the heterogeneous network, cloud, and IoT environments, the edge paradigms have been used, not just to manage the various services, but also to ensure the safety, availability, sustainability, reliability, and efficiency of these services. Yet, the development of edge applications is still a new research direction that faces many issues, such as the processing capability of wearable devices, lack of memory usage, portability and flexibility of edge applications, and parallel computational of IoT tasks. Furthermore, it needs to identify and explore new challenges based on the context-awareness concept for further defining the critical exploitation, development, and deployment of edge technologies for IoT, cloud, and network paradigms. Additionally, the design of new edge applications requires keeping up with the increasing sophistication and diversity of threats to security that have been triggering a critical demand for developing and deploying reliable, secure, timely and efficient edge systems in smart environments.

On the other hand, when considering future advancements in this domain, one should also be aware of the progress within network systems, notably 5G, which will target the network intelligence and orchestration solutions and move towards large-scale introduction in 2020. Due to the importance of network communication in IoT domains, many research works are conducted on edge computing to speed up the transmission of IoT data. For example, in [60], the benefits of edges are used for enabling the patients to consume the healthcare services easily at competitive costs. Additionally, they provide faster and more efficient healthcare feedback to patients. However, some lacking networks in edge computing are still not yet discussed, such as the investigation of SDN characteristics, development of new protocols at SDN controller, backup mechanisms, optimization of resource utilization and service hosting, migration, replication, selection of network services, or decision making.

Meanwhile, IoT technologies have proved their usability and efficacity by infiltrating various domains of our life. However, the correct exploitation of IoT advantages depends on technical advancements in edge and/or cloud services. Notably, based on our review, we found that the fogging paradigm is 95% constituted by the selected papers, unlike the other edge paradigms. Moreover, it is the most popular edge technology used as an extender, middle layer, and collaborator of cloud services. Thus, in the next sections, we will examine these three roles that could affect the advancement of smart IoT environments.

4. Roles of Edge Paradigms

Edge computing offers ubiquitous and reliable local services that could be incorporated into the cloud environment in different ways. Thus, in this section, we focus on examining the state-of-the-art edge paradigms by studying these different integration options, i.e., understanding edge computing as (1) an extender, (2) middle layer, and (3) collaborator of cloud services.

4.1. Edge as an Extender of Cloud Services

Extending cloud services is one of the critical concepts of edge computing, which aims at bringing centralized cloud services near the end-users and processes application requests at the local edge of networks (e.g., 5G mobile network). Consequently, the interaction between end-users and traditional cloud platforms, the execution time of IoT applications, and the cost of computing capabilities are reduced at the edge. Moreover, there are specific platforms that further support the rapid advancement of the edge paradigm and provide the requirements for designing and testing edge applications (Table 2), such as a FogSim simulator [53], Cisco Kinetic platform [49], OpenFog [64] and Google’s Go language [42]. Consequently, there are many real implementations of edge technologies, such as the development of wearable cognitive assistance spanning on Google Glass at the edge [47], where the computation-intensive tasks of the wearers are offloaded to nearby cloudlets or foglets to achieve a tight end-to-end latency constraint.

The mF2C project [41,71] is an example of edge projects (Table 2), which aim to design, implement and validate novel management edge and cloud computing systems. For the orchestration of IoT devices, the MQTT broker [59] has been pre-installed at Fog nodes based on SDN (Software-Defined Networking) for the improved delivery performance of IoT applications. Likewise, in [42], the fog nodes have been exploited to manage the heterogeneous smart objects directly in complex IoT scenarios. In [68], fogging has been used to automate web site performance optimization at the edge servers. This novel idea illustrates the importance of edge location by providing dynamic and customizable optimization based on local network and client device conditions. Further, the catching technique has been implemented within fog servers to speed up the content delivery process to end-users. Similarly, the solution proposed in [67] has considered fog servers as a main functional provider of computing and storage services. Another solution for reducing the waste of IoT resources has been discussed in [71], where a Fog model based on previous QoE records has been proposed to improve the QoS of IoT devices. Furthermore, historical records have been exploited to manage the IoT resources in the fog. Similarly, WiCloud [57] has been developed as a mobile edge computing platform with OpenStack for enhancing location-awareness and managing inter-mobile-edge communication and data acquisition for an innovative service.

In [72], a balanced and matching strategy has been proposed for achieving the user requirements. The proposed algorithm computes IoT resource sharing among the fog nodes inside the same fog domain, where each domain is selected according to the utility metric for a couple of nodes, which determine an ordered list of preferred pairing nodes for each node by considering the communication cost and pricing benefits. Then, each node in the fog domain sends requests to its preferred pairing node. In this work, the quality of the service and the mobility of IoT devices or the fog nodes are, however, not discussed, since the proposed solution operates inside a single fog domain. Considering the advantages of fog computing, the authors in [74] have discussed the possibility of using smartphones for ultra-violet radiation (UV) measurement. Additionally, the results have been compared with a regular digital UV meter to validate the usage of smart mobiles. In [55], the authors have proposed a cloud-aware framework based on mobile edge features for designing elastic mobile applications. This framework is a PaaS programming model that aims at providing a high-level programming model for large-scale IoT applications, using the Fog computing approach to reduce latency and network traffic. Consequently, it offers a simplified programming abstraction and supports IoT applications, dynamically scaling at runtime. However, the drawback of this framework is the involvement of different mobile devices, such as smartphones, which means that it is crucial to determine where to find a better placement based on dynamic constraints, such as available resources and migration cost for mobile fog processes as well as for the other edge technologies (e.g., foglet).

Summary: Despite the ability of edge computing to bring about cloud services and facilitate the analysis of the data closer to their IoT sources, there are various drawbacks that are still implied by the advancement of the edge paradigm as an extender of cloud services, such as the deployment and management of local resources [54,58,63], assurance of the availability and sustainability of edge services [48,50], use of Big Data analytics based on the edge for IoT data-driven service management automation [68], development of suitable programing models for edge applications [38,42] and coordination between different edge paradigms [37,65]. As a result, the investigation into how to build a stable and hybrid edge infrastructure for receiving and processing information locally is urgently needed.

4.2. Edge as a Middle Layer

Despite the investigation of the advantages of edge technologies for adapting and obtaining the most efficient results according to users’ requirements in real-time, the running of the requested services with isolated and limited edge resources is considered a big challenge associated with the advancement of edge technologies (Table 2 and Table 3). For that reason, several works exploit edge technologies as a middle layer, allowing users to use unlimited cloud resources (Table 3). For example, in [43], Fogging has been suggested as an intermediate layer to provide low latency, mobility support and location awareness for smart healthcare and elderly-care applications, which are developed as part of OpSIT-Project-Germany. An interesting work is cited in [60], where an efficient cloudlet architecture within SDN is proposed for supporting the integration of the observation of the physiological conditions of the patient under supervision. As a result, the proposed edge architecture has increased the accessibility of patient monitoring applications and further enhanced the quality of healthcare services.

In [44,45], the Nippon Telegraph and Telephone Corporation (NTT) developed an edge-accelerated web platform to reduce the response time in cloud applications. Similarly, multi-fog nodes based on a hierarchical game approach [70], multiple cloudlets [76], and a fog-based smart gateway [75] have been proposed to facilitate the interaction of IoT devices with various cloud platforms.

In [58], the HomeCloud framework has focused on solving two research issues regarding automated orchestration and dynamic offloading. Additionally, it has defined a new application delivery in edge and centralized cloud services by exploiting the benefits of NFV (Network Function Virtualization) and SDN (Software Defined Networking) technologies. In [69], fogging has been proposed as an automated service negotiator that aims at reaching a satisfactory agreement between the customers and providers of services by predicting the resources and pricing before starting the processing of IoT tasks on the cloud platforms. Accordingly, the purpose of using fog, which resides between the underlying IoTs and the cloud, is to manage resources, process IoT data, and ensure security measures for IoT clients. However, the authors do not discuss how the mobility and heterogeneous services influence this dynamic resource estimation and pricing model. In contrast, in [56], the cloudlet-mesh architecture provides mobility management and security mechanisms to serve all mobile devices connected to it.

Summary: Despite the success of edge technologies as a middle layer ensuring the delivery of requested cloud services to the IoT applications, while avoiding the communication bottlenecks of networks [45,46,47], the heterogeneity of IoT and cloud environments brings a big challenge that emerges with the middle layer and requires the integration of interactive multiagent system applications for managing the communication between these heterogeneous paradigms [52,60]. Additionally, there is a significant need for coordinating and optimizing the technological resources in both edge and cloud platforms to build an infrastructure for flexible, scalable, and efficient combined edge-cloud services.

4.3. Coordinated Fog-to-Cloud Resources

Fog-to-cloud (F2C) is another new concept at the edge that has been merged in a collaborative model to facilitate an optimal match between cloud and fog resources (Table 3). In [73], the distributed data flow model has been proposed to allow the distributed execution of applications on different IoT devices in order to utilize virtual computing infrastructures across fog and cloud platforms. Notably, the authors have highlighted that IoT applications should be able to run in both fog nodes and the cloud, not just construct a peer-to-peer collaboration to process the output of one node that appears as the input of another node. Similarly, in [65], the advantages of merging cloud and fog resources have been discussed (e.g., the reduction of the execution time of applications). Additionally, the authors have proposed a coordinated and hierarchical architecture for the management of F2C resources. Furthermore, in [80], the benefits of F2C have been illustrated by describing how the different resources found on centralized and distributed platforms could address the management of the Worldsensing group [81].

In [78], a new workload allocation solution was proposed to highlight how the cooperation of F2C could enhance the performance of cloud platforms regarding reducing communication latency. Similarly, in [77], a second fog layer has been proposed to improve the service allocation and communication of combined fog-cloud infrastructures. The mF2C [41] is another example of innovative edge projects that aim at building a global stack of cloud-fog resources to optimize the execution of services in real-time. The authors in [66] have introduced a Fog-to-Fog communication algorithm to reduce the overall end-to-end delay for F2C by allowing fogs to communicate with each other. Furthermore, the authors have highlighted how the edge technologies could play two roles at the same time, which are: middle layers and collaborators for cloud computing.

In [79], a resource-aware placement algorithm of IoT applications has been deployed in the fog-cloud computing paradigm to manage the IoT resources. Furthermore, it focuses on meeting the network challenges related to the evolving fog-cloud architecture, such as latency, network usage, and energy consumption. However, only static network topologies have been presented. Yet, in some cases, such as surveillance systems, the IoT devices could be placed in a congested area that requires the exploitation of dynamic network characteristics (e.g., Network Function Virtualization), which would result in a lower latency.

Summary: As one can see, the existing works focus only on the combination of fog and cloud platforms in term of the communication network (e.g., low latency) and management of F2C resources. Additionally, they neglect the other requirements of cognitive IoT applications that should be addressed in F2C platforms, such as how to extract the most relevant information from video surveillance in the transportation area in order to control traffics; how to support decisions and actions for disaster applications in the F2C paradigm; or how to use the benefits of F2Cs to build sustainable smart city infrastructure for responding to the preferences and requirements of the end-users. Yet, many challenges still remain associated with fully addressing the F2C model, which is not just an advanced extension of the shared fogging-cloud resource, but also a new way to distribute, secure, allocate, evaluate, analyze, orchestrate, and manage resources across networks, rather than just build modern collaborative computing at the network edge.

5. Case Study

As discussed earlier, edge computing resolves the deficiencies of IoT applications by bringing the cloud resources closer to IoT sources, managing IoT resources, and avoiding network issues. To do that, the edge providers focus on providing location-based services to respond to dramatically changing local IoT conditions. To better understand the benefits of nearby edge services, we have conducted a case study using the iFogSim simulator [82], which provides the necessary fog and IoT functionalities and can be used to measure the impact of resource management techniques on different aspects, such as network utilization and latency.

5.1. Description of the Case Study

We have chosen distributed pan–tilt–zoom (PTZ) camera systems [83] as a use case, since these facilities are considered as one of the major human inventions that can monitor different events surrounding us in real-time. Moreover, these connected IoT devices generate significant data traffic, which is transmitted to edge or cloud platforms for analyzing intensive data, identifying anomalies, and then acting according to timely decisions. Our aim throughout this study is to examine the impact of the location of cameras surveilling a given area and consuming geo-services. Moreover, it aims at better understanding the Edge-Cloud-of-Things and its implications, mainly its network and energy characteristics. To do that, we consider that the number of surveilled areas can vary from 1 to 5, where each area contains three smart cameras that are able to access fog and cloud servers. Based on iFogSim tools [82], we have made five cases. The type of computing resources used in this simulation and the expected latency between them are shown in

Table 4. Description of network links.

Source	Destination	Latency (ms)
PTZ_CTRL (pan–tilt–zoom _control)	camera	1
VIDEO	Camera	6
GW (WiFi gateway)	ISP GW	2
ISP GW (Internet Service Provider)	cloud	200
Camera	GW	2

and

Table 5. Type of computing resources.

Type	Downlink Bandwidth (Mbps)	Uplink Bandwidth (Mbps)	RAM (MB)	Processing Capability (Mips)	Rate per Mips
Cloud	10,000	100	40,000	20,000	0.001
ISP GW	10,000	10,000	4000	2000	0
GW	10,000	10,000	4000	2000	0
camera	270	10,000	1000	1400	0

, respectively. The experimental setup of the first case, with one area and three smart cameras, is shown in Figure 1.

5.2. Discussion of Results

The findings, after performing the simulation, are shown in Figure 2, Figure 3 and Figure 4. These demonstrate that, when the cloud resources are closer to the users, the users experience fewer network issues. Moreover, the location of IoT devices creates some problems for cloud providers. For instance, multiple access to remote cloud servers causes an increase in energy consumption, compared to fog platforms (Figure 2 and Figure 4). In this context, the location of IoT sensors has become a significant criterion, which the providers of services should take into account for providing responses in real-time and enlarging the IoT system lifetime. Therefore, the advantages of edge technologies contribute to saving the idea of bringing local services closer to the customers and building sustainable smart IoT infrastructures (Figure 2, Figure 3 and Figure 4). However, the traditional clouds and IoT paradigm remain two complementary and associated technologies, where the role of cloud providers is to host IoT data and make them available to customers by using automated, scalable, flexible, and unlimited services presented as X-as-a-Service based on the pay-as-you-go concept (such as Storage-as-a-Service, as in our case study).

Yet, from a customer point of view, what could be the factors that aid choosing a suitable service? Is it a local service with a short latency or a remote service with robust functionalities? This dilemma has already been studied in relation to the cloud environment in order to determine the optimal cloud service selection, such as the work in [84], which aims to study the impact of the quality of service on cloud-based industrial IoT applications. However, with the integration of cloud with the edge paradigm, it is crucial to define the criteria that could aid in determining the best service, since both of these paradigms have different characteristics (such as latency). Maybe, the IoT paradigm could solve this challenge, since it has the ability to gather nearby users’ information about their location, preferred services, and feedback after consuming services. Thus, these collected data could be used, not only to return results to the original end-users, but also to learn from the previous experiences and predict the future actions of all IoT domains. Consequently, IoT technologies drive new business activities based on their contextual location-services. For example, the Internet of vehicle-based cloud services [85] has created great opportunities for managing data collected from vehicles and their surroundings. Additionally, the shared-transportation data could be used for developing and optimizing the transportation applications designed for surveillance, control, situation recognition, emergency management, or prediction. As a result, the fusion of these three powerful technologies has emerged as a revolutionary paradigm that enables the interaction and cooperation of any kind of physical or virtual objects with edge and cloud services through the Internet. This latter is actually the fourth part of this fusion that we could not neglect.

Indeed, network communication plays two roles, which are: Benefitting the client by offering the services of IoT, cloud and edge paradigms, and at the same time, acting as a bridge for the communication between the client and service. The fifth generation (5G) is the latest generation of mobile and wireless networks that describes well those roles in terms of virtualization and softwarization. For example, SDN (Software-Defined Networking) and NFV (Network Function Virtualization) are two technologies that revolutionize 5G technologies by offering end-to-end real-time network services, high performance, resource allocation management, low latency, reliability, and flexibility in ubiquitously providing different kinds of network services.

On the other hand, the power of making the 5G automated could be measured with the ability and effectivity of IoT devices to collect data and process them to extract information in real-time. Thus, as the 5G technologies are agile and reliable as the quality of different services offered is guaranteed, such as in wearable devices that monitor the daily activities of people with Alzheimer’s disease. As a result, the observed drawbacks of cloud service provisioning for IoT applications, such as response time and unstable connections, are being resolved by this new high-performance communication network. Similarly, the communication between edges and central clouds is further optimized.

Nevertheless, the automation of 5G networks cannot be achieved only through the direct application of smart IoT, edge and cloud services, but also by using predictive analytics to learn from big tracking data in order to make the network services efficient and reactive. For example, smart city technologies connect various kinds of public IoT utilities (e.g., cameras) for responding to daily public services. Simultaneously, they can produce the most accurate and timely information (e.g., social network impression of citizens concerning public services), which could be used to extract knowledge for proposing new intelligence methodologies on edge, cloud and network services (such as middleware for edge-cloud selection services). In this context, deep learning techniques for processing data in edge-cloud systems are required to meet different clients’ requirements. Furthermore, they would help to effectively and efficiently study how and when edge or cloud services should be selected as candidates for properly processing and adopting IoT data.

6. Challenges and Research Directions

As with any emerging technology, the promising advancements are also accompanied with numerous challenges that are not yet well summarized in the existing literature. For that reason, we have used our state-of-the-art review also to identify the general drawbacks associated with the edge computing paradigm and predict the future challenges.

Based on the review, we found that the majority of works have focused on transporting data from nearby IoT source devices over a network, storing it on edge nodes, performing analyses, and then returning results to IoT clients. In other words, the existing works have focused on using the advantages of edge computing, as the best solution for meeting the IoT computing needs, such as using fog nodes for reducing response times in real-time IoT applications, balancing network traffic, reducing the transmission latency between edge/cloud servers, extending the lifetime of IoT devices, and avoiding the traffic peaks in IoT networks. However, this adapted strategy for integrating the benefits of edge computing to assist IoT devices does not really meet the natural interaction of edge technologies with different heterogeneous organizations (e.g., 5G networks). Thus, the investigation of adequate strategies for determining an efficient communication and synchronization between different IoT, cloud and edge elements, such as the selection of cloud/edge services, scheduling policies, catching mechanisms, security of shared information, interactivity, cognition, self-configurability, dynamicity, usability, flexibility, interoperability, and adaptability of edge applications with the other different technologies, is required.

In the meantime, the knowledge produced by IoT devices is not addressed well in the existing works, such as that in [18]. Yet, the critical research question in this respect is how to use this knowledge, which could further influence the adjustment of edge applications, notably, the feedback of the end-clients that expresses clearly their real needs and satisfaction after using an application. Indeed, the point of user views can be gathered easily from social networks (e.g., Facebook) that are considered the best source of application reviews for getting the real preferences of end-clients. Thus, it is necessary to use these opinions for optimizing services, because they could suddenly and deeply change the behaviors of IoT, edge and cloud applications.

Therefore, in this section, we focus on identifying the strategies that could help the providers of cloud-edge services to know in advance where, how, and when edge services could be applicable or not suited to IoT technologies. Thus, we classify the challenges accompanying the advancement of the Edge-CoT paradigm on four main axes: The first concerns middleware platforms that provide many characteristics, such as enhancing the selection of services and communication across heterogenous environments; the second concerns edge analytics for treating collected IoT data; the third is concerning social networks, as pure and valuable information provided by the end-users; and the last axis concerns security, as a fundamental challenge in the Edge-CoT environment.

6.1. Interoperable Middleware Platforms for Edge Computing

The adoption of IoT and cloud technologies in different sectors would ensure the reliability, interoperability, sustainability, and scalability of automation systems [86]. For that reason, many research works have proposed interoperable middleware as a solution for enhancing the communication between remote cloud services and IoT applications. Moreover, they have focused on designing standard IoT-cloud middleware that could create a unified environment, allowing several IoT devices to communicate and interact with cloud platforms [87]. Additionally, the middleware could enhance the functionality of IoT applications by providing adapted cloud services and hiding all the details of heterogeneity in the IoT paradigm. Therefore, the IoT-cloud middleware plays an essential role in ensuring the integrity of different IoT devices, providing the abstraction and adaptation necessary for applications in diverse domains [88]. For example, in [89], the authors have integrated an agent-oriented middleware, called ACOSO (Agent-based COoperating Smart Objects), and a sensor-cloud infrastructure, named BodyCloud, for the collection and analysis of sensor streams. The proposed solution aims to develop decentralized smart objects within IoT and take the benefits of cloud computing by managing data processing and storage needs with lower costs. Another interesting work has highlighted the convergence of cloud services and the Internet of Things [90] by designing a framework to make IoT devices rely on a cloud platform for data processing and storage. Since the generated data from device networks of IoT are considered useful information for gaining knowledge, the authors have focused on optimizing the data transportation from geo-distributed devices and cloud platforms. As a result, this solution acts as a bridge that guarantees the reliable and efficient Big Data transfer from distributed IoT-devices to the cloud. Similarly, in [91], the authors have designed and implemented APIs for the IoT cloud to minimize the deployment time of applications, optimize services, and provide the effective utilization of resources. To cope with the growth and evolution of IoT systems, the research in [92] adapts multi-agent systems and machine learning techniques to develop a framework for self-adaptive and self-organizing IoT applications.

Meanwhile, the need for more IoT applications with a better processing speed has become inevitable for smart environments, such as healthcare applications [93]. The integration of edge computing with IoT-cloud architecture has perhaps offered high-speed networks for transferring vast amounts of data from the IoT sensors to local or remote servers. However, the ability of IoT sensors to join and leave the network in real-time could change the communication of IoT devices with clouds, fogs, or fog-to-cloud platforms. Notably, the 5G network is composed of various objects based on different infrastructures, such as mobile cellular networks, MANET (Mobile Ad Hoc Networks), WSN (Wireless Sensor Networks), and VENET (Vehicular Networks). Consequently, the huge volume of IoT data is held by the heterogeneous networks. Yet, this is a strong reason for shifting towards a decentralized middleware model. In this case, the middleware could suitably manage and refine IoT data in the heterogeneous networks. Furthermore, the characteristics of middleware could help IoT devices to interact with centralized and decentralized platforms easily, rate the proposed services, and then select which one of them could offer the best results on a smaller scale in real-time.

Until now, several related challenges have not yet been addressed, for example, how can a centralized middleware guarantee the integrity, usability, interoperability, adaptability, and availability of selected services for the IoT end-clients? What are the most critical factors necessary for developing anticipatory middleware that users request? How can the middleware use the caching of results on the heterogeneous networks for processing contextual IoT data as well as optimizing the selection of services in the centralized and decentralized environments?

6.2. Edge Analytics for Big Data

IoT technologies have improved the quality of citizens’ lives by providing the most efficient services to a broad set of devices in many domains. The smart city is one of the projects of the IoT paradigm that has used sensors to manage services, including hospitals, public lights, and water supply networks. Consequently, a significant amount of IoT data (including mobile and immobile data) is being generated. To meet the requirements of IoT data, cloud resources are used to involve the most sophisticated activities of acquisition, storage, and processing. Until now, the collected data from various domains have become the Big Data that have created new critical support for the progress of IoT domains. Indeed, Big data provide flexible, useful and reliable knowledge for improving operations and strategies in various areas, such as automated and intelligent methods for the automotive industry [94,95]. To do that, the IoT applications use cloud platforms to analyze data remotely and then extract the most relevant information that could be used to take more informed actions. However, data transmission is experiencing some problems that need to be solved, such as the interruption of the network and the mobility of users [96]. Yet, there is a great need of edge technologies to enhance this big innovation of Big Data and support the transmission of IoT data to centralized clouds and vice versa [97].

New benefits and opportunities of edge technologies have emerged to handle the valuable Big Data. One of the benefits of edge computing for Big Data and IoT applications has been cited in [98], where an autonomic edge computing platform has been described to support deep learning for the localization of epileptogenicity using multimodal rs-fMRI (resting state-functional magnetic resonance imaging) and EEG (electroencephalography) Big Data. Additionally, the proposed solution could monitor, evaluate, and measure the epileptic brain. As a result, it could help doctors and patients to detect seizures and predict future epilepsy in real-time. Thus, edge computing provides competent real-time mechanisms for the meaningful investigation of Big data, including data collection, data processing, extraction of significant data, and learning from Big Data.

Despite this fusion of edge, Big Data, and IoT paradigms, there is a great need to consider the collaboration of edge computing with cloud computing, not just for optimizing services (such as storage [99]) or communication networks (e.g., low latency), but also for learning from the previous experiences in the fusion of Big Data and cloud platforms (e.g., the deployment of artificial intelligence strategies in cloud-aware systems [100,101,102]). Moreover, this fusion could aid in the understanding of how to use the local edge resources as powerful complement collaborators for clouds and the Big data paradigm in terms of services (e.g., computing and storage) and communication facilities (e.g., SDN). Consequently, several new challenges emerging from the fusion of edge computing, cloud computing, and the Big Data paradigm should be addressed.

6.3. Social Networks into Edge Computing

Recent advances in social networks, such as Facebook, have significant impacts on the development of services in various areas, including education, smart city, or commerce. In fact, social networks are expected to more precisely provide personal information about the customers, such as names, places to travel, games, diseases, type of jobs, or most popular TV shows. That means social networks allow the providers of services to know in advance how to personalize their services according to human behavior. On the other side, the increase in the number of mobile devices further supports the development of several mobile social networking applications [23]. These latter have been considered as an advanced social network version for providing a significant source of information [103], understanding the real needs of clients, and optimally improving the infrastructure of smart environments, such as the use of clients’ locations as a parameter for recommending the best cloud services [104]. Therefore, the virtual social information expresses the real-world of human social behavior, which should be strongly explored in depth. Further, the heterogeneity of social network data provides great opportunities for research and industrial applications for creating a new critical point of convergence for the modern information infrastructure of knowledge retrieval and discovery. Thus, social networks have been emerging as a priority for research and development across the most recent applications in edge and cloud environments.

Nevertheless, the share of part or full personal information [105,106], such as credit card numbers, could be stolen and used in illegal activities. That leads us to ask how edge computing could protect sensitive information, which imposes serious considerations related to the data security of citizens? Where should the collected personal data be processed in a distributed or centralized environment? Perhaps the blockchain technology provides the necessary techniques for addressing the performance and security issues in the cloud [107,108]. However, it is not yet understood what the best practices and strategies are that the providers of services could follow to manage the implicit and explicit personal data that are very sensitive, and any unauthorized access to them can have serious consequences. Accordingly, the exploitation of the most relevant personal social information leads to solving these relevant research challenges.

6.4. Security in Edge Computing

The rapid development of IoT techniques has paved the way for new and improved services in various popular domains. The smart healthcare domain is one of the examples of a field that integrates IoT technologies to make patients’ lives easier by adopting wearable devices in remote care and digital health programs in a modern model of hospital-centric care [109]. Similarly, IoT could be used and blended with cyber-physical systems of the smart grid to be more robust and resilient [110]. Likewise, IoT could keep up with the ubiquitous use of social networks in our daily lives by protecting the location privacy of the end-users and minimizing potential hacking, thefts of personal information and unauthorized access [111].

At the same time, the IoT paradigm is rapidly merging with our quotidian activities, as nearby technologies and physical devices, such as vehicles, sensors, actuators, and any other embedded devices, will be connected and communicate with clouds or edge nodes, share information, and produce IoT data on a massive scale. By 2025, the complexity of IoT systems is expected to exceed 100 billion connected devices [112]. Consequently, this big innovation will introduce new challenges for the security of IoT systems and cloud-edge applications.

For controlling the explosive growth of the amount of heterogeneous IoT data [113], the providers of services must gain expertise, not only in the development of edge and cloud applications for the IoT paradigm, but also in finding secure techniques to face the threat of the cyber-attacks. The edge services could fill the gap between security and practical usage in IoT, since they can gather and treat the most relevant indoor and outdoor information (such as reviews, directions, calling, and using IoT applications) in a short time, compared to remote services [114]. However, the edge computing is still in its early stages. Therefore, there is a significant need to define security strategy guidelines in the edge environment, such as security for architecture and protocol design for IoT services in 5G networks, security policies in fog-cloud cooperation, privacy of sharing IoT data, applying machine learning techniques in edge computing for detecting the anomaly in network traffics and cybercrime in IoT devices, and employing cryptography in edge computing. Thus, there is a big demand for the development of new secure edge methods for ensuring the accessibility, resilience, and security of IoT infrastructures.

7. Conclusions

The Internet of things paradigm generates a significant heterogeneous amount of data daily, which could be handled by centralized or decentralized platforms. In this paper, we have outlined recent advancements as well as challenges associated with edge computing. The motivation for moving to edge computing is discussed regarding Edge-CoT computing architecture-related issues. Furthermore, we have highlighted the importance of this collaborative model by studying the distributed PTZ camera systems as smart IoT devices demanding a high interaction with remote and local geo-services, which we presented as a case study in the paper. Finally, based on the literature, we have suggested multiple promising research directions that could be pursued in the future.