Fog Computing for Realizing Smart Neighborhoods in Smart Grids

Abstract

Cloud Computing provides on-demand computing services like software, networking, storage, analytics, and intelligence over the Internet (“the cloud”). But it is facing challenges because of the explosion of the Internet of Things (IoT) devices and the volume, variety, veracity and velocity of the data generated by these devices. There is a need for ultra-low latency, reliable service along with security and privacy. Fog Computing is a promising solution to overcome these challenges. The originality, scope and novelty of this paper is the definition and formulation of the problem of smart neighborhoods in context of smart grids. This is achieved through an extensive literature study, firstly on Fog Computing and its foundation technologies, its applications and the literature review of Fog Computing research in various application domains. Thereafter, we introduce smart grid and community MicroGrid concepts and, their challenges to give the in depth background of the problem and hence, formalize the problem. The smart grid, which ensures reliable, secure, and cost-effective power supply to the smart neighborhoods, effectively needs Fog Computing architecture to achieve its purpose. This paper also identifies, without rigorous analysis, potential solutions to address the problem of smart neighborhoods. The challenges in the integration of Fog Computing and smart grids are also discussed.

1. Introduction

According to an estimate by Cisco, there will be 75 billion connected devices by 2025 [1,2]. This exponential growth in the number of devices per person is due to the proliferation of mobile devices (e.g., wireless sensors, mobile phones tablets, etc.) as more and more consumer services are provided through wireless sensor networks. These numbers will soon be superseded by sensing/acting devices placed virtually everywhere (the so-called Internet of Things and pervasive sensor networks). And therefore, it is estimated that around 90 percent of the data generated by the sensor networks will be stored and processed locally rather than in the cloud [3]. Smart Grid domain sensors are no exception. According to an estimation, 1 million smart meters installed in the Smart Grid community would generate around 2920 Tb data in quantity with a sampling rate of 4 per hour [4].

In North America, the number of smart meters has grown from 6 % of households in 2008 to 89 % in 2012. These numbers continue to grow as more communities and businesses are installing smart meters. Austin Energy in Texas has implemented 50,000 smart meters with a sampling rate of 5 min to produce around 800 TB of data.

In the smart grid system, large amounts of sensors are deployed across a wide area of generation lines, transmission lines, distribution lines, and consumer buildings to implement complex monitoring and control functions. The massive volume of real-time data collected by smart meters will help the grid operators gain a better understanding of the large-scale and highly dynamic power system [5]. As the number of smart meters increases from thousands to millions, the current state-of-the-art centralized data processing architecture will no longer be sustainable with such a big data explosion.

A network of smart meters, also called Advanced metering infrastructure (AMI) provides intelligent communication between consumers and power service providers based on the information exchange between the two sides. AMI provides immediate feedbacks on predicting peak power usage and hence, enables the service provider to rapidly address the grid demands. It has analytical tools that enable the interaction with operation and management systems, for example, power outage management system (OMS), geographic information system (GIS), consumer information system (CIS) that manages the electricity billing and customer information, and distribution management system (DMS) that provides power quality management and load forecasting based on smart meter data [6]. It is crucial to act in real-time on smart meters data to prevent cascading outage [7] and recover from a power outage in the shortest time.

In addition, sending the huge smart meter sensor data to the centralized cloud servers will require prohibitively high network bandwidth for the transmission and, high round trip time. If we have a distributed architecture, the communication cost between the service providers and AMI infrastructures will be minimized since they are physically located close to each other. The communication bandwidth needed is mainly for the information exchange between each distributed node and the central server, which will not vary much because it will not be affected by the increasing number of smart meters or the sampling frequency of smart meter data. Data security and privacy are also critical issues when the sensitive smart meter data is aggregated in the centralized cloud server [8] because the high-frequency metering data which is required for efficient smart grid operations reveals consumers behavioral patterns.

The Electric Vehicle (EV) and Vehicle to Grid (V2G) technology are an important component of smart grid [9]. They provide the grid, support by delivering services such as Demand Response Management (DRM), spinning reserve, voltage and frequency regulations whenever needed. To the service provider, the EVs are a potential independent distributed power backup source. But on the challenging side, EVs are dynamic storage sources which are difficult to schedule due to their unpredictable usage by the consumers and, their numbers are increasing continuously. EVs store the energy produced by local renewable energy sources (RES). According to an estimation, China has set a goal to install 150–180 GW of wind power and 20 GW of PV solar power by 2020. The integration of large renewable energy sources like wind and photovoltaic (PV) solar energies into the power system will continue to grow for local and global energy requirements. Importantly, the integration of RES in Smart grid makes it possible to meet MicroGrid operations and energy security issues [9]. On the other hand, these RES are intermittent in nature due to their dependency on weather, and therefore, the forecast is quite unpredictable. Therefore, for effective V2G operations, reliable MicroGrid operations and real-time RES energy forecasting, we need a computing paradigm near these sources.

As discussed above, the main challenge is to build a scalable and secure distributed architecture for real-time data processing for reliable smart grid operations. We need a mechanism to store and process data near to where it is produced. An architecture is proposed recently as Fog Computing architecture which is promising for distributed real-time, scalable and securing & privacy preserving IoT applications. The term “Fog” was first introduced by Cisco [10]. Fog is like a cloud near the ground, so this term aptly described cloud services near sensors networks.

We discuss in detail the challenges of smart grids and how Fog Computing is a promising solution in Section 6. To summarize, Fog Computing is crucial in Smart Grid domain to ensure:

• Real-time control and monitoring of smart grid operations.
• Scalable and distributed smart grid architecture.
• Security and privacy-preserving smart grid operations.
• Network bandwidth and traffic management.
• Reliable smart grid operations and power outage prevention.
• Effective distributed power generation and RES integration.
• Optimized smart grid operations and minimized losses.
• Effective MicroGrid operations and V2G integration.

Fog Computing, which has applications in several application domains is defined as a highly virtualized technology that provides computation, storage, and networking services between IoT devices and traditional Cloud servers, along with the IoT devices to the Cloud continuum. Fog Computing, as defined by Atlam et al. [11] is required because of:

• The need for the geographical distribution of resources rather than a centralized one as in the Cloud;
• The need to incorporate large networks of sensors communicating, usually by wireless access, Zigbee, Bluetooth and related communication technologies;
• The need to support real-time communication with IoT sensor networks and mobile devices;
• The need to support heterogeneous devices and interoperability with different service providers;
• The need for on-line analytic and interplay with the Cloud.

Together with the support of Cloud Computing, applications require both fog localization and Cloud centralization. Fog Computing plays a vital role in at least the following fields: smart grids, smart healthcare, smart manufacturing, smart connected vehicles and smart agriculture. Fog Computing provides proximity and location awareness, geo-distribution, and hierarchical organization, which makes it a suitable platform for resource constraint wireless sensor and actuator networks.

Figure 1 shows Fog Computing application domains which are smart grids, smart cities, smart healthcare, smart manufacturing, smart connected vehicles, smart agriculture and, the high-level architecture of Fog Computing. In

Table 1. Comparison between Cloud Computing and Fog Computing.

Attributes	Cloud Computing	Fog Computing
Definition	On demand compute, storage and networking services on centralized Cloud servers located far from IoT devices and users accessed via core network	Compute, storage, and networking services in continuum between IoT sensor networks and Cloud servers
Applications	Applications which are not latency sensitive like software services, platform services, e-commerce applications, web applications, and so forth	IoT applications which are latency-sensitive like medical services, smart healthcare, smart grid, smart connected vehicles, augmented reality, and so forth.
Architecture	Centralized	Decentralized and hierarchical
Programming	Distributed programming	Distributed and parallel programming
Storage capacity	High storage data servers	Data hubs near IoT sensor networks
Reliability	Less reliable in terms of connectivity, performance and security	More reliable in terms of connectivity, performance, and security
Security & Privacy	Low	High. Provided on Fog networks through Blockchain and other technologies and along cloud-to-things continuum
Communication mode	IP networks	WLAN, WiFi, LAN, WAN, ZigBee, wired communication, cellular networks
Computing capacity	High computing capacity	Moderate computing capacity
Mobility support	Low	High
Geographical distribution	Global, centralized	Less global, geographically distributed
Application latency	High	Low

, we compare Cloud and Fog Computing.

The key contributions of the paper are the following:

• We present an extensive literature study of Fog Computing and its fundamentals along with discussing various technologies which led to the emergence and foundation of Fog Computing and, how Fog Computing is different and better than the previous technologies. We present various application domains where Fog Computing finds application like smart grids, smart healthcare, smart manufacturing, smart connected vehicles along with describing the use-cases.
• We describe smart grid principles and their challenges. We define and formalize the problem of smart neighborhoods and show the applicability of Fog Computing to solve these challenges. We propose a potential Fog Computing architecture to address these challenges.
• We propose, based on the literature study, the potential solution techniques which should be used to realize smart neighborhoods. We discuss with the help of a use case scenario of using artificial intelligence-based predictive techniques towards addressing the challenges of smart neighborhoods.
• We discuss the challenges in Fog Computing and smart grid integration.

It is important to study Fog Computing technology fundamental principles and what it offers to define smart neighborhoods problem in smart grids. The rest of the paper is organized as follows. In Section 2, we introduce the related works on Fog Computing and the categories of research work in various areas in Fog Computing. Section 3 describes the related technologies prior-to Fog Computing that helped to lay the foundation for Fog Computing. In Section 4, we describe the applications of Fog Computing in various domains along with some use cases. Section 5 introduces the smart grids, smart neighborhoods concepts. In Section 6, we describe challenges in smart grid applications and how Fog Computing is a promising solution. Thereafter, we formalize the problem of smart neighborhoods. To solve these challenges, we propose a potential Fog Computing based solution architecture in Section 7. In the same section, we propose potential solutions and techniques to address the challenges. We also propose a potential use case of artificial intelligence based forecasting methods with Fog architecture towards solving QoS, reliability issues. Towards the end of this section, we discuss the challenges in the integration of Fog Computing and smart grids and, the Fog simulation platforms presently available. Finally, conclusions are made in Section 8.

The originality, scope, and novelty of this paper is the definition and formulation of the problem of smart neighborhoods in the context of the smart grid (Section 6). The formulation of the problem is done through an extensive literature study of Fog Computing fundamentals, related foundation technologies for Fog Computing, Fog Application domains, Smart Grid and Community Microgrid fundamentals. We also propose potential solutions to the problem of smart neighborhoods by a Fog Computing architecture. This proposal is made without rigorous analysis as it is based on the literature review, Artificial intelligence and Blockchain techniques.

2. Related Works

In this section, we describe the taxonomy of the research in Fog Computing based on which (in Section 8) we propose Fog Computing architecture and potential solution methodologies towards realizing smart neighborhoods. It broadly covers research articles from application domain perspective, architecture perspective, networking perspective, resource management perspective and, security & privacy perspective.

Table 2. Overall category of papers cited in this work.

Domain	Research Works
Fog architectures, Frameworks and Programming Models	[12,13,14,15,16,17,18,19,20,21,22,23,24]
Resource management, Provisioning and Control	[25,26,27,28,29,30,31,32,33,34,35,36]
Fog Applications in healthcare, smart grids, smart vehicles and other domains	[8,37,38,39,40,41,42,43,44,45,46,47,48]
Security and Privacy in Fog Computing	[49,50,51,52,53,54,55,56]

summarizes the research works according to these categories. The objectives of these papers are achieving quality of service, resilience, reliability, mobility, scalability, cost optimization and, security & privacy of IoT applications. These issues described in detail in Section 5. Fog Computing has applications in various domains like smart grids, smarts healthcare, smart manufacturing, smart connected vehicles, smart agriculture and so forth. Each application domain has unique domain-specific working principles and technologies that form the basis of its functionality. Even though these domains have different functionality, they all are operated by sensors networks for providing services and therefore, require Fog Computing.

We describe a short survey of literature which are most relevant and helpful for us to formalize the problem later, and propose potential architecture and solutions. The literature is divided into four categories.

• Fog architectures, frameworks and programming models related research.
• Resource management, provisioning and control related research.
• Fog application domains related research.
• Security and privacy related research.

2.1. Fog Architectures, Frameworks and Programming Models

The first category of research works defines Fog Computing and Fog nodes comprehensively and tries to standardize the field of Fog Computing. They propose Fog architectures, frameworks, programming models and provide techniques to develop them. Open fog consortium is a consortium of high tech companies and academic institutions that is working to standardize Fog Computing and develop an architecture to support Cloud and IoT infrastructures [12].

Vaquero et al. [13] propose a comprehensive definition of the fog and the comprehending technologies as such as Cloud Computing, wireless sensor networks, software defined networks (SDN), network virtualisation (NFV) as the enablers of Fog Computing to realize its potential. Several comprehensive surveys, covering evolution, architecture, networking technologies and research challenges are described in References [14,15,16,17]. The authors of Reference [14] describe a review of current literature in Fog Computing with an emphasis on architectures and algorithms for fog systems. Baktir et al. [15] emphasize how Fog Computing can benefit from Software Defined Networking (SDN) and, review the SDN technology. They support their research by discussing SDN features and provide use cases that Fog and Edge Computing can benefit from. They also provide future directions for the development of SDN to realize Fog Computing. Li et al. [18] discuss on architectural design and system management of edge computing by providing a survey in the edge computing area. They try to define Fog and edge computing by comparing with related computing concepts, like peer-to-peer computing, mobile grid computing, and so forth.

Varshney et al. [19] discuss the difference between edge computing, Fog Computing, and Cloud Computing along with surveying various system architectures, application characteristics, and platform abstractions of these three technologies. A three-tier Fog architecture is discussed by References [20,21] by aiming to reduce the service delay and comparing the reduction in power consumption, cost and latency with respect to cloud systems. Programming models and frameworks for Fog Computing that are geo-spatially distributed, mobile, scalable and latency-sensitive are discussed by various works. Mobile Fog [22] proposes a high-level programming model that can dynamically scale based on the workload on Fog nodes. The efficacy of their model is shown with camera networks and connected vehicle applications. Giang et al. [23] propose a Distributed Dataflow (DDF) programming model for IoT devices using infrastructures ranging from Fog to the Cloud and evaluate their model. On the same lines, FogFlow by Cheng et al. [24] offers a programming model that allows developing IoT services over the cloud, edge and fog resources that is geo-distributed, hierarchical and, heterogeneous. It defines three logical divisions (1) service management; (2) data processing, and (3) context management. The authors demonstrate the efficacy of their programming model using smart city application domain by implementing a use case application for anomaly detection of energy consumption in smart cities.

2.2. Resource Management, Provisioning and Control

The IoT sensors and fog nodes are heterogeneous, mobile and dynamic, both spatially and temporally. The service requests from users vary dynamically. Fog nodes should be able to manage resources, scale dynamically and balance the application workloads. Therefore, resource management and service provisioning are essential for application performance and QoS, which is the backbone of Fog Computing. This second category of research discusses these areas. The subcategories under resource management, provisioning and control are service provisioning, orchestration and migration, service placement, scheduling, load balancing and application offloading.

Mostafa et al. [25] provides a comprehensive and detailed survey of resource management approaches in Fog Computing. All the resource management approaches are thoroughly compared with each other based on performance metrics, use cases, methodology, and so forth, and their advantages and disadvantages are also discussed. The authors of Reference [26] propose a theoretical online algorithm for service provisioning that dynamically configures Fog and Clouds based on historical patterns without any assumptions on the arrival patterns of mobile IoT applications. Deng et al. [27] propose a mathematical model for the problem of optimal workload allocation between Fog nodes and Cloud to achieve the trade-off between power consumption and service delay. Urgaonkar et al. [28] propose a solution to the problem that, where IoT services should be migrated in response to the user mobility and variation of the demand; they use Markov Decision Process (MDP) that takes into account server response time and network state to make the decision.

Fog Computing uses a network of distributed systems, and one of the issues in distributed networks is service orchestration. There are several works which propose solutions to the problem of orchestration of services in Fog Computing and propose frameworks [29,30]. Reference [29] proposes a fog orchestrator by facilitating an inter-operable resource pool to deploy and schedule resources to application workflows and control QoS. Foggy by Santoro et al. [30] proposes a framework and a software platform for workload orchestration and resource negotiation in a multi-tier, highly distributed, heterogeneous and decentralized Fog Computing environment. Reference [31] proposes a framework that integrates fog nodes with cloud server to enable auto-scaling and handling edge resources dynamically.

In the same category of service provisioning, there are works which address the problem of task scheduling, service placement, VM placement, caching, control and monitoring while taking into account QoS requirements and optimization in fog networks [32,33]. Task scheduling in Fog Cloud nodes is studied by various researchers [28,34]. The authors of Reference [34] formulate the task completion and time minimization problem with the consideration of task scheduling and VM image placement as a mixed-integer nonlinear programming (MINLP) problem. At the same time to solve the problem of Fog orchestration, authors of Reference [35] propose, using SDN architecture and address scalability issues of Fog Computing. For load balancing in Fog networks along with considering energy efficiency, Singh et al. [36] discuss various load balancing algorithms and perform their comparative analysis.

2.3. Fog Application Domains

Research works under this third category discuss solving various critical problems in the field of smart healthcare, smart cities, smart grids, smart connected vehicles and smart manufacturing by using Fog Computing.

Smart connected vehicles and transport applications make travel safer by reducing traffic congestion and providing real-time information to passengers. To address these issues, References [37,38,39] describe the need and role of fog servers to create vehicular network architecture and also discuss the challenges to realize such a system are. Also, in smart connected vehicles, to increase the quality of experience of a user requesting a video, the authors of Reference [40] propose caching and cache node selection algorithm. They propose an algorithm that uses constraints vehicle connectivity degree, Zone of interest, node capacity, user preference and its location to decide the cache location.

Smart healthcare promises a health system that provides healthcare services by using technologies such as wearable devices, IoT devices to access information in real-time, connect people, and healthcare institutions in an intelligent manner. Fog Computing is emerging as a key enabler of smart healthcare for ensuring adequate patient safety. References [41,42] use Fog Cloud architecture to enable effective healthcare services for patients by making data available to doctors, healthcare providers and helping them make informed decisions with minimum latency. Similarly, Reference [43] is based on a computational distributed Fog Computing approach to process the sensed data from the patients of dementia and Chronic obstructive pulmonary disease from the home environment to send to the doctors. This helps them to make fast decisions with the comfort of the home provided to the patient. For treating better patients with Parkinson’s disease, Monteiro et al. [44] propose a Fog Computing interface for processing speech data of patients wearing a smartwatch. Clinicians access this data for quick decision making based on the patient’s condition. Another application is monitoring brain states with Fog Computing architecture. Zao et al. [45] uses augmented brain-computer interface can detect users brain states in real-life situations using wireless EEG headsets, smartphones and ubiquitous computing services. Another experiment for real-time data retrieval of parameters heartbeat, temperature and humidity to reduce time delay using Fog Computing is done in Reference [46]. The performance of the data retrieval process is compared between Fog Computing and Cloud Computing, and results are shown to prove that Fog Computing outperforms than Cloud Computing.

Smart grids are the focus of our research in the present work. We describe the concept of the smart grid and the power system operation and, then propose the solutions with Fog Computing. In literature, there are few works which describe in totality connecting various components of smart grids and real scenario of smart neighborhoods with smart meters. Some works use Fog Computing for real-time processing of smart meter data. For example, Reference [8] creates a network of smart meters which are connected to a master node, and the master node monitors the smart meter data. Through their proof of concept, they ensure smart meter data takes less time to process without the need to be sent to the cloud servers. Similarly, FogGrid by Barik et al. [47] use Intel Edison as a fog device with dual-core, dual-threaded 500 MHz Intel Atom CPU along with a 100 MHz Intel Quark micro-controller to process smart meter data and report the energy consumption and processing time to show the efficacy of their approach. A three-tier fog based smart grid model is proposed in Reference [48], and this paper also proposes encrypting customer key to ensure security.

2.4. Security and Privacy in Fog Computing

The fourth category of papers addresses security and privacy challenges in Fog Computing and have been extensively studied. One of the major drawbacks of Cloud Computing is that the user has limited or no control over its data. Cloud is prone to security attacks. Fog Computing promises privacy and security due to distributed localized processing of the data. The fog service provider needs to ensure the consumer decides about which parties have the access rights of their sensitive data.

A survey of security and privacy issues in Fog Computing is done by References [49,50]. To go further in specific security and privacy issues, the authors of Reference [51] discuss security and privacy aspects in Fog Computing and propose an approach that employs fog to improve the distribution of certificate revocation knowledge among IoT devices. Fog Computing nodes form a distributed network and blockchain technology which is a trusted shared, distributed, decentralized, database archive, that caches a record of transactions across the network can be easily integrated with Fog Computing. A Blockchain-based secure framework for Fog Computing is proposed by Reference [52]. Another blockchain based secure framework by integrating sensors, Fog nodes and the cloud server is proposed by Tuli et al. [53]. Their framework ensures reliability, security, QoS. They test their framework by implementing the use case of sleep apnea analysis and show the efficacy of their framework.

Reference [54] proposes a protocol for service provisioning at the edge of the network, called Named Data Networking(NDN) which authenticates consumer and the service provider. Another research for the security of sensitive medical sensor data is proposed by Giri et al. [55]. It uses four-step protocol (a) setup phase, (b) registration phase, (c) authentication phase, and (d) data delivery phase before sending data from the medical sensor to the fog server to verify both the sensor and the fog server. A secure authenticated key exchange scheme is proposed for Fog node authentication by the authors of Reference [56]. They propose a three-step authentication process of (1) registration phase, (2) fog server registration phase, and, (3) login and authentication phase for the fog node authentication process.

3. Related Technologies and Their Characteristics

There are various previous technologies which laid the foundation of Fog Computing. These technologies provided solutions to one or more issues of low latency, mobility, location awareness, heterogeneity, security, privacy and resilience and, also proposed software and hardware architectures to solve these issues. But they were not addressing all the issues. Fog Computing aims to address all the issues. In this section, we discuss all the previous technologies and the challenges they addressed.

3.1. Mobile Computing (MC)

Mobile Computing aims to do processing and application execution on mobile devices such as mobile phones, tablets and other portable devices [57]. The mobile devices are connected to Bluetooth, Zigbee, WiFi and other cellular protocols. The mobile users are constantly on the move and are resource-constraint as compared to Fog and Cloud Computing.

Also, there are issues of network heterogeneity and low bandwidth. Although recent advancements in mobile hardware and wireless protocols have made the computing more effective, there are challenges of distributed architecture, latency, adaptation to changing mobile environments and security [58].

3.2. Mobile Cloud Computing (MCC)

Mobile Cloud Computing is a natural extension of Mobile Computing where the necessity of computing, storage and communication is provided by combining the capabilities of mobile devices with local clouds [59]. Computation intensive applications on mobile devices are offloaded to nearby servers. MCC also suffers various issues like high latency as application offloading is made on centralized cloud servers and, it is not suitable for applications where the pervasiveness of devices is required [17].

3.3. Edge Computing (EC)

Edge Computing brings cloud capabilities on the edge devices. The computing and storage happen near to the sensors or IoT devices but not on the IoT devices. Cloud services are brought closer to IoT devices. Due to the proximity with users and the IoT network, issues of latency, connectivity and security are addressed in Edge Computing as these issues are prominent in MCC and MC [60].

In edge computing, edge nodes are restricted to the edge of the network, such as WiFi access points or gateways. The computation done actually on IoT devices is called Mist Computing which is also used for specific applications. Edge nodes are resource-constrained devices, so the quality of service is an issue in them.

3.4. Multi-Access Edge Computing (MEC)

MEC provides Cloud Computing platform at the edge of the radio access network. It offers processing, storage at the edge of the network, thereby reducing latency for mobile end users. It efficiently utilizes mobile backhaul and core networks [61]. Its an initiative from the European Telecommunication Standards Institute (ETSI). It was initially restricted to mobile devices; however, after 2017, it covers non-mobile devices also. Its a combination of mobile computing and edge computing. MEC suffers from issues of heterogeneity of different wireless and mobile networks, scalability of IoT devices, and security issues [62].

In summary, Fog Computing aims to provide computing, storage, networking services from anywhere between IoT devices and cloud server. Reference [63] mentions that “fog is inclusive of cloud, core, metro, edge, clients, and things”, and “fog seeks to realize a seamless continuum of computing services from the cloud to the things rather than treating the network edges as isolated computing platforms”, and “fog envisions a horizontal platform that will support the common Fog Computing functions for multiple industries and application domains, including but not limited to traditional telecom services”. Fog Computing has a distributed, hierarchical architecture that supports multiple application domains. We compare the various related computing paradigms and Fog Computing in Figure 2. It shows that Fog Computing is distributed Computing between IoT sensor networks and Cloud servers.

3.5. How Fog Computing Is Reliable than Other Computing Paradigms

According to OpenFog Consortium [12], Fog Computing services will continue to operate even if a Fog node, inter-node link or the connection with the Cloud server goes down. SDN and NFV widely adopted across WAN, LAN and access networks in Fog Computing ensure network device management and flexibility [64]. According to Reference [13], the integration of SDN with fog network will address the challenge of mobility and unreliable link. Fog Computing, coupled with NFV, will improve the services by the virtualization of gateways, switches, load balancers, and other network devices by placing the virtual machines on fog nodes. Several authors have used SDN and NFV for reliability issues which are discussed in Section 2.1. Another important technology that addresses reliability issues in Fog Computing is blockchain. Due to the distributed hierarchical architecture of fog networks, blockchain is easily integrated into it. As discussed in the related works Section 2.3, there are several works [52,53] that address reliability and security issues in Fog Computing by integrating blockchain technology on distributed Fog nodes. The other computing paradigms do not consider all the sensor networks, all the network types, are restricted to mobile devices, are not geospatially distributed and have security and privacy issues. This makes Fog Computing more reliable and versatile than other computing paradigms.

4. Applications of Fog Computing and Use Cases

Fog Computing is becoming crucial in various domains. Every domain, whether it is healthcare, smart grids, manufacturing, smart vehicles or smart cities, is operating with sensor networks. We need fast processing of sensor data and effective operations. Figure 1 represents a high-level Fog Computing architecture and its application in these different domains. In this section, we describe briefly application different domains and how Fog Computing can play an important role to address challenges in these domains.

4.1. Smart Grids

The power sector is being revolutionized by incorporating Smart Grids and Micro Grids. Smart grids use Information and Communication Technologies (ICT) for two-way communication between utility companies and customers and use sensors along the power pipeline to automate processes and, better power management. Smart Grids purpose is to manage power demands, minimise power wastage, detect faults in the power lines, and integrating renewable energy resources for clean and safe electricity. Using Fog Computing, various sensors functioning in the power pipeline will collect the data and send to the Fog nodes rather than the Cloud Server, and make the smart grid applications effective in a real environment [48].

4.2. Smart Healthcare

Healthcare sector is an important application of Fog Computing because of the need for urgent patient care and early diagnosis. To reduce the cost of treatment and frequent visits to the hospitals, smart wearables are attached to the patient bodies which measure the various physiological parameters like blood pressure, oxygen level, respiration rate and so forth. And these physiological data needs to be processed in real-time for making urgent decisions. Fog nodes and servers provide data processing and analytic near the sensors. Therefore, quick preventive measures can be taken by doctors and hospital support staff to ensure patient safety and effective treatment [65,66].

4.3. Smart Manufacturing

The manufacturing industry is undergoing a revolution as industrial systems use the Industrial Internet of Things (IIoT) to deploy sensors and automate systems, to measure, monitor and analyze data, to improve performance, and increase profit opportunities for manufacturing operations. The amount of data from these manufacturing plants can be measured in the petabytes (1 million gigabytes).

Manufacturing devices generate vast volumes of real-time data from the connected sensors on industrial control systems, autonomous drones, industrial robots, video surveillance cameras covering plants, and so on. IIoT applications require real-time processing, low latency, local storage, reliability and high bandwidth, all of which can be fulfilled by Fog Computing architecture [67,68].

4.4. Smart Connected Vehicles

The market value for Self-Driving Cars (SDC) is growing exponentially and is predicted to be valued over 555 billion US dollars by 2025 [69]. For SDCs to be aware of its surroundings and also communicate with other SDCs requires a lot of computing power and real-time feedback. Vehicular Cloud Computing services are centralized. On the contrary, Vehicular Fog Computing will play a critical role in real-time decision making by doing real-time communications with various vehicles and analyzing the huge sensor data in a dynamic mobile environment [39,70].

4.5. Smart Cities

Fog Computing can effectively handle issues that communities deal with every day by making them smart like agriculture, pollution, parking, waste management, parks, public safety, water management, future infrastructure planning, and so on [71].

In modern-day agriculture and farming, various sensors monitor plants growth, climatic conditions. All these data can be efficiently processed at the Fog nodes to provide insights and better crop production without huge manual efforts.

For smart pollution control, sensors are installed on buses and other vehicles, and different areas of the cities. Processing data on local fog nodes can give real-time results. Fog Computing can enable a holistic view of vehicle traffic and crowd patterns (including vehicle types, speeds, densities, movements), using both historical and real-time data. This can provide critical insights for smart cities future economic and infrastructure planning.

For smart waste management, garbage collection needs to be done optimally. Sensors on the garbage bins tell about the amount and type of garbage, and that data can be sent to the nearby fog nodes for processing. This processing results can be used by recycling companies to tract the amount of garbage coming to the plants and save manual efforts.

A smart water system can help in monitoring city-level water consumption, groundwater monitoring, prediction of future water use, wastewater treatment, and so forth. These will rely on the support from Fog/Cloud Computing infrastructure.

In case of a terrorist attack or a natural calamity that breaks through a centrally located facility, by distributing data processing over distributed fog nodes, instead of centralized way in a data center, there is less probability that services will go completely offline.

5. Smart Grids towards Smart Neighborhoods

The traditional power grids do not use ICT technologies to manage the power supply, and thus it leads to power outages and ineffective grid operations. These days, traditional grids are getting replaced with smart grids.

Traditional homes have appliances that are operated locally and manually. These devices have limited control. So managing how much energy they use is nearly impossible. Consumers receive electricity bills once a month. The customers have no way to correlate the amount of money spend on electricity with how is it used. Furthermore, for renewable energy resources integrated with smart homes, there is no way to measure the amount of electricity produced by them. The smart home within smart grids promises energy-efficient controllable appliances and real-time access to energy usage data. Here we describe smart grids and MicroGrids concepts which will help us in formalizing the problem of smart neighborhoods.

5.1. Smart Grids Concepts

A smart grid uses digital technology that allows for two-way communication between the utility provider and its smart home customers [72]. It is the IoT sensing networks across the generation, transmission, distribution lines and eventually to smart homes within smart neighborhoods that effectively makes the grid “SMART”. This two-way communication is represented in Figure 3. In Figure 3, dashed lines represent the flow of electricity while the solid lines represent the communication channel. Power generators generate electricity which is sent to the transmission lines, from transmission lines it goes to the distribution lines and, finally to the consumers in neighborhoods. The sensor data generated along this electricity channel has properties of volume, variety, veracity and velocity and therefore, demands real-time processing for quick control and monitoring of smart grid operations and outage prevention [73].

ICT technologies are integrated into smart grids for effective power management and, reliable and secure supply of power at all times. Smart grids also ensure clean, sustainable energy by integrating with renewable sources of energy like solar PV panels and wind turbines. This provides customers with ways to proactively manage energy which is convenient, cost-effective and environmentally friendly [74].

5.2. Advanced Metering Infrastructure

It is important to talk about smart meters and the functions they perform to understand smart grid operations. Apart from measuring the power consumption by individual smart appliances, smart meter communicates with other intelligent devices in the home and the grid utility provider. It notifies the consumer of a power outage, consumption behavior and utility provider for system monitoring and customer billing. It monitors the power quality and performs automatic remote turn on and turn off operations of appliances [75]. Smart meters are not working separately but are organized as networks. So, every household smart meter can communicate with each other.

Advanced metering infrastructure (AMI) is an integrated system of smart meters, communication networks and, energy & data management system that aids in two way communication between energy providers and energy consumers [75]. Smart meters in every smart home are connected to form a network which gains perspective about operations mentioned above by processing data on fog nodes. As shown in Figure 4, Home area network has smart home appliances sending data to smart their meters. The network of smart meters in a neighborhood is connected to a gateway and send the collective data to the Fog server for that region for processing. The fog server runs the meter data management system.

5.3. Vehicle to Grid (V2G) Integration

The transportation sector is undergoing massive changes as more and more vehicles are becoming electricity powered. EVs are one of the feasible solutions to the challenges such as global climate change, energy security and geopolitical concerns on the availability of fossil fuels. The smart grid has essentially no storage (other than its 2.2% capacity in pumped storage [76]), so generation and transmission must be continuously managed to match fluctuating customer load and effective demand response management. The EVs integration in smart grid serves as an independent distributed energy storage source which can potentially address above challenges. The energy stored in the EV batteries can potentially provide power to grid in needs. This is called Vehicle to Grid technology (V2G). Whenever there is a power request from the utility provider, EVs should provide power within minutes. This near real-time response can be possible with a local processing technology available near the EVs. V2G has to fulfill the needs of the consumer/driver and the grid operator. The driver needs enough stored energy on-board for the driving needs. The grid operator needs power usage to be turned on and off at precise times. Intelligence controls should be available near the EVs for satisfying consumers and the grid operator [77]. The typical services that the V2G operation can offer are summarized as follows:

• Store power;
• Load shifting by peak shaving and valley filling;
• Renewable energy resource integration;
• Frequency and voltage regulation;

Integration of a large number of EVs into the smart grid is a also a major challenge which requires an intensive assessment and observation for operation and control of smart grid operations.

5.4. Renewable Energy Source Integration

Renewable energy sources such as PV panels and Wind turbines are sources of energy integrated in local neighborhoods that can provide a variety of benefits including improved reliability, security, efficient energy system if they are properly operated in the electrical distribution system [78]. They cause minimal local environmental damage and net emissions of greenhouse gases (GHGs). RES integration makes the Grid operate as MicroGrid which can function independently from the main grid. Increasing number of consumers are installing PV panels and Wind turbines locally to control their power needs at reduced cost and, also trade power to other consumers. Various studies have found that a large number of utilities as well as consumers that have installed RES at their facilities to realize benefits mentioned above [78]. Also, in order for the reliability of MicroGrid and challenge of intermittent power generated by RES, EVs are used.

The typical services that the RES integration in MicroGrid plays are as follows:

• Reliable MicroGrid operation;
• Clean environment friendly energy availability;
• Secure and privacy preserving MicroGrid operation;
• Energy supply to store in EVs;
• Energy cost reduction;
• Frequency and voltage regulation;

Figure 5 shows the renewable energy sources, solar panels and wind turbines and, EVs integration in smart grid at community level. They make it possible to realize MicroGrids.

5.5. Community MicroGrid

Smart grid integrates renewable sources of energy like PV panels and wind turbines to provide a sustainable green source of energy [79]. PV panels and wind turbines have multiple sensors. Each sensor is able to record current climatic parameters like wind speed, light intensity, and so forth. The electricity generated by PV panels and wind turbines makes the grid to function independently for household power usage. An independent grid makes the smart grid reliable and is commonly called MicroGrid. Also, due to energy storage units situated in the community neighborhood, Microgrid works more independently. We focus on MicroGrid at the community level, which is called community MicroGrid. A community MicroGrid is a self-sustained local energy grid where it can disconnect from the traditional smart grid and operate autonomously [80]. It provides power back up to the grid in case of emergencies when there is a fault at the power lines or, when the distribution lines are cut off from the households or, due to security issues. To summarize, Community MicroGrid supports the power transmission system by supplying power and, to the customers, it enhances local reliability, reduce feeder losses, support local voltages, perform voltage sag correction and provides uninterruptible power supply [81]. Households have storage batteries which store excess power generated by renewable sources. This excess energy can be traded to other households or used in case of shortage of power to reduce costs.

As the renewable resources of energy like PV panels and wind turbines are dependent on the weather forecast, prediction of weather parameters is crucial. Fog Computing will play an important role in realizing community MicroGrids by providing a secure platform for real-time analytics of forecasting weather parameters and future energy production. Along with forecasting, quick fault detection in the sensor devices of PV panels or wind turbines through data analytics can prevent potential power hazards.

5.6. Demand Response Management

Demand response management (DRM) is the key component to increase the efficient use of electricity by remote monitoring and control of electricity load by setting efficient electricity prices with the objective to shift the high demand of electricity users to an off-peak period. It effectively reduces power generation costs and consumer bills [82]. With this, DRM improves energy efficiency and reduces overall electricity wastage for both consumers and producers. Let’s understand the need and importance of DRM more with the help of an example. Most of the households in a neighborhood have almost the same electricity consumption patterns for 24 h. During evening times, around 5–6 p.m., users come home from their workplaces and turn on all their appliances like TVs, lights, electric vehicle charging, and so forth. Therefore, power consumption during the evenings is much higher compared to afternoons or night when the consumer is not using most of the appliances. But, power generation companies generate power as per the peak hours of power consumption by the majority of the consumers in the neighborhood. This leads to power wastage and loss for the power service providers because other than a short duration of peak hours, the power generated can not be utilized for non-peak hours. DRM effectively schedules smart home appliances in such a way that distributes the peak hour power consumption to non-peak hours so that power consumption throughout 24 h does not have high spikes and power consumption curve is flattened. This also benefits the consumers because they get to pay less for the same electricity usage.

The real-time monitoring of electricity consumption and scheduling strategies of appliances will implement DRM and improve grid reliability and utilization. Fog Computing will be a key technology enabler to implement DRM.

6. Smart Grid Challenges and Problem Formalization for Smart Neighborhoods

In this section, we describe the key challenges of smart grids and MicroGrids and, define the problem of smart neighborhoods.

6.1. Real-Time Control of Monitoring of Grid Operations

The smart grid has to improve regarding efficiency, energy management, reliability, and cost- effectiveness by considering its real-time implementation. We need real-time communication between consumers and utility providers to fulfil the demand and supply needs effectively. Most of the approaches for demand response management, meter data management system (MDMS), outage management, assume perfect forecasting of the renewables, the demands, and the market, which is difficult to achieve in practice. Scheduling of appliances and EVs should be done in minutes timeframe to adapt to the needs of every consumer in the neighborhood. The intermittent nature of renewable energy sources (such as wind power and solar power) adversely affects the MicroGrid voltage, frequency, and so on and this becomes more complicated in neighborhoods areas integrated with small-scale renewable energies [83]. Hence, we need an infrastructure that provides whole grid coordinated monitoring, control and automation in real-time. Fog Computing is a promising solution to address these challenges.

6.2. Scalable and Distributed Smart Grid Architecture

Scalability is one of the most important issues for the deployment of smart grid with respect to AMI infrastructure, RES integration and V2G integration. The number of smart meters is increasing rapidly from hundreds to thousands in neighborhoods. Similarly, more and more consumers are installing RES locally to save costs. The number of electric vehicles is increasing at a high pace and serve as a reliable power storage source for MicroGrid. Therefore, is is important to have scalable and distributed processing platform to address these challenges. Due to the natural distributed nature of power lines, there is a need for the distribution of intelligence throughout the lines since centralized Cloud systems are too slow for this goal [84]. Also, when a local defect is identified and handled on the power line, then it does not propagate as a cascading effect to create bigger faults.

Managing increased variability in the smart grid will need advanced large scale distributed intelligence control and optimization algorithms to be run on distributed processing nodes. To address the challenges of renewable resource variability, effective demand response management, V2G integration, MicroGrid integration, effective AMI operations, and outage management requires systems and control technologies. A distributed Fog Computing based hierarchical architecture is a promising solution to address these challenges.

6.3. Reliable Power Supply

The well-known “8-14” blackout in the USA and Canada brought about a dramatic economic loss for both countries in 2003 [85]. One of the foremost requirement of consumers is an uninterrupted supply of power at all times. The microGrid is a complex system. The electricity prices vary with the overall load of the entire grid in real-time, and the renewable source power generation is intermittent and uncertain. Some smart appliances are non-interruptible and non-power shiftable. This makes it challenging to control peak power values and effective DRM [86]. This can potentially lead to grid imbalance. Any small defect in the production or consumption lines can propagate, which can potentially lead to a power outage. The sensors along the lines can malfunction. In face of the above challenges, it is crucial to have an infrastructure that does fault analysis in real-time by analysing huge sensor data. It should provide coordinated monitoring, control and automation. Distributed Fog Computing architecture is the most promising solution that ensures high performance, service reliability and prevention of power hazards.

6.4. Security and Privacy Preserving Grid Operations

Smart Grids and MicroGrids are prone to cyber-attacks with some major attacks that happened in the past [87]. Smart meter data is the most important component for the service providers. The communication network for sensor networks may not be secure, and communication protocols may not be robust. It is susceptible to cyber-attacks. Also, smart meter readings can be tampered by unauthorized users. Intercepting, tampering, misrepresenting, or forging the smart meter data will cause imbalance to the grid and is potentially hazardous. The attacker can access sensitive information about consumers and is potentially hazardous. The consumers who are the owners of their data have almost no control over their data. They have no control and authority of who can access their data. They have less or no information, whether the party accessing the data is authorized or not [88]. Grid customers are connected over a vast network of computerized meters and infrastructure; they and the infrastructure itself is vulnerable to scalable network-borne attacks. A number of researchers discussed privacy issues in smart meters [89,90].

Due to the RES integration at consumer sites, consumers want to trade the extra electricity produced to other households. They want to trade electricity via a secure channel and to authorized consumers. Therefore, to address the challenges mentioned above and to ensure secure trading of electricity in a community among households, we need a secure and privacy-preserving distributed architecture. Fog Computing based distributed architecture addresses all these challenges.

6.5. Effective RES and V2G Integration

The smart grid is significantly affected by the integration and deep penetration of renewable energy sources because of their variability. Wind power and solar power is more complex, and unpredictable variations in weather make it strongly intermittent, leading to imbalances. The intermittent nature of RES adversely affects the grid voltage, frequency, and so on. A lot of challenges are initiated by this integration, such as effective forecasting, energy storage management, demand management systems, voltage control, and power system stability [91,92]. Also, the number of EVs is continuously increasing, and this is a challenge for the Grid as how it can integrate and keep pace with the increasing numbers of EVs. Additionally, EVs should be charged during off-peak hours to reduce the consumers’ cost, but to the disadvantage of the consumer, they will have less control over the time of charging of their EVs. Charging the EVs at peak hours will increase the peak power and potentially cause instability to the grid [93]. Therefore, to effectively address the variability of RES challenges, we need a processing infrastructure which can predict power produced by RES in real-time. For effective V2G integration, it is crucial to schedule EVs in real-time based on the users’ demands.

6.6. Optimize Power Production and Consumption and Minimize Loses

Even though the smart grid is becoming increasingly complex, one of the basic purposes is to optimize production, transmission and consumption of power and increase profit to power companies and consumers. A huge transmission, distribution energy transport network is created over the years, but it is not optimized. The goal should be to make smart grids and MicroGrids more efficient by integrating information and communication technologies. The optimization objectives are energy efficiency improvement, supply and demand balance, emission control, operation cost reduction. We need a distributed smart infrastructure to run optimization algorithms on a complex smart grid with AMI, RES and V2G integration. One such use case is that the consumers desire power to be available at the lowest prices. The power producers want the power to be produced at the lowest price, maximize their profit and least power wastage. Every household consumer has variable power consumption patterns. This process is dynamic, and power prices vary depending on the time of use (due to peak and non-peak hours) and change in real-time by utility providers. The prices are set by the power producer based on the consumption patterns of consumers, peak hours usage and the power production [94]. This requires continuous power forecasting and cost optimization techniques to be run and requires localized computing resources. Therefore, for optimization issues in smart grid real-time distributed Fog Computing architecture is a feasible solution.

We define Smart Neighborhood as a data-driven decentralised community MicroGrid infrastructure to achieve community-level grid reliability, demand response management, secure and privacy preserving operations. It integrates local renewable energy sources and electric vehicles for local production and storage of electricity and use automation, control, optimization and analytics for effective MicroGrids.

7. Fog Computing Based Architecture towards Smart Neighborhoods and Potential Solutions

As shown in Figure 6, we propose a distributed Fog Computing architecture for processing smart meter data and to address the challenges mentioned above. We introduce an intermediate layer between smart meter networks and cloud server, which is a hierarchical distributed network of fog nodes attached with fog servers for individual smart neighborhoods. These networks are geographically distributed rather than being centralized. The data from smart meters from one neighborhood is sent to fog network 1. Whereas, the data from smart meters from the second neighborhood is sent to fog network 2. More than one smart neighborhoods are connected to the same fog server. Smart homes in one neighborhood have smart appliances that have sensors which measure the power consumed. These appliances communicate through Bluetooth, Zigbee, Wifi and other short-range protocols with other appliances. Streaming minute-sampled data appliance level data is generated recorded by the smart meters. As clear, real-time analytics of power data is the most important requirement to ensure availability of smart grid services.

A neighborhood will have a number of houses depending upon the processing capacity of the fog network it is attached to. Fog server will perform the following operations:

• Data storage
• Distributed and scalable Fog Computing architecture
• Real-time service
• Resource management, provisioning and control
• Resilient and Reliable Grid operations
• Security and Privacy preserving architecture
• Coordination between Cloud and grid sensor networks

The fog server can be analogous to a cloud server with comparatively less computation and storage capacity that receives data from individual fog networks connected to it. The fog server is a highly virtualized environment where different VMs are meant for executing various fog applications. The fog server also sends relevant data to the centralized cloud server for large scale data processing and service provisioning. It is important to know that Fog Computing will not replace Cloud Computing altogether; rather, it will act as a vital supplement to the Cloud Computing [95].

7.1. Identifying Potential Solutions

In this section, we propose potential methods and technologies that will address the challenges mentioned above, ensures demand response management and cost optimization. Our solution techniques will also address the problems of real-time service delivery, reliable power supply and security & privacy-preserving service. Section 5 “Challenges in Fog Computing” discusses the challenges in detail in Fog Computing and Section 7 presents challenges to realize smart neighborhoods. The potential solution proposed by us is shown in Figure 7. We propose that artificial intelligence and blockchain techniques coupled with Fog Computing architecture are at the core to solve the challenges.

We explain our solution approach with the help of use cases of community MicroGrid and challenges we already discussed.

• Artificial Intelligence based prediction methods: Prediction is the most effective method to ensure power outages are minimized, and demand response management is achieved. Predictions using machine learning, deep learning and swarm intelligence techniques are used in various domains of study in many fields [96,97,98,99,100]. In our case of community MicroGrids, as discussed in Section 7, consumers need a reliable power supply, and any anomaly in the power consumption should be quickly detected. Here predictions techniques are the most effective. Since minute-sampled power consumption data at the appliance level is available through smart meters, predictions of power consumption for the next couple of days in every smart home can be made. In this manner, the power supplier/service provider will be able to know how much power should be to be generated and help them in making decisions. At the same time, forecasting the generated power by the power producers will help utility providers become aware of the power production in coming days and therefore, inform consumers about the costs based on the production. This will do the scheduling of appliances with customer preferences and cost minimization. With this, demand-side management with minimum energy loss will be effectively achieved through predictions.
  For reliability issues or detecting faults like sensor faults, malfunctioned devices or security threats also, forecasting is the most effective tool. We can analyse and forecast future power consumption values from the previous month’s data. Large deviations from the predicted consumption could be a potential fault, and customers and service provider will be alerted the customer about the possible faults (after eliminating false alarms). They will be able to locate faults at the appliance level because granular data analysis and real-time fault detection are done on Fog nodes.
  Potentially high number of use cases can be handled with forecasting techniques as sensor data, and computing resources are available near the sensors. Another use case is predicting power produced by wind turbines and PV panels. With weather data of past weeks and months, we can predict how much power will be produced in the coming weeks, months. This will allow consumers to make informed decisions like, planning how much power they need to buy and decision about selling power to other users at maximum profit. Importantly, renewable sources of energy effectively reduce the carbon footprint by the neighborhood and contribute to the environment.
  To do effective resource management and load balancing, we can use prediction techniques. We can predict how much application processing load fog network of a particular region will have. Suppose one region having a fog network have a higher load then QoS will be compromised. In this case, to balance the load and transfer the processing request to a less loaded neighboring fog network will be an effective solution. It will also help in deciding what appropriate fog nodes locations based on the workload of the area are.
• Blockchain for secure fog platform: To address the security and privacy challenges mentioned above and secure trading of excess power among households, we need a secure platform trusted by the users. Blockchain can be used effectively to trade electricity among users in a network. Blockchain is a shared and distributed ledger used to record transactions by multiple untrusting nodes in a network. Blockchain is gaining popularity because applications can be operated securely in a distributed manner, which previously had to go through a third party intermediary which could not be trusted [101]. The transaction is made secure through validation by all the distributed fog nodes. A literature review on applications of blockchain technology is presented in Reference [102], describing the usability of blockchain in IoT sensor domains. Each user can manage their own data, and each fog node stores only encrypted fragments of user energy data [103]. Therefore, it is possible to achieve complete privacy of user data and secure trading without any third party having access to and control of the data.

Use Case of Using Artificial Intelligence Based Methods towards Realizing Smart Neighborhoods

To clearly demonstrate the use of prediction methods to address the challenges mentioned in the above sections, we present a solution architecture using Fog Computing and forecasting methods. We demonstrate how this architecture will ensure real-time detection of anomalous power consumption as, one of the many use cases it can solve. As shown in Figure 8, there are four layers: IoT layer, edge layer, a core layer and the cloud layer. The edge layer and core layer together form the fog layer. As we move from IoT layer to the cloud layer, the latency of application execution increases. The computational capacity of devices also increases as we move from IoT layer to the cloud layer. Edge layer is responsible for real-time decision making for anomaly detection, and the core layer is responsible for training resource-intensive prediction models (such as neural networks).

The smart meters are at IoT layer that generates the power consumption data at the appliance level. This data is sent to the edge layer, where data ingestion and preprocessing happens. The edge layer decides in real-time and sends the preprocessed data to the core layer. The forecasting of power consumption for the next days for all the appliances is performed at the core layer. After that, forecasting results are fetched by the edge layer. To detect anomalous power consumption with this approach, anomalous patterns can be defined by various methods such as rule-based methods and statistical-based algorithms [104]. Finally, the edge layer device activates the actuator in real-time and alerts the user in case of anomalies. Therefore, on incoming power consumption sensor data from smart meters, anomalies can be detected in real-time using our distributed Fog Computing architecture with forecasting methods.

7.2. Challenges in Fog Computing and Smart Grid Integration

The major challenges in the integration of Fog Computing and Smart Grid integration are described below:

• Security and Privacy: Security and privacy issues in Fog and Smart Grid integration are still a challenge. As discussed, Blockchain is a promising solution to address security and privacy challenges, but the research of their integration is still at the initial stages, and it is not tested by integrating with whole smart grid infrastructure [105]. Smart Grid is a complex architecture with the integration of AMI, RES and EVs. These components form an infrastructure which is distributed, heterogeneous and operates at the edge of the network. Sharing information on the distributed network, migrating services, load balancing and resource management between heterogeneous fog nodes at neighborhood level and, across the distributed transmission, distribution power channel is susceptible to malicious attacks.
• Fog Programming models and interoperability issues: There is no fixed set of rules and methods that describe the functionality, organization, and implementation of the Fog Computing based smart grid system. Fog Computing itself has no standardized programming models and frameworks, though there are several works in this direction [106]. Fog nodes need to do a lot of coordination among different stakeholders in the power infrastructure. The geographical distribution of sensors and uncertain network connectivity and interoperability with Cloud server is a challenge for reliable service [19].
• Reliability and scalability: Networks play an essential role in reliable communications in smart grid infrastructure. However, the traditional networks are not designed for high scalability of fog nodes as the number of smart meters, EVs are increasing continuously. SDNs are a promising solution [107,108] but they should be integrated and tested on whole power distribution network for network reliability. Another challenge is that fog nodes are not as resource-rich as Cloud servers and therefore, service provisioning and load balancing could delay the real-time performance of the smart grid services. AI integration on Fog devices is promising but at the same time is challenging. Artificial intelligence algorithms like neural networks give accurate predictions but require more processing capacity and data to give prediction results. Running these algorithms can affect the real-time performance of Demand response management and other smart grid services. Similarly, delay in accurate prediction of power produced by RES will cause reliability and performance issues.

7.3. Simulation Platforms for Fog Computing

The most effective way of trying and testing the potential ICT solutions to smart neighborhoods would be by creating a simulation environment. A simulation platform allows development and analyses of various solutions without the need for developing costly prototypes (the costs and the time includes purchasing, programming, maintaining, and configuring multiple pieces of equipment). There are many proven software and simulation tools already available for the simulation of the Fog Computing environment. Hence, we at the Electrical Engineering and Computer Science department are planning to develop a simulation platform for trying out Fog Computing-based solutions to smart neighborhoods.

For example, MATLAB/Simulink helps simulation of wireless applications, and the tool General-purpose Petri Net Simulator (GPenSIM, which is also based on MATLAB [109,110]) supports simulation of the discrete-events systems. The second author of this paper developed GPenSIM. Also, EmuFog [111] and iFogSim [112] are the tools for Fog Computing simulations.

It is noteworthy that the simulation environment that is planned is not only for on-site (on the lab) testing, it will support online usage too as researchers can test and run their application using their laptop from anywhere.

8. Conclusions

Fog Computing is becoming vital to IoT service reliability and security as billions of IoT devices will be connected to the internet providing services in smart healthcare, smart cities, manufacturing, smart grids, smart connected vehicles and numerous other domains. In this paper, we presented the Fog Computing paradigm, a short survey on the research done so far. We discussed key related technologies which came before Fog Computing. We discussed various challenges to realize Fog Computing fully. Next, we described smart grids and community MicroGrid key concepts and their challenges. With all this background, we formalized the problem of smart neighborhoods and the challenges to realize it. To solve those challenges, we proposed based on the literature review, a distributed hierarchical Fog Computing architecture. We also proposed potential solutions with a use case towards realizing smart neighborhoods. Finally we discuss the challenges in the integration of smart grid and Fog computing.