Machine Learning in Wireless Sensor Networks for Smart Cities: A Survey

Abstract

Artificial intelligence (AI) and machine learning (ML) techniques have huge potential to efficiently manage the automated operation of the internet of things (IoT) nodes deployed in smart cities. In smart cities, the major IoT applications are smart traffic monitoring, smart waste management, smart buildings and patient healthcare monitoring. The small size IoT nodes based on low power Bluetooth (IEEE 802.15.1) standard and wireless sensor networks (WSN) (IEEE 802.15.4) standard are generally used for transmission of data to a remote location using gateways. The WSN based IoT (WSN-IoT) design problems include network coverage and connectivity issues, energy consumption, bandwidth requirement, network lifetime maximization, communication protocols and state of the art infrastructure. In this paper, the authors propose machine learning methods as an optimization tool for regular WSN-IoT nodes deployed in smart city applications. As per the author’s knowledge, this is the first in-depth literature survey of all ML techniques in the field of low power consumption WSN-IoT for smart cities. The results of this unique survey article show that the supervised learning algorithms have been most widely used (61%) as compared to reinforcement learning (27%) and unsupervised learning (12%) for smart city applications.

1. Introduction

A smart city is an urban area that uses remote sensors and the Internet of Things (IoT) enabling technologies to collect data from different locations and uses to enhance the quality of life of the people. The low power, low data rate wireless sensor networks (WSN) are used for monitoring and control applications in smart cities. The WSN nodes are used as the underlying technology infrastructure in the IoT. In the IoT, the “things” refer to the tiny embedded physical sensing devices (i.e., WSN nodes) connected to the internet to perform a specific application. Currently, a new revolutionary technique known as artificial intelligence (AI) and machine learning (ML) is evolving as the future of fully automated IoT applications. Machine learning is a part of AI, in which, the computer algorithms learn by themselves by improving from past experiences. A detailed survey of ML algorithms was performed in [1] until the year 2013. As the ML and IoT, technologies are emerging rapidly, therefore, the authors extend their survey work also. The IoT applications in smart cities are smart traffic monitoring [2], smart grids [3], smart waste management [4], smart agriculture [5], smart medical healthcare [6], etc.

Table 1. List of abbreviations in alphabetical order.

Acronym	Description
5G	5th Generation Cellular Networks
6G	6th Generation Cellular Networks
AMQP	Advanced Message Queuing Protocol
ANN	Artificial Neural networks
BLE	Bluetooth Low Energy
CoAP	Constrained Application Protocol
DDS	Data Distribution Service
DL	Deep Learning
DT	Decision Tree
IoT	Internet of Things
k-NN	K Nearest Neighborhood
LDA	Linear Discriminant Analysis
LoRaWAN	Long Ranged Wide Area Network
LTE	Long Term Evolution
MAC	Medium Access Control
MLMDP	Machine LearningMarkov Decision Process
MLP	Multi-Layer Perceptron
MQTT	Message Query Telemetry Transport
NB-IoT	Narrowband IoT
PCA	Principle Component Analysis
QoS	Quality of Service
RL	Reinforcement Learning
RFID	Radio frequency Identification
SARSA	State-Action-Reward-State-Acton
SVM	Support Vector Machines
TCP	Transmission Control Protocol
UDP	User Datagram protocol
WSN-IoT	WSN based IoT
WSN	Wireless Sensor Networks

provides a full form of all important abbreviations used in this paper.

The major problems in WSN based IoT (WSN-IoT) are fully autonomous operation, maximum network lifetime, energy efficiency, quality of service (QoS), cross-layer optimization, high bandwidth requirement, sensor data analysis, cloud computing, communication protocol design, etc. Currently, the industrial IoT (IIoT) or industry 4.0 is the biggest revolution for smart industries, smart manufacturing sector, automobile sector, smart cities and medical healthcare sector. Worldwide, various major companies like Microsoft, Google and Amazon are working on the development of AI and ML-based algorithms in advanced IoT applications for smart cities.

Machine learning can be applied in WSN-IoT for dynamic updating of routing tables in WSNs, node localization in mobile WSN-IoT nodes, identification and separation of faulty nodes for network optimization and prediction of the amount of energy harvesting in energy harvesting WSN (EH-WSN). Through this paper, the authors have tried to answer the following research questions: Why machine learning methods are used in WSN-IoT? What is its superiority of using ML over traditional optimization methods in WSN-IoT? Why is the smart city a typical use case of IoT applications?

IoT offers new opportunities for smart cities to use data to manage traffic, reduce pollution and make better use of infrastructure. The following are the advantages of using machine learning in traditional WSN-IoT:

• WSNs are generally deployed in a dynamically changing environment. Therefore, self-adaption to the new environment is expected from a fully automated IoT scenario.
• Unknown parameter monitoring requires automatic adjustment of network topology and configurations, e.g., temperature measurement in a glacier or volcano monitoring.
• Lack of accurate mathematical models of the unknown parameters in WSN-IoT.
• WSN-IoT deals with a large amount of sensor data, therefore the correlation between different data set may be of critical concern.
• Integration of WSN in IoT using cloud-based services for better monitoring and control.
• Future predictions and possible actions in WSN-IoT.
• The IoT generates a large amount of data from millions of sensor nodes. Machine learning is powered by data and generates useful information from previous data. Machine learning uses past IoT data to identify hidden patterns and builds models that help predict future behavior and events.

As WSN-IoT are resource-limited (finite bandwidth and power availability) therefore, there are some limitations for running ML-based inferences on IoT nodes also such as:

• A large number of computations are required to process the more amount of data, hence computation complexity increases.
• Additional power consumption.
• Training of WSN-IoT nodes for various ML algorithms requires complex operations and multi-domain skilled programmers.

The following are the contributions of this survey article in the field of WSN-IoT:

• In this paper, ML techniques are proposed as an optimized solution for traditional WSN-IoT problems in smart cities.
• Design guidelines of the WSN-IoT framework using AI and ML have been proposed.
• An in-depth literature survey of WSN-IoT in smart cities is presented in detail for ML engineers and data scientists.

This paper is organized as follows: Section 2 provides operation of WSN-IoT in smart cities, Section 3 provides machine learning for WSN-IoT, Section 4 provides open research issues in WSN-IoT, which can be solved by machine learning techniques, Section 5 provides a literature survey of machine learning in WSN, Section 6 provides a summary of ML techniques in WSN-IoT, Section 7 provides the survey report and, finally, Section 8 provides the conclusion and future work.

2. Wireless Sensor Networks Based Internet of Things (WSN-IoT)

The operation of WSN-IoT in a smart city is shown in Figure 1. Here, the WSN nodes are deployed in smart city applications such as smart traffic monitoring, smart grids in buildings, remote health care monitoring, smart agriculture and industrial applications.

The function of an IoT-WSN node deployed in a smart city is to continuously monitor and control any physical quantity like temperature, humidity, pressure, acceleration, etc. The main function of these sensor nodes is to sense the data and send it to the main WSN-IoT gateway node. From the gateway node, the data is sent to the cloud server. At the IoT cloud, cloud computing takes place. The IoT cloud is directly connected to remote servers, user mobile phones, computers, mobile phone towers, etc. The IoT and machine learning tasks require a large amount of data processing and memory requirements. Therefore, the IoT cloud server is designed as a high processing, high-performance computer with huge storage capacity. However, the WSN end nodes have small computing capabilities with limited processing, small storage and finite non-rechargeable battery power supply.

The WSN-IoT end nodes based on the IEEE 802.15.4 standard have the maximum data rate of 250 kbps only. In WSN-IoT, the end nodes are powered by two AA-size batteries (1.5 volts, 1000 mAh), and the gateway is connected to the mains power supply. Furthermore, if the WSN-IoT nodes are powered by renewable energy harvesting power supplies [7], then machine learning algorithms can also be utilized to predict the future available energy in IoT-WSNs. In the battery management system, machine learning techniques can be used for tracking maximum power point technique (MPPT) algorithms [8,9]. As these WSN sensor nodes generate a lot of sensor data, therefore, machine learning algorithms can also be applied to them for data analysis, data prediction and other suitable tasks.

In smart cities, the WSN-IoT networks contain the connectivity technologies/protocols as shown in

Table 2. WSN-IoT technologies for smart cities.

S. No.	IoT Technologies	IEEE Standards	Power/Energy Consumption	Data Rate	Frequency Band	Typical Distance Range	Smart City Applications Services
1	Bluetooth Low Energy (BLE) [10]	IEEE 802.15.1 [11]	Lowest	Medium 24 Mbps	2.4 GHz	Small (<5 m)	Smart Home Automation, Smart Grids
2	Radio Frequency Identification (RFID) [12]	IEEE 802.15 [13]	Low	Small 500 kbps	915 MHz	10 m	RFID Fast-Tags, Company Gates Entry/Exit
3	ZigBee [14]	IEEE 802.15.4 [15]	Low	Small (250 kbps)	2.4 GHz	100 m	Temperature Measurement
4	Wi-Fi [16]	IEEE 802.11 [17]	High	100 Mbps	2.4 GHz	1 km	Laptop/Mobile Phone, Internet Service
5	Long Ranged Wide Area Network (LoRaWAN) [18]	IEEE 802.11 ah [19]	Low	50 kbps	868 MHz (Europe), 15 MHz (America) 923 MHz (Asia)	10 km	Connecting Low Power devices in WAN
6	Cellular Mobile Communication (4G/5G) [20]	IEEE 802.16e WiMax (4G) [19] IEEE 1914 New Radio (5G) [21,22,23]	High	4G-100 Mbps, 5G-1 Gbps	450 MHz–6 GHz	100 km	Remote monitoring and control using Smart Phones

. Table 2 shows IoT communication technologies such as Bluetooth [10], based on IEEE 802.15.1 standards [11], RFID [12], IEEE 802.16 [13], ZigBee [14] based on IEEE 802.15.4 [15], Wi-Fi [16], based on IEEE 802.11 [17], LoRa-WAN [18] based on IEEE 802.11 series [19], 4G/5G cellular networks [20], based on WiMAX standard IEEE 802.16e [21] and 5G based on IEEE 1941 [22,23], respectively.

At the WSN-IoT node device level, the machine learning algorithms can be applied from the cloud for its autonomous operation. The IoT device sends the sensor data to the cloud server. From the IoT cloud, the user can monitor and control the application using a mobile phone, laptop or desktop PC and personal digital assistant. Currently, many popular cloud service companies provide free, but the limited amount of sensor data to be stored in their cloud storage. For example, Microsoft Azure IoT, Amazon web service (AWS), Google cloud platform, Cisco IoT cloud connect, IBM Watson IoT and Thing speak IoT by Mathworks Incorporation, USA.

Table 3. Higher-level communication protocols in WSN-IoT.

S. No.	Communication Protocols	IEEE Standard	Remarks
1.	Message Query Telemetry Transport (MQTT) protocol [24]	IEEE P1451-99 [28]	Works with TCP for Data Security, Load Balancing
2.	Advanced Message Queuing Protocol (AMQP) [25]	IEEE P1451-99 [28]	Works with TCP for Smart Electronics Gadgets, QoS
3.	Constrained Application Protocol (CoAP) [26]	IEEE P1451-99 [28]	For Cloud Services
4.	Data Distribution Service (DDS) protocol [27]	IEEE P1451-99 [28]	Works with UDP for Data Delivery, Machine to Machine Communication
5.	Smart Transducer Interface protocol [29]	IEEE P21451 [30] interoperability interfaces	Sensor Connectivity, interoperable communication

shows open system interconnect (OSI) model layer data communication network protocols of an upper higher level separately. Some very popular higher-level IoT protocols are message query telemetry transport (MQTT) protocol [24], advanced message queuing protocol (AMQP) [25], constrained application protocol (CoAP) [26] and data distribution service (DDS) protocol [27]. These IoT protocols are based on the IEEE P1451-99 [28] standard for harmonization of internet of things (IoT) devices and systems. In smart city IoT applications, these protocols are used with TCP, UDP and cloud-based services. In WSN, a smart transducer interface protocol [29] is used for sensor management. It is based on an IEEE standard called the P21451 interoperability interface standard [30].

3. Preliminaries in Machine Learning for WSN-IoT

Machine learning is the field of artificial intelligence (AI) that provides systems the ability to automatically learn and improve from previous experience without being explicitly programmed. It aims to develop new computer programs, which can access data and use it to learn for themselves. Machine learning (ML) for WSNs means learning sensor nodes and networks from their past experiences and making predictions based on them. The main application of machine learning is at the IoT cloud for data analysis, device management, network management, network security and authentication services. At the user end, intelligent control techniques can be developed, in which human interaction is not required for example driverless cars, driverless trains, etc.

From the machine learning point of view, the data flow in WSN-IoT is shown in Figure 2. Here, the following are the steps for data flow: (1) sensor data acquisition, (2) data communication between end node to the gateway node, (3) data aggregation at the gateway, (4) data ingestion, storage and retrieval at the cloud, (5) data analysis, (6) sensor nodes connectivity (7) and data security tasks. In simple words, the ML techniques can be applied in WSN-IoT for localization, coverage, connectivity problems, MAC and routing layer issues, data aggregation, fault detection, event monitoring, energy harvesting, QoS and network security issues. Figure 3 shows a general flowchart for the machine learning, machine testing and WSN action process in the WSN-IoT scenario.

3.1. Training Process

Here, first of all, data is acquired from a particular application. The features are extracted from this raw data. For example, if the data is image data then the colors, pixels, brightness and contrast of all images database are extracted. Then the features are classified according to the requirement of the machine learning process. Now some training examples are applied to the basic initial algorithms for their learning or improvement. Thus, algorithms are trained and optimized according to the data patterns.

3.2. Testing Process

Now the next step is to deploy this trained WSN in any real-life application. In real life, the unknown data is taken as input and the features are extracted from it. These extracted features are applied to an already trained algorithm. The output of the trained algorithm is classified as data predictions.

3.3. WSN Actions

Finally, based on predicted output data the necessary actions by the WSN are decided.

4. Open Research Problems in WSN-IoT Which Can Be Solved by Machine Learning Techniques

The following are the currently open research issues in WSN-IoT, which can be solved by ML techniques.

4.1. IoT Node Localization

In a WSN scenario, the current location identification of a sensor node is called node localization. In mobile WSN nodes, path planning is a very important step. Node localization is considered a classification problem because all the nodes are divided (classified) into range-based and range-free nodes. Several ML algorithms like, SVM, K-NN and RL-based techniques (Q-learning, SARSA) are used in WSN-IoT for node localization as a classification problem [31].

4.2. IoT Node Coverage and Connectivity

In a WSN scenario, the sensing coverage is the field of interest (FOI), in which at least one sensor node covers all the points. Therefore, the optimal placement of sensor nodes is a design issue. To maximize the WSN lifetime, the connectivity should be proper between the neighbor nodes [32].

4.3. Routing Layer Issues

The processing for sending the data packets from one node to another via intermediate nodes is called routing. In the routing process, long routing tables are maintained by the gateway nodes, which consists of the source and destination address of all the packets in the network. In WSN, the sensed data is sent towards the main gateway node by the end nodes. If the routing path is very long then unnecessary energy is wasted in a WSN network. Therefore, smart routing algorithms need to be designed carefully to find the optimal routes between end node and gateway nodes. Several machine learning techniques such as decision tree, random forest, ANN, SVM and Bayesian learning are used to find the optimal path in WSNs [33].

4.4. MAC Layer Issues

The MAC layer controls the medium accessing technique in WSN. The sensor MAC (SMAC) protocol is generally used in WSN. Reinforcement learning (RL) based algorithms are used for MAC protocol design in WSN. The RL-MAC techniques control the sleep, wake, transmission and reception in sensor networks [34].

4.5. Sensor Data Aggregation

In a smart city, thousands of small sensor nodes are deployed to measure the same physical quantity, e.g., temperature, humidity, light, carbon dioxide (CO₂) gases, etc. Several sensors may report the same information to the gateway. This large amount of data is difficult to handle by the gateway. Therefore, sensor data aggregation is important in WSNs for smart city applications. Data aggregation means collecting and summarizing useful information from multiple sources. In this process, the data redundancy and data accuracy are improved. Data aggregation saves the power consumption of WSN nodes and hence improves the network lifetime also. Machine learning is useful for data aggregation. The cluster aggregates the data from the cluster head and transmits it to the base station. ML techniques based on artificial neural networks (ANN) and quality (Q)-learning algorithms are useful for data aggregation tasks in WSN-IoT [35].

4.6. Event Monitoring and Target Detection

In a smart city application, the WSNs are deployed for event monitoring and target detection such as intrusion detection and traffic monitoring. In WSN, node failure, target recovery and tracking latency from sensing nodes are required. Various ML techniques like Bayesian, Q learning and genetic algorithms are used for event monitoring and target tracking in WSNs. Applying, ML techniques in WSNs can be useful to detect an event or target from the complex image sensor data [36].

4.7. Energy Harvesting

Energy harvesting is the process of extracting environmental energy from the sun, wind, tides, radio waves, etc., and converts it into the corresponding electrical energy. The broad objective of energy harvesting is to save our limited available fossil fuels (coal, oil and gases). However, in the smart city application, energy harvesting can also be used for achieving maximum network lifetime in rechargeable battery-based WSN-IoT nodes. Furthermore, the ML techniques are used in energy harvesting WSN-IoT for future available energy prediction tasks. The ML algorithms like regression technique and reinforcement learning techniques (Q-learning) are suitable for energy harvesting applications. Generally, solar energy, radio frequency (RF) waves and wind energy are used with rechargeable battery-based WSNs. Harvested energy prediction, battery power management are the tasks that can be optimized by using the ML algorithms in traditional WSN-IoT [37].

4.8. Node Query Processing

In WSNs, the end nodes, cluster heads and gateway nodes perform various types of queries such as sensor data aggregation, routing paths, synchronization and control operations, packet delivery with each other, etc. The k-nearest neighborhood (k-NN) based ML techniques are used for sensor data queries in WSN.

Table 4. WSN-IoT research issue with ML-based solution.

S. No.	WSN-IoT Research Issue	Machine Learning Technique	Solution/Remarks
1	IoT Node Localization [31]	K-NN, Reinforcement Learning (RL)	Efficient Distance Estimation, Range Estimation
2	IoT Node Coverage and Connectivity [32]	Decision trees, ANN, Evolutionary Computation	Classification of Connected and Failed Nodes in the sensor network, Identification nodes with poor and good connectivity
3	Routing Layer Issues [33]	Decision Tree, Random Forest	Prediction of optimal routing path depending upon data traffic.
4	MAC Layer Issues [34]	SVM, Decision Tree, ANN	Efficient channel assignment
5	Sensor data aggregation [35]	k-means, SVM, Reinforcement	Decide optimal cluster head in WSN nodes, Dynamic configuration of WSN nodes
6	Event Monitoring and Target Detection [36]	PCA, Deep Learning, Evolutionary Computing, Bayesian Learning	Efficient event monitoring and multiple target tracking
7	Energy Harvesting [37]	SVM, Deep Learning, Evolutionary Computing	To predict the amount of battery energy required to maximize network lifetime, Prediction of energy harvesting availability in the future.
8	Node Query processing [38]	k-NN	Node Beacon sending, Handshake for data transfer

shows some WSN-IoT research issues with ML-based solutions [38].

5. Literature Survey of Machine Learning in WSN-IoT

Our literature survey of machine learning algorithms for WSN-IoT is shown in Figure 4. Our literature survey is divided into the following categories:

• Supervised Learning,
• Unsupervised learning,
• Reinforcement learning.

5.1. Literature Survey of Supervised Machine Learning for WSN–IoT

In supervisor learning, data are labeled. In other words, we provide an input data variable (x) to the system. The system predicts output data (y) depending upon the type of input and system function. The objective of supervised learning is to approximate the mapping function so that when a new unknown input data is applied then, the output (y) can be predicted [39,40].

y = f (x)

(1)

In this section, we will discuss various supervised learning algorithm, which can be applied in WSN-IoT applications. The supervised learning algorithms are used in WSN for target tracking, localization of Nodes, event monitoring, data security, fault detection, etc. There are two types of supervisor learning as regression and classification.

5.1.1. Regression

In regression, the output variable (y) has some continuous numerical value like rupees, height, weight, etc. Regression is applied to solve various issues in WSNs such as localization, connectivity problem, data aggregation and energy harvesting [41].

5.1.2. Classification

In classification, the output variable(y) is a category of objects like the type of colors (e.g., red or yellow), type of diseases (e.g., fever or fracture), etc. There are the following types of classification problems as k-NN, decision tree, ANN, Bayesian learning, etc [42].

(A). K-nearest neighborhood (k-NN) [43]: This is an instance-based supervised learning algorithm. Here, all training instances are stored in a master database. When a new instance query (x_q) arrives then this new query is compared with the stored database and classified results are derived. In the k-NN algorithm the distance between each data points is calculated by using the Euclidian distance formula

$$\begin{gathered} d\left(q,p\right)=\sqrt{{\left(q_1-p_1\right)}^2+{\left(q_2-p_2\right)}^2+\dots +{\left(q_n-p_n\right)}^2} \\ Or \\ d\left(q,p\right)=\sqrt{{\displaystyle \sum }_{i=1}^n{\left(q_i-p_i\right)}^2} \end{gathered}$$

(2)

where $q$ and $p$ are data points and $d$ is the distance between them. In [44] data streaming in IoT using the k-NN algorithm is proposed. The KNN algorithm is also used for the early detection of agriculture pests, diseases, sensor node failure and fault detection issues [45].

(B). Decision tree (DT) [46]: In the DT algorithm, the main task is to calculate the attribute of the root node from each level. This process is called attribute selection. There are two methods for attribute selection as information gain and Gini index. The average amount of information is called entropy and is given as

$$Entropy=-{\displaystyle \sum }_{i=1}^n{p}_i*log(p_i)$$

(3)

Gini index is a metric to measure how often a randomly chosen element would be incorrectly identified.

$$Gini Index=1-{\displaystyle \sum }_{i=1}^n{p}_i^2$$

(4)

where pi is the probability of occurrence of an i^th event, n is the number of training examples. It is a classification method for predicting labels of data by iterating the input data through a learning tree. During this process, the feature properties are compared relative to decision conditions to reach a specific category. In WSNs, decision tree (DT) algorithms like ID3 are used to identify the link reliability, mean time to failure (MTTF) and mean time to repair (MTTR). Figure 5 shows an example of the DT algorithm, in which the decisions are taken by a WSN node system for different conditions. Here, if the 1st condition of WSN node battery voltage is greater than 2.7 volts (i.e., V_battery > 2.7 volts) is not true, then all decision flows towards the left-hand side of the tree. On the left-hand side, the 2nd condition to be checked is supply availability. On the other hand, if the 1st condition (i.e., V_battery > 2.7 volts) is true, then all decisions flow towards the right-hand side of the tree.

(C). Artificial neural networks (ANNs) [47,48,49]: ANN is inspired by the human brain architecture learning mechanism. The basic unit of AAN is perceptron, which is equivalent to a neuron in the human brain as shown in Figure 6. Backpropagation is most the common learning algorithm in ANN. An activation function is any step function, a cosine function or a sigmoid function, etc. The error ($E$) in backpropagation is calculated until the convergence meets as

$$E={\displaystyle \sum }_{i=1}^n\frac{1}{2}{\left(T_k-O_k\right)}^2$$

(5)

where T_k is the target value, O_k is the actual output value and (n) is the number of training examples. In backpropagation algorithms, the learning is determined by the sum of weighted links between the inputs layer, middle layer and output layer.

Finally, the weights (w) are updated using the rule as:

$$w_{\left(new\right)}=w_{\left(old\right)}+\Delta w$$

(6)

(D). Support Vector Machine (SVM) [50,51]: SVM is the most popular algorithm, which can be used for both classification and regression of data. Generally, it is used for classification in the WSN-IoT scenario. In SVM, we plotted each data value as a point in an n-dimensional X-Y plane as shown in Figure 7.

The SVM algorithm is based on the primal optimization problem as

$$\begin{gathered} Minimize \frac{\left|\left|w\right|\right|}{2} \\ such that, y_i\left(w.x_i+b\right)-1\ge 0 \\ and, y_i\left(w.x_i+b\right)+1\ge 0 \\ for i=1,2,3\dots n \end{gathered}$$

(7)

where, w is the width between two support vectors planes, (x_i) is the data point value, (y_i) is the assigned label to each data point respectively, (b) is the distance of hyperplane from the origin and (n) is the total number of training examples. In WSN, the SVM technique is used for the localization of mobile sensor nodes.

(E). Bayesian Learning [52,53,54]: Bayesian learning is an important statistical technique to calculate the conditional probability of an event. Bayesian learning is based on the very popular Bayes’ theorem for conditional probability as:

$$P\left(A|B\right)=\frac{P\left(B|A\right)\times P\left(A\right)}{P\left(B\right)}$$

(8)

where, P(A) is called the probability of occurrence of event A, P(B) is called the probability of occurrence of event B, P(A|B) is called conditional probability of occurrence of event A if B is given and P(B|A) is called conditional probability of occurrence of event B if A is given earlier. In WSN, it is used for calibration, clustering and detecting outlier (i.e., outsider or different types) nodes in a group of several nodes deployment.

5.2. Literature Survey of Unsupervised Machine Learning for WSN–IoT

In unsupervised machine learning, only the input data variable (x) is known to the machine. The output data variable (y) is not known to the machine or system. The objective of unsupervised ML is to know the probability distribution model of input data [55].

Clustering Technique

In this method, the inherent groups in the data are identified, such as a particular type of customer database that purchases a specific type of goods from specified stores in smart cities. Figure 8 shows three clusters of class red, green and blue [56,57,58]. To make a cluster between data points of two vectors $X$ and $Y$ the Euclidian distance ($d$) is calculated as:

$$d\left(X,Y\right)=\sqrt{{\displaystyle \sum }_{i=1}^n{\left(X_i-Y_i\right)}^2}$$

(9)

The k-means clustering [59,60,61], PCA [62,63,64] and ANN [65,66] algorithm are most widely used for clustering in WSN-IoT.

5.3. Literature Survey of Reinforcement Learning (RL) for WSN–IoT

In the reinforcement learning algorithm, there are two main components called the agent and the environment as shown in Figure 9 [67,68]. The agent refers to the WSN-IoT node and the environment refers to the type of application deployment, e.g., temperature monitoring, pollution monitoring, smart agriculture monitoring, etc [69]. The WSN-IoT node agent performs the actions (A_t) of sensing and actuation at time state (t). The application environment changes its state (S_t) and provides a reward (R_t) to the agent as shown in Figure 9. There are many types of reinforcement learning algorithms given as (A) Q-learning [70,71], (B) SARSA [72], temporal difference (TD) [73] and deep Q learning [74]. The Q value is defined as a sum of discounted rewards starting from the state (s_k), taking action (a_k) following any policy (π) as:

$$Q^{\pi }\left(s,a\right)=E\left[{\displaystyle \sum }_{k=0}^{N-1}{\gamma }^k r\left(s_k,a_k\right)\right]$$

(10)

where, E = expectation, s = state, a = action, k = numerical integer value (0 to N-1), γ = discount rate and r = rewards value.

6. Summary of Literature Survey of ML Techniques for WSN-IoT

Finally, this section provides a summary of all machine learning (ML) techniques proposed as an optimized solution for WSN-IoT problems. In a smart city, the major challenges are as follows: smart education, smart classrooms, smart traffic monitoring, rain water harvesting, smart grids in smart buildings smart healthcare in hospitals, smart agriculture, industrial IoT (or Industry 4.0), smart waste management, smart governance, smart environment monitoring, etc. Now, we will map each problem of smart cities with the solution provided by machine learning algorithms in WSN-IoT. The WSN node localization problem is considered as a classification or multivariate regression task in the ML domain. Therefore, SVM classification [50] or SVM regression model [51] algorithms are applied as a solution for the node localization problems in WSN-IoT.

The security issues are tackled by correlation techniques and handled by using the Bayesian learning technique as [52,53,54]. Cluster head selection tasks in WSN-IoT are considered clustering tasks in the ML domain. The k-NN [59,60,61], PCA [62,63,64] and ANN [65,66] have been used for clustering. WSN node energy management is considered a prediction problem in the ML domain. The Q-Learning [71,72] has been applied to predict the energy issues. Similarly, in energy harvesting based WSN (EH-WSN) predicts future energy availability using reinforcement learning algorithms like Q-Learning [73], SARSA [74] and deep Q-learning [75] have been applied. Event monitoring and fault detection problems are considered as classification models. These are solved by SVM [76,77] and rule-based Learning [78,79,80,81] algorithms.

The routing of data packets in WSN-IoT is considered a classification problem in the domain of machine learning. The routing optimization algorithms such as genetic algorithms [82] and classification algorithms such as Markov decision process (MDP) in decision tree [83], random forest [84] and Q-learning (QELAR) [85] have been used.

At the MAC layer, the packet scheduling task is considered a regression task in the machine learning context. Therefore, SVM [86], deep neural networks [87] have been applied at the MAC layer. The QoS (latency, bandwidth and coverage) in IoT is considered a prediction problem in IoT. Therefore, Q-Learning, ANN [88] and SVM [89] have been used as the solution. Spectrum sharing in WSN-IoT is a self-learning problem, which can be solved by a deep reinforcement learning [90,91] technique. Data aggregation is a technique in which redundant information is removed before processing by the server. Data aggregation is treated as a regression problem and is solved by SVM [92] and reinforcement [93] respectively.

Table 5. Summary of literature survey of ML techniques for WSN-IoT.

S. No.	Smart City Applications	WSN-IoT Issues in Smart Cities	Addressed ML Algorithms as Solutions by Researchers Worldwide	Our Remarks
1	Smart traffic Monitoring	WSN Node Localization	SVM Classification [50] and SVM regression model [51]	Localization is considered as Classification and Multivariate Regression task in ML context.
2	Rain Water Harvesting	Security Issues in WSN	Bayesian Learning [52,53,54]	Security is dealt with Correlation, Encoding, Decoding task in ML
3	Smart Grids in Smart Buildings	Node Clustering, Cluster Head Selection, Data Pattern Analysis	k-NN [59,60,61], PCA [62,63,64] and ANN [65,66]	Classification problem
4	Smart Healthcare in Hospitals	WSN Node Energy Management	Q-Learning [71,72]	Energy Management is considered a Prediction task in ML
5	Smart Agriculture	Energy Harvesting	Q-Learning [73], SARSA [74] and Deep Q-Learning [75]	The Energy Harvesting process is considered a Prediction task in ML
6	Industrial IoT (Industry 4.0)	Event/Condition Monitoring, Object/Fault Detection,	SVM [76,77] and Rule based Learning [78,79,80,81]	Event detection is handled by Classification techniques in the ML domain
7	Smart Waste Management	Routing of data packets	Genetic algorithms [82], MDP in Decision Tree [83], Random Forest [84] and Q-Learning (QELAR) [85]	Route optimization, Routing as a Classification problem in ML.
8	Smart Governance	Scheduling and Heterogeneity at MAC Layer	SVM [86] and Deep Neural Networks [87]	Scheduling and Heterogeneity problems are Regression tasks in ML
9	Quality of Life of People	QoS (Latency, Bandwidth, Coverage, Link Quality) in IoT	Q-Learning and ANN [88], SVM [89]	QoS in IoT is solved by Prediction, Classification tasks.
10	Energy Efficient Street Lighting, Smart environment monitoring	Spectrum Sharing	Deep Reinforcement Learning [90,91]	Spectrum sharing is Self-learning, rewards system tasks in ML.
11	Pandemic medical treatment (e.g., COVID-19 or Corona Virus)	Data Aggrigation	SVM [92] and Reinforcement [93]	Data Aggregation is treated as a Regression task in the ML context.

shows the summary of the literature survey of ML techniques for WSN-IoT.

The ML and WSN-IoT pair can act as a boon for the medical healthcare sector in smart hospitals in smart cities. For example, as per a world health organization (WHO) report [94], an international pandemic called corona virus disease (COVID-19) caused the death of 716,075 people worldwide until the end of the year 2020. In smart hospitals, the advanced ML techniques with efficiently deployed WSN-IoT can be applied for the treatment of infected patients placed in quarantine. The sensors attached to biomedical instruments can send patient’s data over the internet to the doctors for medical diagnosis. Thus, doctors need not go near to the patients and hence avoid/reduce the virus spread in the smart cities and society.

7. Our Survey Report of ML Techniques in WSN-IoT

We performed this survey task by visiting various websites, journals, magazines and research papers for machine learning in WSN-IoT for smart cities. By surveying on the internet for ML techniques addressing WSN-IoT problems, we included many papers available as shown in

Table 6. WSN-IoT technology-wise research papers included in this survey.

S. No.	WSN-IoT Problem	No. of Research Papers Included
1	Routing	45
2	Switching	34
3	Data Aggregation	24
4	Energy Harvesting	12
5	Node Localization	15
6	Security	52
7	QoS	37
8	Event Detection	36
9	Fault Detection	56
10	Congestion Control	18
11	Anomaly Detection	117
12	Connectivity	27

from the year 2010 to 2021.

Table 7. ML algorithms for WSN-IoT.

S. No.	ML Algorithms for WSN-IoT	Percentage (%)
1	Reinforcement	12%
2	Q-Learning	15%
3	SVM	18%
4	ANN	14%
5	Bayesian	9%
6	Rule-based	8%
7	Decision Tree	4%
8	Random Forest	2%
9	k-NN	6%
10	k-means	8%
11	PCA	4%

shows category-wise ML algorithms addressing WSN-IoT issues with percentage contribution. Figure 10 shows the graphical representation of major ML algorithms in WSN-IoT.

Figure 11 shows a quick look and easy to understand graphical representation of major ML algorithms used in WSN-IoT.

Table 8. Conclusion of usage of ML techniques in WSN-IoT problems.

S. No.	ML Technique in WSN-IoT	Dominance
1.	Reinforcement Learning	27%
2.	Supervised Learning	61%
3.	Unsupervised Learning	12%

shows the dominance of ML techniques in WSN-IoT problems. Figure 12 shows a quick graphical representation of dominance of RL, supervised and unsupervised learning algorithms in WSN-IoT.

8. Conclusions and Future Work

In this paper, we outlined various machine learning algorithms in WSN-IoT for smart city applications. In this paper, we performed a detailed survey of ML techniques in WSN-IoT for the smart city challenges. From this survey, it was concluded that the supervised learning algorithms have been used with the highest amount, i.e., 61% as compared to RL at 27% and unsupervised at 12% only. The ML algorithms are so versatile and powerful that a single type of ML algorithm can be used for multiple tasks in WSN-IoT in smart cities. For example, the powerful SVM algorithm can be used for classification and for regression tasks in WSN-IoT. In the future, a more powerful and complex algorithm will emerge, which minimizes human intervention. The futuristic IoT-based solution for smart cities will include machine learning techniques. For example, the heart stroke rehabilitation system [95] in smart health care using LDA, MLP and SVM algorithms. Next-generation smart cities will have ultra-dense cellular IoT networks using high-performance machine learning algorithms [96,97,98,99,100].