# Machine Learning Algorithms and Fault Detection for Improved Belief Function Based Decision Fusion in Wireless Sensor Networks

^{1}

^{2}

^{3}

^{4}

^{5}

^{*}

## Abstract

**:**

## 1. Introduction

- Offset fault: This type of fault occurs when the sensing unity has done some bad calibrations, which is the reason for the extra constant added to the expected data.
- Gain fault: This type of fault occurs when the change rate of sensed data varies from the expected data in a specific period of time.
- Stuck-at fault: This type of fault occurs when there is no variation in the sensed data, and the sensed data series is zero.
- Out of bounds: This type of fault occurs when there are normal readings and some readings are out of bounds from the normal trend.

#### 1.1. Motivation and Problem Statement

#### 1.2. Contributions

- EELM, EKNN, ESVM, and ERELM are proposed for the classification purpose.
- We have improved the existing belief function-based decision fusion approach.
- To analyze the DA of enhanced classifiers, we have induced four different faults: offset fault, gain fault, stuck-at fault, and out of bounds fault) in our dataset and then used four different classifiers (EKNN, EELM, ESVM, and ERELM) to detect faults in our scenario.
- Then, we have calculated the ER for the enhanced classifiers.
- Then, we calculated the TPR for each enhanced classifier to analyze how classifiers are performing in our scenario.
- We have tested the classifiers on two datasets, the sensit vehicle dataset and on the other dataset that was prepared by inducing four different faults.

## 2. Related Work

## 3. System Model

#### 3.1. Belief Function Theory

#### 3.1.1. BBA Construction

#### 3.1.2. BBA Combination

#### 3.2. Fusion Rules

#### 3.2.1. NB

#### 3.2.2. WMV

## 4. Existing and Enhanced Classifiers

#### 4.1. KNN

#### 4.2. EKNN

- Firstly, clusters of training samples using the clustering ensemble method are made. In the process of the ensemble method, the k-means algorithm is applied, which uses the random initialization to cluster the samples. Using simple majority voting, the class label of the cluster center is identified and combined. For the k-means algorithm, we have used the Euclidean distance function.
- Then, a new test sample is allocated for the nearest cluster label. Using the evaluation set, the quality of the obtained clusters is analyzed. The label is assigned to the sample using the determined nearest cluster.
- Clustering and evaluation steps are repeated several times. The process is repeated many times until satisfactory good cluster centers are obtained.
- Finally, using the simple majority voting, we attained the group of the best cluster centers obtained from the clustering technique as our final classifier.

Algorithm 1 Pseudocode of EKNN | |

1: | Input: Original dataset |

2: | Output: Classification and DA |

3: | Load the training and test data |

4: | for i = 1 to max iteration do |

5: | Partitioning the training set into k clusters |

6: | Determining the class label of cluster centers using majority voting |

7: | Evaluating the quality of cluster centers |

8: | end for |

9: | Designing the final classifier |

10: | end |

#### ELM

#### 4.3. EELM

- Firstly, we divided the dataset into five subsets. In every subset, there were four training sets and one testing set.
- We preferred to encode the solution with dimensions of (L$\phantom{\rule{0.166667em}{0ex}}\times \phantom{\rule{0.166667em}{0ex}}($n$\phantom{\rule{0.166667em}{0ex}}+\phantom{\rule{0.166667em}{0ex}}1\left)\right)$. The input weight values were first (L$\phantom{\rule{0.166667em}{0ex}}\times \phantom{\rule{0.166667em}{0ex}}$n) dimensions. Bias values were the other L dimensions.
- Then, we initialized the parameters with random values including hidden layers and hidden neurons.
- From Step 2, we obtained the weights and biases. We trained the ELM classifier using these weights and biases. We had a vector solution. Further, we had to convert it into a two-dimensional matrix for the training process.
- Then, we calculated the fitness value using Equation (9).
- After the calculation of the fitness value, we increased the number of iterations from 200 to 250 and updated the features in every set.
- After increasing the number of iterations, we trained the ELM classifier again with the weights and biases obtained from Step 6. Then, we calculated the fitness value one more time using Equation (9).
- A greedy selection process was used to enhance the fitness value. Then, we calculated the probability value for every solution. A solution with the highest fitness value will be selected.
- Weights and biases obtained from Step 8 were used to train ELM and then again calculate the fitness value for each solution.
- Repeat Step 8 to enhance the current fitness value.
- If the fitness value is not enhanced, then increase the number of iterations and calculate the fitness value again for the new randomly-generated solution.
- Optimal fitness values from previous solutions were compared with the current value of fitness. If current fitness value dominated, the previous value kept that value and solution.
- If the stopping criteria have been accomplished, then stop; otherwise, move to Step 5.
- After this process, we obtained the optimal weight and bias in the range of [−1, 1] using Algorithm 2.

Algorithm 2 Pseudocode of EELM | |

1: | Input: Original dataset |

2: | Output: Classification and DA |

3: | begin: |

4: | Initialize: event target is detected by n sensors |

5: | for Each observation do |

6: | ${x}_{i}$ (1 ≤ i ≤ n) is received by sensor ${S}_{i}$ |

7: | end for |

8: | Classify the object using EELM |

9: | Randomly generate input weights ${w}_{i}$ and hidden biases ${b}_{i}$ |

10: | Calculate the fitness value till the solution is achieved |

11: | Calculate hidden layer output matrix H |

12: | Compute the output weights matrix using Equation (16) |

#### SVM

#### 4.4. ESVM

- Initialization: Firstly, for the solution space, we have defined the data error penalty coefficient C. Then, parameter g was defined, which is the the genotype data arranged in different combinations. Initially, N genotype data were produced and grouped, which were used by the GA as a starting point.
- Fitness calculation: The feature fitness was calculated to analyze whether each optimization criteria was satisfied or not. After calculating the satisfied optimized criterion and their respective solutions, then the output would be produced. If optimization criteria were not satisfied, then move towards Step 3.
- Selection: Fitness values were analyzed and the features that had the highest fitness values were moved to the next group. Features with the highest fitness values were selected.
- Crossover: Features in the group were matched, and their pairs were exchanged depending on the crossover probability conditions.
- Variation: In this step, variation probability was applied to all features to check which combination of features would provide the solution.
- Calculation analysis: In this step, we determined whether features in the newly-generated group satisfied the end condition or not. If the end condition was not satisfied, move back to Step 3. Halt the process when the end condition is fulfilled.
- Now, our SVM model was well trained, and output was the accurate classification result using Algorithm 3.

Algorithm 3 Pseudocode of ESVM | |

1: | Input: Original dataset |

2: | Output: Classification and DA |

3: | Data preprocessing |

4: | Input Parameter: Initialize parameter |

5: | A hypothesis space in the high-dimensional feature space kernel function is used |

6: | Training model |

7: | Fitness calculation |

8: | Satisfactory termination condition |

9: | Optimal parameter (C, g) |

10: | Model establishment |

11: | Performance evaluation |

12: | end |

#### 4.5. RELM

#### 4.6. ERELM

- Input layer: This is the first layer of the network, and ERELM starts its working from this layer. The input layer does not have the weighted inputs and does not take input from the previous layer. The number of neurons at the input layer is equal to the number of features in the dataset.
- Hidden layer: This is the second layer of the model. The values from the input layer are passed to the hidden layer. We have used three hidden layers, and the number of hidden neurons on the hidden layers were 10, 20, and 30 for different iterations. The number of hidden neurons were greater than the number of features. The weights and outputs of the previous layer were used for the matrix multiplication to calculate the output of the hidden layer. The bias value was also added to this output.
- Output layer: The output of hidden layer was the input of this output layer. The sigmoid function was used to convert the output into a probability score of each class.

- Mutation: In mutation, we randomly created vectors of three distinct candidate solutions from our dataset.
- Crossover: Then, we randomly swapped these vectors, and was crossover constant was chosen from range [0, 1]. Swapping will be continued until satisfactory results are obtained and the termination condition is met.
- Selection: Then, vectors are compared, and best solution is selected.
- The process is repeated until the goal is achieved.

Algorithm 4 Pseudocode of ERELM | |

1: | Input: Original dataset |

2: | Output: Classification and DA |

3: | Weights are optimized through the differential evaluation method |

4: | Assign the input weights wi and biases bi as received from RELM. |

5: | Calculate the hidden layer output matrix H, where H = hij (i = 1, …, N) and j = (1, …, K) and hij = g(wj · xi + bj ) |

6: | Calculate the output weight matrix using the Moore–Penrose generalized inverse of the H matrix through Equation (16) |

7: | Updated weights will be given as context neurons to the input and hidden layers. |

8: | end |

## 5. Faults in WSN and Performance Metrics

## 6. Simulation Setup

#### Results and Discussion

## 7. Conclusions and Future Work

## Author Contributions

## Funding

## Conflicts of Interest

## Abbreviations

BBA | Basic Belief Assignment |

CNN | Convolutional Neural Network |

ELM | Extreme Learning Machine |

EKNN | Enhanced K-Nearest-Neighbor |

EELM | Enhanced Extreme Learning Machine |

ERELM | Enhanced Recurrent Extreme Learning Machine |

ESVM | Enhanced Support Vector Machine |

EKNN | K-Nearest-Neighbor |

MLP | Multilayer Perceptron |

NB | Naive Bayes |

RELM | Recurrent Extreme Learning Machine |

SVM | Support Vector Machine |

TPR | True Positive Rate |

WMV | Weighted Majority Vote |

WSN | Wireless Sensor Network |

## References

- El Hindi, K.; AlSalamn, H.; Qassim, S.; Al Ahmadi, S. Building an Ensemble of Fine-Tuned Naive Bayesian Classifiers for Text Classification. Entropy
**2018**, 20, 857. [Google Scholar] [CrossRef] - Sriranga, N.; Nagananda, K.G.; Blum, R.S.; Saucan, A.; Varshney, P.K. Energy-efficient decision fusion for distributed detection in wireless sensor networks. In Proceedings of the 2018 21st International Conference on Information Fusion (FUSION), Cambridge, UK, 10–13 July 2018; pp. 1541–1547. [Google Scholar]
- Samet, A.; Lefevre, E.; Yahia, S.B. Belief function classification with conflict management: Application on forest image. In Proceedings of the 2014 Tenth International Conference on Signal-Image Technology and Internet-Based Systems (SITIS), Marrakech, Morocco, 23–27 November 2014; pp. 14–20. [Google Scholar]
- Zhang, Z.; Hao, Z.; Zeadally, S.; Zhang, J.; Han, B.; Chao, H.C. Multiple attributes decision fusion for wireless sensor networks based on intuitionistic fuzzy set. IEEE Access
**2017**, 5, 12798–12809. [Google Scholar] [CrossRef] - Rossi, P.S.; Ciuonzo, D.; Kansanen, K.; Ekman, T. On energy detection for MIMO decision fusion in wireless sensor networks over NLOS fading. IEEE Commun. Lett.
**2015**, 19, 303–306. [Google Scholar] [CrossRef] - Kithulgoda, C.I.; Pears, R.; Naeem, M.A. The incremental Fourier classifier: Leveraging the discrete Fourier transform for classifying high speed data streams. Expert Syst. Appl.
**2018**, 97, 1–17. [Google Scholar] [CrossRef][Green Version] - Zidi, S.; Moulahi, T.; Alaya, B. Fault detection in wireless sensor networks through SVM classifier. IEEE Sens. J.
**2017**, 18, 340–347. [Google Scholar] [CrossRef] - Zhang, W.; Zhang, Z. Belief function based decision fusion for decentralized target classification in wireless sensor networks. Sensors
**2015**, 15, 20524–20540. [Google Scholar] [CrossRef] [PubMed] - Jiao, L.; Wu, H.; Bie, R.; Umek, A.; Kos, A. Multi-sensor Golf Swing Classification Using Deep CNN. Procedia Comput. Sci.
**2018**, 129, 59–65. [Google Scholar] [CrossRef] - Gupta, S.; Mittal, M.; Padha, A. Predictive Analytics of Sensor Data Based on Supervised Machine Learning Algorithms. In Proceedings of the 2017 International Conference on Next Generation Computing and Information Systems (ICNGCIS), Jammu, India, 11–12 December 2017; pp. 171–176. [Google Scholar]
- Venkatesan, C.; Karthigaikumar, P.; Paul, A.; Satheeskumaran, S.; Kumar, R. ECG Signal Preprocessing and SVM Classifier-Based Abnormality Detection in Remote Healthcare Applications. IEEE Access
**2018**, 6, 9767–9773. [Google Scholar] [CrossRef] - Abuassba, A.O.; Zhang, D.; Luo, X.; Shaheryar, A.; Ali, H. Improving Classification Performance through an Advanced Ensemble Based Heterogeneous Extreme Learning Machines. Comput. Intell. Neurosci.
**2017**. [Google Scholar] [CrossRef] - Amzar Omairi, Z.; Ismail, H. Modeling battery state of charge in wireless sensor networks based on structured multi-layer perceptron. J. Adv. Res. Appl. Sci. Eng. Technol.
**2016**, 5, 36–45. [Google Scholar] - Pan, L.; Li, J. K-nearest neighbor based missing data estimation algorithm in wireless sensor networks. Wirel. Sens. Netw.
**2010**, 2, 115. [Google Scholar] [CrossRef] - Shah, H.A.; Koo, I. Reliable Machine Learning Based Spectrum Sensing in Cognitive Radio Networks. Wirel. Commun. Mobile Comput.
**2018**. [Google Scholar] [CrossRef] - De Cock, M.; Dowsley, R.; Horst, C.; Katti, R.; Nascimento, A.; Poon, W.S.; Truex, S. Efficient and private scoring of decision trees, support vector machines and logistic regression models based on pre-computation. IEEE Trans. Depend. Sec. Comput.
**2017**. [Google Scholar] [CrossRef] - Jan, S.U.; Lee, Y.D.; Shin, J.; Koo, I. Sensor fault classification based on support vector machine and statistical time-domain features. IEEE Access
**2017**, 5, 8682–8690. [Google Scholar] [CrossRef] - Al-Jarrah, M.; Al-Dweik, A.J.; Kalil, M.; Ikki, S.S. Decision Fusion in Distributed Cooperative Wireless Sensor Networks. IEEE Trans. Veh. Technol.
**2018**. [Google Scholar] [CrossRef] - Varshney, N.; Moore, M.; Klamer, D. Decision fusion in a wireless sensor network with a large number of sensors. In Proceedings of the 7th International Conference on Informa-tion Fusion, Stockholm, Sweden, 28 June–1 July 2004. [Google Scholar]
- Strauman, A.S.; Bianchi, F.M.; Mikalsen, K.Ø.; Kampffmeyer, M.; Soguero-Ruiz, C.; Jenssen, R. Classification of postoperative surgical site infections from blood measurements with missing data using recurrent neural networks. In Proceedings of the 2018 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI), Las Vegas, NV, USA, 4–7 March 2018; pp. 307–310. [Google Scholar]
- Lee, K.S.; Lee, S.R.; Kim, Y.; Lee, C.G. Deep learning-based real-time query processing for wireless sensor network. Int. J. Distrib. Sens. Netw.
**2017**, 13. [Google Scholar] [CrossRef] - Swain, R.R.; Khilar, P.M. Soft fault diagnosis in wireless sensor networks using pso based classification. In Proceedings of the TENCON 2017—2017 IEEE Region 10 Conference, Penang, Malaysia, 5–8 November 2017; pp. 2456–2461. [Google Scholar]
- Swain, R.R.; Khilar, P.M. Composite fault diagnosis in wireless sensor networks using neural networks. Wirel. Pers. Commun.
**2017**, 95, 2507–2548. [Google Scholar] [CrossRef] - Yuan, Y.; Li, S.; Zhang, X.; Sun, J. A Comparative Analysis of SVM, Naive Bayes and GBDT for Data Faults Detection in WSNs. In Proceedings of the 2018 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C), Lisbon, Portugal, 16–20 July 2018; pp. 394–399. [Google Scholar]
- Naz, A.; Javaid, N.; Javaid, S. Enhanced Recurrent Extreme Learning Machine Using Gray Wolf Optimization for Load Forecasting. In Proceedings of the 2018 IEEE 21st International Multi-Topic Conference (INMIC), Karachi, Pakistan, 1–2 November 2018; pp. 1–5. [Google Scholar]
- Sher, A.; Khan, A.; Javaid, N.; Ahmed, S.H.; Aalsalem, M.; Khan, W.Z. Void Hole Avoidance for Reliable Data Delivery in IoT Enabled Underwater Wireless Sensor Networks. Sensors
**2018**, 18, 3271. [Google Scholar] [CrossRef] - Khan, A.; Ahmedy, I.; Anisi, M.H.; Javaid, N.; Ali, I.; Khan, N.; Alsaqer, M.; Mahmood, H. A Localization-Free Interference and Energy Holes Minimization Routing for Underwater Wireless Sensor Networks. Sensors
**2018**, 18, 165. [Google Scholar] [CrossRef] - Javaid, N.; Hussain, S.; Ahmad, A.; Imran, M.; Khan, A.; Guizani, M. Region based cooperative routing in underwater wireless sensor networks. J. Netw. Comput. Appl.
**2017**. [Google Scholar] [CrossRef] - Javaid, N.; Shakeel, U.; Ahmad, A.; Alrajeh, N.A.; Khan, Z.A.; Guizani, N. DRADS: Depth and reliability aware delay sensitive cooperative routing for underwater wireless sensor networks. Wirel. Netw.
**2017**, 1–13. [Google Scholar] [CrossRef] - Di, M.; Joo, E.M. A survey of machine learning in wireless sensor netoworks from networking and application perspectives. In Proceedings of the 2007 6th International Conference on Information, Communications and Signal Processing, Singapore, 10–13 December 2007; pp. 1–5. [Google Scholar]
- Forster, A. Machine learning techniques applied to wireless ad-hoc networks: Guide and survey. In Proceedings of the 2007 3rd International Conference on Intelligent Sensors, Sensor Networks and Information, Melbourne, QLD, Australia, 3–6 December 2007; pp. 365–370. [Google Scholar]
- Jayaraman, P.P.; Zaslavsky, A.; Delsing, J. Intelligent processing of k-nearest neighbors queries using mobile data collectors in a location aware 3D wireless sensor network. In Proceedings of the International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, Cordoba, Spain, 1–4 June 2010; pp. 260–270. [Google Scholar]
- Kulkarni, R.V.; Forster, A.; Venayagamoorthy, G.K. Computational intelligence in wireless sensor networks: A survey. IEEE Commun. Surv. Tutor.
**2011**, 13, 68–96. [Google Scholar] [CrossRef] - Cattivelli, F.S.; Sayed, A.H. Diffusion LMS strategies for distributed estimation. IEEE Trans. Signal Process.
**2010**, 58, 1035–1048. [Google Scholar] [CrossRef] - Chen, H.; Gao, F.; Martins, M.; Huang, P.; Liang, J. Accurate and efficient node localization for mobile sensor networks. Mobile Netw. Appl.
**2013**, 18, 141–147. [Google Scholar] [CrossRef]

Classifiers | Goals | Limitations | Techniques and Methods |
---|---|---|---|

KNN, NB, neural network, decision tree [3] | Classification of forest high-resolution remote-sensing image | No association between data mining and conflict management | Generic framework and automatic method for weighting factors |

Data distribution-based intuitionistic fuzzy set construction algorithm [4] | Multi-attribute decision fusion model | Negative and positive non-ideal solutions are not elaborated | Intuitionistic fuzzy set |

Multi-input and multi-output decision fusion classifier [5] | Needed amount of energy for WSN | Performance decreases with high SNR | On-off keying scenario |

Fourier-based, stream-based classifiers [6] | Handle energy dissipation | Did not exploit parallelism | Deep tree-based structures |

CNN, SVM [9] | Golf swing classification method | No discussion about relevancy, redundancy for sensors | Architecture of vanilla convolutional neural network |

Gaussian NB [10] | Processing of sensor dataset using various machine learning algorithms | Feature selection is not used | Supervised machine learning |

PCA, ANN, SVM [11] | Classification of ECG signal | No feature selection | Adaptive filter |

ELM [12] | Enhanced ELM | Applications of the proposed methods are not discussed | Regularized-ELM, ${L}_{2}$-norm-optimized ELM and kernel-ELM |

MLP [13] | To detect state of charge | Their is no regression method used | Structured MLP architecture |

KNN [14] | To predict data accurately | Temporal correlation is lacking | Linear regression model to describe the spatial correlation |

KNN [15] | To develop a reliable spectrum sensing scheme | No feature selection or extraction | Majority voting |

SVM [16] | Fault detection | Regression is not elaborated | Decision tree classification protocol |

SVM [17] | Fault detection | Overfitting problem | K-fold cross-validation technique |

SVM, NB, and gradient lifting decision tree [18] | Fault detection | No DA | Non-linear mapping algorithm |

Fault Types | EKNN | EELM | ESVM | ERELM |
---|---|---|---|---|

Offset Fault | 4.52% | 4.79% | 4.81% | 4.86% |

Gain Fault | 4.90% | 5.23% | 5.96% | 6.13% |

Stuck-at Fault | 4.77% | 5.13% | 6.9% | 5.65% |

Out of Bounds | 4.81% | 4.91% | 5.82% | 6.77% |

Fault Types | ERELM | ESVM | EELM | EKNN |
---|---|---|---|---|

Offset Fault | 97.9% | 93.3% | 92.3% | 92.2% |

Gain Fault | 96.5% | 90.0% | 91.5% | 80.0% |

Stuck-at Fault | 97.3% | 91.6% | 95.7% | 90.5% |

Out of Bounds | 98.8% | 98.4% | 91.6% | 90.4% |

Fault Types | ERELM | ESVM | EELM | EKNN |
---|---|---|---|---|

Offset Fault | 98.8% | 94.1% | 81.2% | 82.0% |

Gain Fault | 98.1% | 93.5% | 84.5% | 81.2% |

Stuck-at Fault | 98.2% | 93.0% | 83.5% | 81.0% |

Out of Bounds | 98.0% | 93.2% | 83.6% | 81.4% |

Fault Types | ERELM | ESVM | EELM | EKNN |
---|---|---|---|---|

Offset Fault | 97.4% | 97.9% | 83.0% | 80.0% |

Gain Fault | 95.0% | 95.0% | 82.8% | 73.0% |

Stuck-at Fault | 97.1% | 98.9% | 83.3% | 80.2% |

Out of Bounds | 97.5% | 97.9% | 82.9% | 80.1% |

Fault Types | ERELM | ESVM | EELM | EKNN |
---|---|---|---|---|

Offset Fault | 97.2% | 93.2% | 81.1% | 62.8% |

Gain Fault | 97.1% | 93.2% | 81.0% | 62.4% |

Stuck-at Fault | 95.8% | 90.5% | 80.4% | 61.0% |

Out of Bounds | 97.0% | 94.8% | 70.0% | 63.2% |

Fault Types | ERELM | ESVM | EELM | EKNN |
---|---|---|---|---|

Offset Fault | 97.8% | 84.8% | 48.3% | 58.8% |

Gain Fault | 97.0% | 85.2% | 70.1% | 58.8% |

Stuck-at Fault | 97.3% | 84.5% | 48.3% | 58.9% |

Out of Bounds | 97.4% | 84.4% | 48.9% | 58.9% |

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Javaid, A.; Javaid, N.; Wadud, Z.; Saba, T.; Sheta, O.E.; Saleem, M.Q.; Alzahrani, M.E. Machine Learning Algorithms and Fault Detection for Improved Belief Function Based Decision Fusion in Wireless Sensor Networks. *Sensors* **2019**, *19*, 1334.
https://doi.org/10.3390/s19061334

**AMA Style**

Javaid A, Javaid N, Wadud Z, Saba T, Sheta OE, Saleem MQ, Alzahrani ME. Machine Learning Algorithms and Fault Detection for Improved Belief Function Based Decision Fusion in Wireless Sensor Networks. *Sensors*. 2019; 19(6):1334.
https://doi.org/10.3390/s19061334

**Chicago/Turabian Style**

Javaid, Atia, Nadeem Javaid, Zahid Wadud, Tanzila Saba, Osama E. Sheta, Muhammad Qaiser Saleem, and Mohammad Eid Alzahrani. 2019. "Machine Learning Algorithms and Fault Detection for Improved Belief Function Based Decision Fusion in Wireless Sensor Networks" *Sensors* 19, no. 6: 1334.
https://doi.org/10.3390/s19061334