# LSTM and Bat-Based RUSBoost Approach for Electricity Theft Detection

^{1}

^{2}

^{3}

^{4}

^{5}

^{*}

## Abstract

**:**

## 1. Introduction

- Most machine learning techniques require manual feature extraction; as a result, their performance is limited to low-dimensional data and they are not satisfactory for large time series data.
- The problem of class imbalance is a serious concern in electricity theft detection (ETD). In the literature, very little attention has been paid to solving the class imbalance problem.
- The existing machine learning algorithms—i.e., support vector machine (SVM) and logistic regression (LR)—are inefficient in ETD and have a high false positive rate (FPR).
- The state-based solution requires specific hardware devices and has a high cost of installation.
- In most cases, the available dataset has an enormous number of missing values and outliers, which may lead the to the overfitting of the classifier.
- The hyper-parameters of the algorithms are not tuned for optimal classification.

- The smart meter data collected from the State Grid Corporation of China (SGCC) [12] have missing values and outliers. In this paper, we perform data pre-processing using interpolation and normalization methods. These methods help to get the dataset on a common scale and compute the missing values.
- In order to better extract and memorize features from large time series data, we utilize the LSTM block, which efficiently extracts useful information to truly represent electricity theft cases.
- In order to tackle the imbalanced data, RUSBoost is employed to handle the class imbalance problem and performs better than existing data balancing techniques. It performs two operations: RUS first under-samples the data, then Adaboost predicts final classification. This technique improves its performance by learning from previous mistakes, which shows the effectiveness of the model.
- Along with RUSBoost, a metaheuristic method—the bat algorithm—is utilized for the efficient parameter optimization of a classifier.
- Moreover, for comparative analysis, the precision, recall, F1-score and receiver operating characteristics (ROC) curve are used to compute the accuracy of the model.

## 2. Literature Review

## 3. Proposed System Model

#### 3.1. Data Pre-Processing

#### 3.2. Feature Extraction

#### 3.3. Bat Algorithm

Algorithm 1: Bat algorithm. |

1: Initialize bat population by X_{i} (i = 1, 2, 3 … n) |

2: Initialize velocity V_{i}, loudness A_{i} and pulse rate r_{i} |

3: Define the frequency f_{i}, at position X_{i} |

4: Maximum number of iterations is s, and t is the current iteration |

5: while (t < s) |

6: Adjust the f_{i}, ${v}_{i}^{t}$ and ${x}_{i}^{t}$ to find new solutions |

7: Update the f_{i}, ${v}_{i}^{t}$ and ${x}_{i}^{t}$ as given in Equations (9), (10) and (11) |

8: if (rand > r_{i}) |

9: Choose the best solution |

10: Find the local solution among the selected best solutions |

11: end if |

12: Go for new solution |

13: if (rand < A_{i} and f_{i} < f_{∗}) |

14: Select the solution |

15: Change the loudness A_{i} and pulse rate r_{i} |

16: end if |

17: Rank the bats on the basis of minimum cost function |

18: end if |

19: Generate a final result |

#### 3.4. Classification of ETD

## 4. Simulation Results and Discussion

#### 4.1. Dataset Information

#### 4.2. Simulation Environment

#### 4.3. Performance Metrics

#### 4.3.1. F1-Score

#### 4.3.2. ROC Curve

#### 4.4. Benchmark Models and Their Configurations

#### 4.4.1. SVM Model

#### 4.4.2. LR Model

#### 4.4.3. Hybrid CNN–LSTM Model

#### 4.5. Performance of LSTM–RUSBoost Model for ETD

#### 4.6. Performance Comparison

#### 4.6.1. SVM Model Results

#### 4.6.2. LR Model Results

#### 4.6.3. Hybrid CNN-LSTM Model Results

#### 4.7. Summary of Results

## 5. Conclusions and Future Work

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Das, A.; McFarlane, A. Non-linear dynamics of electric power losses, electricity consumption, and GDP in Jamaica. Energy Econ.
**2019**, 84, 104530. [Google Scholar] [CrossRef] - Northeast Group LLC. Electricity Theft and Non-Technical Losses: Global Markets, Solutions, and Vendors. 2018. Available online: http://www.northeast-group.com (accessed on 22 February 2020).
- Hussain, Z.; Memon, S.; Shah, R.; Bhutto, Z.A.; Aljawarneh, M. Methods and Techniques of Electricity Thieving in Pakistan. J. Power Energy Eng.
**2016**, 4, 1–10. [Google Scholar] [CrossRef][Green Version] - Meira, J.A.; Glauner, P.; State, R.; Valtchev, P.; Dolberg, L.; Bettinger, F.; Duarte, D. Distilling provider-independent data for general detection of non-technical losses. In Proceedings of the 2017 IEEE Power and Energy Conference at Illinois (PECI), Champaign, IL, USA, 23–24 February 2017; pp. 1–5. [Google Scholar]
- Lewis, F.B. Costly ‘Throw-Ups’: Electricity Theft and Power Disruptions. Electr. J.
**2015**, 28, 118–135. [Google Scholar] [CrossRef] - Bashkari, S.; Sami, A.; Rastegar, M. Outage Cause Detection in Power Distribution Systems based on Data Mining. IEEE Trans. Ind. Inf.
**2020**. [Google Scholar] [CrossRef] - Messinis, G.M.; Hatziargyriou, N.D. Review of non-technical loss detection methods. Electr. Power Syst. Res.
**2018**, 158, 250–266. [Google Scholar] [CrossRef] - Gul, H.; Javaid, N.; Ullah, I.; Qamar, A.M.; Afzal, M.K.; Joshi, G.P. Detection of Non-Technical Losses using SOSTLink and Bidirectional Gated Recurrent Unit to Secure Smart Meters. Appl. Sci.
**2020**, 10, 3151. [Google Scholar] [CrossRef] - Abid, S.; Alghamdi, T.A.; Haseeb, A.; Wadud, Z.; Ahmed, A.; Javaid, N. An Economical Energy Management Strategy for Viable Microgrid Modes. Electronics
**2019**, 8, 1442. [Google Scholar] [CrossRef][Green Version] - Mujeeb, S.; Javaid, N. ESAENARX and DE-RELM: Novel Schemes for Big Data Predictive Analytics of Electricity Load and Price. Sustain. Cities Soc.
**2019**, 51, 101642. [Google Scholar] [CrossRef] - Adil, M.; Javaid, N.; Ullah, Z.; Maqsood, M.; Ali, S.; Daud, M.A. Electricity Theft Detection using Machine Learning Techniques to Secure Smart Grid. In Proceedings of the 14th International Conference on Complex, Intelligent and Software Intensive System (CISIS‘20), Lodz, Poland, 1–3 July 2020. [Google Scholar]
- State Grid Corporation of China. Available online: https://www.sgcc.com.cn (accessed on 22 February 2020).
- Jokar, P.; Arianpoo, N.; Leung, V.C. Electricity theft detection in AMI using customers’ consumption patterns. IEEE Trans. Smart Grid
**2016**, 7, 216–226. [Google Scholar] [CrossRef] - Leite, J.B.; Mantovani, J.R.S. Detecting and locating non-technical losses in modern distribution networks. IEEE Trans. Smart Grid
**2018**, 9, 1023–1032. [Google Scholar] [CrossRef][Green Version] - Wei, L.; Sundararajan, A.; Sarwat, A.I.; Biswas, S.; Ibrahim, E. A distributed intelligent framework for electricity theft detection using benford’s law and stackelberg game. In Proceedings of the 2017 Resilience Week (RWS), Wilmington, DE, USA, 18–22 September 2017; pp. 5–11. [Google Scholar]
- Jamil, A.; Alghamdi, T.A.; Khan, Z.A.; Javaid, S.; Haseeb, A.; Wadud, Z.; Javaid, N. An Innovative Home Energy Management Model with Coordination among Appliances using Game Theory. Sustainability
**2019**, 11, 6287. [Google Scholar] [CrossRef] - Micheli, G.; Soda, E.; Vespucci, M.T.; Gobbi, M.; Bertani, A. Big data analytics: An aid to detection of non-technical losses in power utilities. Comput. Manag. Sci.
**2019**, 16, 329–343. [Google Scholar] [CrossRef] - Lu, X.; Zhou, Y.; Wang, Z.; Yi, Y.; Feng, L.; Wang, F. Knowledge Embedded Semi-Supervised Deep Learning for Detecting Non-Technical Losses in the Smart Grid. Energies
**2019**, 12, 3452. [Google Scholar] [CrossRef][Green Version] - Toma, R.N.; Hasan, M.N.; Nahid, A.A.; Li, B. Electricity Theft Detection to Reduce Non-Technical Loss using Support Vector Machine in Smart Grid. In Proceedings of the 2019 1st International Conference on Advances in Science, Engineering and Robotics Technology (ICASERT), Dhaka, Bangladesh, 3–5 May 2019; pp. 1–6. [Google Scholar]
- Buzau, M.M.; Tejedor-Aguilera, J.; Cruz-Romero, P.; Gómez-Expósito, A. Detection of non-technical losses using smart meter data and supervised learning. IEEE Trans. Smart Grid
**2018**, 10, 2661–2670. [Google Scholar] [CrossRef] - Avila, N.F.; Figueroa, G.; Chu, C.C. NTL detection in electric distribution systems using the maximal overlap discrete wavelet-packet transform and random under sampling boosting. IEEE Trans. Power Syst.
**2018**, 33, 7171–7180. [Google Scholar] [CrossRef] - Jindal, A.; Dua, A.; Kaur, K.; Singh, M.; Kumar, N.; Mishra, S. Decision tree and SVM-based data analytics for theft detection in smart grid. IEEE Trans. Ind. Inform.
**2016**, 12, 1005–1016. [Google Scholar] [CrossRef] - Buzau, M.M.; Tejedor-Aguilera, J.; Cruz-Romero, P.; Gómez-Expósito, A. Hybrid deep neural networks for detection of non-technical losses in electricity smart meters. IEEE Trans. Power Syst.
**2019**, 32, 1254–1263. [Google Scholar] [CrossRef] - Zheng, Z.; Yang, Y.; Niu, X.; Dai, H.N.; Zhou, Y. Wide and deep convolutional neural networks for electricity-theft detection to secure smart grids. IEEE Trans. Ind. Inform.
**2017**, 14, 1606–1615. [Google Scholar] [CrossRef] - Hasan, M.; Toma, R.N.; Nahid, A.A.; Islam, M.M.; Kim, J.M. Electricity Theft Detection in Smart Grid Systems: A CNN-LSTM Based Approach. Energies
**2019**, 12, 3310. [Google Scholar] [CrossRef][Green Version] - Li, S.; Han, Y.; Yao, X.; Yingchen, S.; Wang, J.; Zhao, Q. Electricity Theft Detection in Power Grids with Deep Learning and Random Forests. J. Electr. Comput. Eng.
**2019**, 2019, 4136874. [Google Scholar] [CrossRef] - Fan, C.; Xiao, F.; Zhao, Y.; Wang, J. Analytical investigation of autoencoder-based methods for unsupervised anomaly detection in building energy data. Appl. Energy
**2018**, 211, 1123–1135. [Google Scholar] [CrossRef] - Ding, N.; Ma, H.; Gao, H.; Ma, Y.; Tan, G. Real-time anomaly detection based on long short-Term memory and Gaussian Mixture Model. Comput. Electr. Eng.
**2019**, 79, 106458. [Google Scholar] [CrossRef] - Drmac, Z.; Gugercin, S. A new selection operator for the discrete empirical interpolation method—Improved a priori error bound and extensions. SIAM J. Sci. Comput.
**2016**, 38, 631–648. [Google Scholar] [CrossRef] - Xu, L.; Skoularidou, M.; Cuesta-Infante, A.; Veeramachaneni, K. Modeling tabular data using conditional gan. In Advances in Neural Information Processing Systems; NIPS, Vancouver Convention Centre: Vancouver, BC, Canada, 2019; pp. 7333–7343. [Google Scholar]
- Kong, W.; Dong, Z.Y.; Jia, Y.; Hill, D.J.; Xu, Y.; Zhang, Y. Short-term residential load forecasting based on LSTM recurrent neural network. IEEE Trans. Smart Grid
**2017**, 10, 841–851. [Google Scholar] [CrossRef] - Khalid, R.; Javaid, N. A Survey on Hyperparameters Optimization Algorithms of Forecasting Models in Smart Grid. Sustain. Cities Soc.
**2020**, 61, 102275. [Google Scholar] [CrossRef] - Wulandhari, L.A.; Komsiyah, S.; Wicaksono, W. Bat algorithm implementation on economic dispatch optimization problem. Procedia Comput. Sci.
**2018**, 135, 275–282. [Google Scholar] [CrossRef] - Yang, X.S. A new metaheuristic bat-inspired algorithm. In Nature Inspired Cooperative Strategies for Optimization (NICSO 2010); Springer: Berlin/Heidelberg, Germany, 2010; pp. 65–74. [Google Scholar]
- Seiffert, C.; Khoshgoftaar, T.M.; Van Hulse, J.; Napolitano, A. RUSBoost: A hybrid approach to alleviating class imbalance. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum.
**2009**, 40, 185–197. [Google Scholar] [CrossRef] - Abadi, M.; Agarwal, A.; Barham, P.; Brevdo, E.; Chen, Z.; Citro, C.; Corrado, G.S.; Davis, A.; Dean, J.; Devin, M.; et al. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv
**2016**, arXiv:1603.04467. [Google Scholar] - Hand, D.; Christen, P. A note on using the F-measure for evaluating record linkage algorithms. Stat. Comput.
**2018**, 28, 539–547. [Google Scholar] [CrossRef][Green Version] - Gu, Y.; Cheng, L.; Chang, Z. Classification of Imbalanced Data Based on MTS-CBPSO Method: A Case Study of Financial Distress Prediction. J. Inf. Process. Syst.
**2019**, 15, 682–693. [Google Scholar] - Douzas, G.; Bacao, F.; Fonseca, J.; Khudinyan, M. Imbalanced Learning in Land Cover Classification: Improving Minority Classes’ Prediction Accuracy Using the Geometric SMOTE Algorithm. Remote Sens.
**2019**, 11, 3040. [Google Scholar] [CrossRef][Green Version] - Smith, A.M.; Lampinen, J.M.; Wells, G.L.; Smalarz, L.; Mackovichova, S. Deviation from Perfect Performance measures the diagnostic utility of eyewitness lineups but partial Area Under the ROC Curve does not. J. Appl. Res. Mem. Cogn.
**2019**, 8, 50–59. [Google Scholar] [CrossRef]

**Table 1.**Mapping of problems addressed and proposed solution. RUSBoost: bat-based random under-sampling boosting; LSTM: long short term memory; ROC: receiver operating characteristics.

Limitation Number | Limitation Identified | Solution Number | Proposed Solution |
---|---|---|---|

L.1 | Imbalanced data | S.1 | RUSBoost |

L.2 | Missing values and outliers | S.2 | Interpolation and normalization |

L.3 | Overfitting | S.3 | LSTM and RUSBoost |

L.4 | High-dimensional data | S.4 | LSTM |

L.5 | Parameter optimization | S.5 | Bat optimization |

L.6 | Reliable Evaluation | S.6 | Precision, Recall, F1-Score, ROC |

Methods | Contributions | Limitations |
---|---|---|

State-based [14] | State-based solution has achieved a high detection accuracy for electricity theft | High cost of hardware installation |

Game theory [15,16] | Game theory-based solutions have a low cost for finding electricity theft | It is necessary to define the utility function for all players in a game, which is time-consuming |

Machine learning [17,18,19,20,21,22,23,24,25,26,27,28] | A data-driven approach is used to effectively detect anomalous consumption behavior | Performance degrades with imbalanced data |

**Table 3.**Performance of supervised machine learning techniques in the literature. SVM: support vector machine; LR: logistic regression; MODWPT: maximum overlap packet transform; MLP: multi-layer perceptron; AUC: area under the curve; CNN: convolutional neural network; SGCC: State Grid Corporation of China; SMOTE: synthetic minority over-sampling technique; RF: random forest; GMM: Gaussian mixture model.

Techniques | Dataset | Contributions | Validation Metrics | Limitations |
---|---|---|---|---|

SVM, DT and LR [19,20] | Smart meter data | To detect the malicious consumers that intentionally steal electricity | Accuracy | No reliable performance metric is used |

MODWPT, RUSBoost [21] | Honduras | Achieved better performance when detecting NTL | MCC, F1-score | No parameter tuning |

LSTM, MLP [23] | Endesa | Integrate auxiliary information and sequential data effectively to detect electricity theft | ROC, PR AUC | Data imbalance |

Wide and deep CNN [24] | SGCC | Capture electricity theft by extracting local and global features from data | AUC, MAP | Data imbalance |

SMOTE, CNN and LSTM [25] | SGCC | Improved performance at detecting fraudulent customers | F1-score, MCC | Overfitting |

CNN, RF and SMOTE [26] | EISA | Local optima are avoided by using RF in the final layer of CNN | F1-score | High execution time |

Auto-encoder [27] | 2015 data of Hong Kong | Auto-encoder improves anomaly detection for commercial buildings | Accuracy | Overfitting |

LSTM, GMM [28] | Numenta Anomaly Benchmark (NAB) | The internal architecture of LSTM is improved, which enhances the performance compared to the traditional LSTM | F1-score | Not robust |

Hyper-Parameters | Values |
---|---|

Batch size | 50 |

Drop out | 0.2 |

Optimizer | Adam |

Epochs | 20 |

UNITS | 50 |

Description | Values |
---|---|

Duration of data collection | 2014–2016 |

Number of fraudulent customers | 1592 |

Number of honest customers | 8560 |

Total customers | 10,152 |

Ground truth | 9% |

Hyper-Parameters | Range of Values | Values |
---|---|---|

$\gamma $ | 2,5,8 | 2 |

C | 0.001, 0.01, 0.1 | 0.1 |

Hyper-Parameters | Range of Values | Values |
---|---|---|

C | 0.001, 0.01, 0.1 | 0.1 |

R | l1 norm, l2 norm | l2 norm |

Hyper-Parameters | LSTM Values | CNN Values |
---|---|---|

Batch size | 50 | 130 |

Dropout | 0.2 | 0.01 |

Optimizer | Adam | Adam |

Epochs | 20 | 40 |

**Table 9.**Confusion matrix values of the proposed model. TN: true negative; FP: false positive; FN: false negative; TP: true positive.

Confusion Matrix | Predicted No | Predicted Yes |
---|---|---|

Actual No | TN = 496 | FP = 117 |

Actual Yes | FN = 92 | TP = 535 |

Limitation Number | Limitation Identified | Proposed Solution | Validation Results |
---|---|---|---|

L.1 | Imbalanced data | S.1 | RUSBoost classifier effectively |

handles the imbalanced data as | |||

shown in Figure 6 | |||

L.2 | Missing values and outliers | S.2 | No direct validation |

L.3 | Overfitting | S.3 | Our proposed LSTM and bat- |

based RUSBoost approach obtain | |||

generalized performance, as shown | |||

in Figure 8 | |||

L.4 | High-dimensional data | S.4 | No direct validation |

L.5 | Parameter optimization | S.5 | The bat algorithm enhances the |

performance of RUSBoost | |||

as shown in Figure 7 | |||

L.6 | Reliable evaluation | S.6 | We obtain a reliable |

evaluation of our model | |||

as indicated in Figure 7 and Figure 8 |

Confusion Matrix | Predicted No | Predicted Yes |
---|---|---|

Actual No | TN = 97 | FP = 516 |

Actual Yes | FN = 8 | TP = 619 |

Confusion Matrix | Predicted No | Predicted Yes |
---|---|---|

Actual No | TN 518 | FP 95 |

Actual Yes | FN 237 | TP 390 |

Models | Accuracy | Precision | Recall | F1-Score | ROC |
---|---|---|---|---|---|

CNN–LSTM | 0.742 | 0.725 | 0.851 | 0.779 | 0.817 |

SVM | 0.577 | 0.545 | 0.70 | 0.702 | 0.572 |

LR | 0.732 | 0.804 | 0.622 | 0.701 | 0.645 |

Proposed Model | 0.879 | 0.889 | 0.9109 | 0.961 | 0.879 |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Adil, M.; Javaid, N.; Qasim, U.; Ullah, I.; Shafiq, M.; Choi, J.-G.
LSTM and Bat-Based RUSBoost Approach for Electricity Theft Detection. *Appl. Sci.* **2020**, *10*, 4378.
https://doi.org/10.3390/app10124378

**AMA Style**

Adil M, Javaid N, Qasim U, Ullah I, Shafiq M, Choi J-G.
LSTM and Bat-Based RUSBoost Approach for Electricity Theft Detection. *Applied Sciences*. 2020; 10(12):4378.
https://doi.org/10.3390/app10124378

**Chicago/Turabian Style**

Adil, Muhammad, Nadeem Javaid, Umar Qasim, Ibrar Ullah, Muhammad Shafiq, and Jin-Ghoo Choi.
2020. "LSTM and Bat-Based RUSBoost Approach for Electricity Theft Detection" *Applied Sciences* 10, no. 12: 4378.
https://doi.org/10.3390/app10124378