Improved Mitigation of Cyber Threats in IIoT for Smart Cities: A New-Era Approach and Scheme
Abstract
:1. Introduction
- We analyzed the applicability of a methodology used in the game security industry to detect abnormal data in large smart city datasets.
- We applied the proposed approach to the cyber-attack dataset of the industrial control system (ICS).
- We presented a process for finding efficient feature sets that can be applied to the processing of large-scale data using a DEA.
2. Related Studies
2.1. Online Game Security
2.2. DEA Method
2.3. Anomaly Detection in Smart Cites
3. Proposed Method
3.1. Insight from Analysis of Game Bot Detection Methods
Feature Category | Description | Related Works |
---|---|---|
Trading network | Examining a game character’s possession event log and transaction event log to derive it as a feature | [29,30] |
Gameplay style | Investigating gameplay styles such as player information, player action, and combat ability | [31,32] |
Social network | Analysis of social network characteristics between players such as part play logs and chat logs | [33,34,35] |
Sequence analysis | Characterized by assuming that the player’s actions, such as action sequences and battle sequences, are one sequence | [36,37,38] |
Self-similarity | Analyzed based on the assumption that the bots have self-similarity, and the action frequency and action type are used as features | [13,39] |
Character movement | Identifying a character’s movement pattern and use movement speed, distance, and location | [40,41] |
Character behavior | Observering the character’s behavior and using it as a feature by applying various statistics | [17,42,43,44,45] |
Author | Feature | |||||
---|---|---|---|---|---|---|
Standard Deviation | Min | Max | Similarity | Skewness | Kurtosis | |
Chung et al. [31] | • | |||||
Kang et al. [32] | • | • | • | • | ||
Lee et al. [13] | • | • | • | |||
Thawonmas et al. [39] | • | • | ||||
Mishima et al. [40] | • | |||||
Chen et al. [41] | • | • | ||||
Yu et al. [42] | • | • | ||||
Han et al. [44] | • | • | ||||
Chen et al. [45] | • | • | • | |||
Park et al. [17] | • | • | • | • | • |
3.2. Feature Extraction
3.3. Modeling and Evaluation
3.4. Proposed Process Applying DEA Method
Algorithm 1: DEA based on the feature selection for anomaly detection in smart city |
|
4. Experiments
4.1. Dataset
4.2. Experimental Setup
4.3. Evaluation Results
4.4. Efficiency Analysis Results
5. Discussion
- The performance of the k-NN, random forest, decision tree, and LightGBM models were evaluated, and the random forest model showed the best performance.
- We tested three different conditions about the time window for extracting features. Moreover, we found that the 90 s for a time window is the best condition with random forest showed an F1-score of 0.99903.
- In addition, a feature selection method based on the DEA, which was previously used to measure the efficiency of the data, was proposed.
- The min, max, and similarity features showed the best efficiency through the experimental result. We tried to find out why these three features are better than others. We found that when criminals attack ICS, the ICS sensors showed abnormal values like over the maximum limit of sensors; under the minimum limitation of sensors, the values do not fluctuate. The feature sets we generated are well reflected this attacked situation and showed relevant results for proposed methods.
- This approach considers the performance of the existing feature selection algorithms, as well as the complexity and size of the feature set.
- In addition to preventing overfitting, which is commonly discussed in data-based abnormal behavior detection, this methodology makes it easy to apply practical feature datasets; furthermore, it guarantees a high performance.
6. Conclusions
Author Contributions
Funding
Conflicts of Interest
Abbreviations
BCC | Banker, Charnes and Cooper |
CCR | Charnes, Cooper, and Rhodes |
CPS | Cyber-Physical System |
DEA | Data Envelopment Analysis |
DMU | Decision Making Unit |
EXP | Experience point |
GBM | Gradient Boosting Machine |
GFG | Gold Farming Guide |
HIL | Hardware-in-the-Loop |
ICS | Industrial Control System |
IoT | Internet of Things |
IIoT | Industrial Internet of Things |
KNN | k-Nearest Neighbors |
MMORPG | Massively Multiplayer Online Role-Playing Game |
NIDS | Network Intrusion Detection System |
STD | Standard Deviation |
SVM | Support Vector Machine |
VRS | Variable Return to Scale |
WSN | Wireless Sensor Networks |
Appendix A
Original Data (80%) | New Data (20%) | ||||||||
---|---|---|---|---|---|---|---|---|---|
Index | DMU (Feature List) | Feature Sets Size | Complexity | Performance | Efficiency | Feature Sets Size | Complexity | Performance | Efficiency |
1 | (’std’) | 1 | 0.021793127 | 0.992683502 | 0.99533754 | 1 | 0.021793127 | 0.991109837 | 0.993514733 |
2 | (’kurt’) | 1 | 0.014263153 | 0.933899788 | 0.936408587 | 1 | 0.014263153 | 0.913419913 | 0.915636294 |
3 | (’skew’) | 1 | 0.00265193 | 0.960925137 | 0.963525362 | 1 | 0.00265193 | 0.940165787 | 0.942447111 |
4 | (’max’) | 1 | 0.002334118 | 0.997300786 | 1 | 1 | 0.002334118 | 0.997579382 | 1 |
5 | (’min’) | 1 | 0.001845837 | 0.996949073 | 1 | 1 | 0.001845837 | 0.997436628 | 1 |
6 | (’similar’) | 1 | 0.141670704 | 0.997534922 | 1 | 1 | 0.141670704 | 0.997435897 | 0.999856121 |
7 | (’std’, ’kurt’) | 2 | 0.03605628 | 0.991016548 | 0.99361503 | 2 | 0.03605628 | 0.98908046 | 0.991330026 |
8 | (’std’, ’skew’) | 2 | 0.024445057 | 0.992445703 | 0.995076857 | 2 | 0.024445057 | 0.990675656 | 0.992958337 |
9 | (’std’, ’max’) | 2 | 0.024127245 | 0.997063315 | 0.999707486 | 2 | 0.024127245 | 0.997150997 | 0.999449403 |
10 | (’std’, ’min’) | 2 | 0.023638964 | 0.996002822 | 0.998645338 | 2 | 0.023638964 | 0.996725046 | 0.999023754 |
11 | (’std’, ’similar’) | 2 | 0.163463831 | 0.995765702 | 0.998107588 | 2 | 0.163463831 | 0.99557459 | 0.997563451 |
12 | (’kurt’, ’skew’) | 2 | 0.016915083 | 0.967819985 | 0.970404035 | 2 | 0.016915083 | 0.952309985 | 0.954522628 |
13 | (’kurt’, ’max’) | 2 | 0.016597271 | 0.996356799 | 0.999017866 | 2 | 0.016597271 | 0.996578272 | 0.998894623 |
14 | (’kurt’, ’min’) | 2 | 0.01610899 | 0.996121753 | 0.998783382 | 2 | 0.01610899 | 0.996151105 | 0.998467751 |
15 | (’kurt’, ’similar’) | 2 | 0.155933857 | 0.991967871 | 0.994300767 | 2 | 0.155933857 | 0.988340291 | 0.990314722 |
16 | (’skew’, ’max’) | 2 | 0.004986048 | 0.99670898 | 0.99939996 | 2 | 0.004986048 | 0.997007268 | 0.999354317 |
17 | (’skew’, ’min’) | 2 | 0.004497766 | 0.996591043 | 0.999282915 | 2 | 0.004497766 | 0.997293062 | 0.999644526 |
18 | (’skew’, ’similar’) | 2 | 0.144322634 | 0.993986558 | 0.996324261 | 2 | 0.144322634 | 0.990954774 | 0.992934378 |
19 | (’max’, ’min’) | 2 | 0.004179955 | 0.997065383 | 0.999759358 | 2 | 0.004179955 | 0.997295374 | 0.999656918 |
20 | (’max’, ’similar’) | 2 | 0.144004822 | 0.997653684 | 1 | 2 | 0.144004822 | 0.998006266 | 1 |
21 | (’min’, ’similar’) | 2 | 0.143516541 | 0.99741784 | 0.999764855 | 2 | 0.143516541 | 0.997864769 | 0.99985942 |
22 | (’std’, ’kurt’, ’skew’) | 3 | 0.03870821 | 0.991969769 | 0.994564209 | 3 | 0.03870821 | 0.987487416 | 0.989676976 |
23 | (’std’, ’kurt’, ’max’) | 3 | 0.038390398 | 0.996591844 | 0.999199142 | 3 | 0.038390398 | 0.996437224 | 0.998647232 |
24 | (’std’, ’kurt’, ’min’) | 3 | 0.037902117 | 0.995765702 | 0.998372055 | 3 | 0.037902117 | 0.996580222 | 0.998791562 |
25 | (’std’, ’kurt’, ’similar’) | 3 | 0.177726984 | 0.994701519 | 0.997040882 | 3 | 0.177726984 | 0.994280812 | 0.996267087 |
26 | (’std’, ’skew’, ’max’) | 3 | 0.026779175 | 0.99670898 | 0.999345529 | 3 | 0.026779175 | 0.996435192 | 0.998669373 |
27 | (’std’, ’skew’, ’min’) | 3 | 0.026290894 | 0.996000941 | 0.998636861 | 3 | 0.026290894 | 0.996722246 | 0.998958087 |
28 | (’std’, ’skew’, ’similar’) | 3 | 0.166115761 | 0.995055333 | 0.997395501 | 3 | 0.166115761 | 0.995289079 | 0.997277433 |
29 | (’std’, ’max’, ’min’) | 3 | 0.025973082 | 0.996358511 | 0.998996209 | 3 | 0.025973082 | 0.997438087 | 0.999676205 |
30 | (’std’, ’max’, ’similar’) | 3 | 0.165797949 | 0.99682987 | 0.999174282 | 3 | 0.165797949 | 0.997579382 | 0.999572283 |
31 | (’std’, ’min’, ’similar’) | 3 | 0.165309668 | 0.996592645 | 0.998936432 | 3 | 0.165309668 | 0.996724113 | 0.998715253 |
32 | (’kurt’, ’skew’, ’max’) | 3 | 0.019249201 | 0.996590241 | 0.99924527 | 3 | 0.019249201 | 0.996004566 | 0.998253356 |
33 | (’kurt’, ’skew’, ’min’) | 3 | 0.01876092 | 0.996472248 | 0.999128261 | 3 | 0.01876092 | 0.996004566 | 0.998254452 |
34 | (’kurt’, ’skew’, ’similar’) | 3 | 0.158585787 | 0.991608557 | 0.99394064 | 3 | 0.158585787 | 0.984833165 | 0.986800556 |
35 | (’kurt’, ’max’, ’min’) | 3 | 0.018443108 | 0.996475564 | 0.999132353 | 3 | 0.018443108 | 0.997580071 | 0.999834128 |
36 | (’kurt’, ’max’, ’similar’) | 3 | 0.158267975 | 0.996826889 | 0.999171287 | 3 | 0.158267975 | 0.997150997 | 0.999143035 |
37 | (’kurt’, ’min’, ’similar’) | 3 | 0.157779694 | 0.996593445 | 0.998937231 | 3 | 0.157779694 | 0.996724113 | 0.998715253 |
38 | (’skew’, ’max’, ’min’) | 3 | 0.006831884 | 0.997062625 | 0.999749963 | 3 | 0.006831884 | 0.997721447 | 1 |
39 | (’skew’, ’max’, ’similar’) | 3 | 0.146656752 | 0.996593445 | 0.998937231 | 3 | 0.146656752 | 0.997862334 | 0.999855821 |
40 | (’skew’, ’min’, ’similar’) | 3 | 0.14616847 | 0.996826889 | 0.999171287 | 3 | 0.14616847 | 0.997008121 | 0.998999901 |
41 | (’max’, ’min’, ’similar’) | 3 | 0.145850658 | 0.997419052 | 0.999764855 | 3 | 0.145850658 | 0.997723392 | 0.99971658 |
42 | (’std’, ’kurt’, ’skew’, ’max’) | 4 | 0.041042328 | 0.996355085 | 0.998955193 | 4 | 0.041042328 | 0.996721311 | 0.998926454 |
43 | (’std’, ’kurt’, ’skew’, ’min’) | 4 | 0.040554047 | 0.996119017 | 0.998719741 | 4 | 0.040554047 | 0.995435093 | 0.99763839 |
44 | (’std’, ’kurt’, ’skew’, ’similar’) | 4 | 0.180378914 | 0.993992225 | 0.996329919 | 4 | 0.180378914 | 0.992986976 | 0.994970721 |
45 | (’std’, ’kurt’, ’max’, ’min’) | 4 | 0.040236235 | 0.996121753 | 0.998723232 | 4 | 0.040236235 | 0.996724113 | 0.998930944 |
46 | (’std’, ’kurt’, ’max’, ’similar’) | 4 | 0.180061102 | 0.996592645 | 0.998936432 | 4 | 0.180061102 | 0.996866097 | 0.998857507 |
47 | (’std’, ’kurt’, ’min’, ’similar’) | 4 | 0.179572821 | 0.996474736 | 0.998818298 | 4 | 0.179572821 | 0.997008121 | 0.998999901 |
48 | (’std’, ’skew’, ’max’, ’min’) | 4 | 0.028625011 | 0.996119017 | 0.998749466 | 4 | 0.028625011 | 0.997150997 | 0.999382881 |
49 | (’std’, ’skew’, ’max’, ’similar’) | 4 | 0.168449879 | 0.996475564 | 0.998819096 | 4 | 0.168449879 | 0.997007268 | 0.998999003 |
50 | (’std’, ’skew’, ’min’, ’similar’) | 4 | 0.167961597 | 0.996239718 | 0.998582712 | 4 | 0.167961597 | 0.996865204 | 0.998856609 |
51 | (’std’, ’max’, ’min’, ’similar’) | 4 | 0.167643785 | 0.996829125 | 0.999173484 | 4 | 0.167643785 | 0.99715343 | 0.999145431 |
52 | (’kurt’, ’skew’, ’max’, ’min’) | 4 | 0.021095037 | 0.99670898 | 0.99935981 | 4 | 0.021095037 | 0.997150997 | 0.999398562 |
53 | (’kurt’, ’skew’, ’max’, ’similar’) | 4 | 0.160919905 | 0.996708206 | 0.999052299 | 4 | 0.160919905 | 0.996578272 | 0.99856915 |
54 | (’kurt’, ’skew’, ’min’, ’similar’) | 4 | 0.160431623 | 0.996355942 | 0.998699194 | 4 | 0.160431623 | 0.997435167 | 0.999427728 |
55 | (’kurt’, ’max’, ’min’, ’similar’) | 4 | 0.160113811 | 0.996710526 | 0.999054595 | 4 | 0.160113811 | 0.997580071 | 0.999572982 |
56 | (’skew’, ’max’, ’min’, ’similar’) | 4 | 0.148502588 | 0.996946206 | 0.999290903 | 4 | 0.148502588 | 0.997864769 | 0.99985822 |
57 | (’std’, ’kurt’, ’skew’, ’max’, ’min’) | 5 | 0.042888165 | 0.996002822 | 0.99859737 | 5 | 0.042888165 | 0.996722246 | 0.99892356 |
58 | (’std’, ’kurt’, ’skew’, ’max’, ’similar’) | 5 | 0.182713032 | 0.996473078 | 0.998816602 | 5 | 0.182713032 | 0.996436208 | 0.998426779 |
59 | (’std’, ’kurt’, ’skew’, ’min’, ’similar’) | 5 | 0.182224751 | 0.996237065 | 0.998580019 | 5 | 0.182224751 | 0.996580222 | 0.998571145 |
60 | (’std’, ’kurt’, ’max’, ’min’, ’similar’) | 5 | 0.181906939 | 0.99682838 | 0.999172785 | 5 | 0.181906939 | 0.996724113 | 0.998715253 |
61 | (’std’, ’skew’, ’max’, ’min’, ’similar’) | 5 | 0.170295715 | 0.996592645 | 0.998936432 | 5 | 0.170295715 | 0.997151809 | 0.999143834 |
62 | (’kurt’, ’skew’, ’max’, ’min’, ’similar’) | 5 | 0.162765741 | 0.996827635 | 0.999171986 | 5 | 0.162765741 | 0.997864161 | 0.99985762 |
63 | (’std’, ’kurt’, ’skew’, ’max’, ’min’, ’similar’) | 6 | 0.184558868 | 0.996473907 | 0.9988174 | 6 | 0.184558868 | 0.997294603 | 0.999286909 |
References
- Neirotti, P.; De Marco, A.; Cagliano, A.C.; Mangano, G.; Scorrano, F. Current trends in Smart City initiatives: Some stylised facts. Cities 2014, 38, 25–36. [Google Scholar] [CrossRef] [Green Version]
- Nam, T.; Pardo, T.A. Conceptualizing smart city with dimensions of technology, people, and institutions. In Proceedings of the 12th Annual International Digital Government Research Conference: Digital Government Innovation in Challenging Times, College Park, MD, USA, 12–15 June 2011; pp. 282–291. [Google Scholar]
- Washburn, D.; Sindhu, U.; Balaouras, S.; Dines, R.A.; Hayes, N.; Nelson, L.E. Helping CIOs understand “smart city” initiatives. Growth 2009, 17, 1–17. [Google Scholar]
- Chourabi, H.; Nam, T.; Walker, S.; Gil-Garcia, J.R.; Mellouli, S.; Nahon, K.; Pardo, T.A.; Scholl, H.J. Understanding smart cities: An integrative framework. In Proceedings of the 2012 45th Hawaii International Conference on System Sciences, Maui, HI, USA, 4–7 January 2012; pp. 2289–2297. [Google Scholar]
- Khatoun, R.; Zeadally, S. Cybersecurity and privacy solutions in smart cities. IEEE Commun. Mag. 2017, 55, 51–59. [Google Scholar] [CrossRef]
- Liu, X.; Nielsen, P.S. Scalable prediction-based online anomaly detection for smart meter data. Inf. Syst. 2018, 77, 34–47. [Google Scholar] [CrossRef]
- U.S. Cybersecurity Firm FireEye Discloses Breach, Theft of Hacking Tools. Available online: https://uk.reuters.com/article/fireeye-cyber/u-s-cybersecurity-firm-fireeye-discloses-breach-theft-of-hacking-tools-idUKKBN28I34H (accessed on 23 December 2020).
- Difallah, D.E.; Cudre-Mauroux, P.; McKenna, S.A. Scalable anomaly detection for smart city infrastructure networks. IEEE Internet Comput. 2013, 17, 39–47. [Google Scholar] [CrossRef]
- Alrashdi, I.; Alqazzaz, A.; Aloufi, E.; Alharthi, R.; Zohdy, M.; Ming, H. Ad-iot: Anomaly detection of iot cyberattacks in smart city using machine learning. In Proceedings of the 2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, USA, 7–9 January 2019; pp. 305–310. [Google Scholar]
- Gaur, A.; Scotney, B.; Parr, G.; McClean, S. Smart city architecture and its applications based on IoT. Procedia Comput. Sci. 2015, 52, 1089–1094. [Google Scholar] [CrossRef]
- Falco, G.; Viswanathan, A.; Caldera, C.; Shrobe, H. A master attack methodology for an AI-based automated attack planner for smart cities. IEEE Access 2018, 6, 48360–48373. [Google Scholar] [CrossRef]
- Liagkou, V.; Kavvadas, V.; Chronopoulos, S.K.; Tafiadis, D.; Christofilakis, V.; Peppas, K.P. Attack detection for healthcare monitoring systems using mechanical learning ual private networks over optical transport layer architecture. Computation 2019, 7, 24. [Google Scholar] [CrossRef] [Green Version]
- Lee, E.; Woo, J.; Kim, H.; Mohaisen, A.; Kim, H.K. You are a Game Bot!: Uncovering Game Bots in MMORPGs via Self-similarity in the Wild. In Proceedings of the Ndss Symposium, San Diego, CA, USA, 21–24 February 2016. [Google Scholar]
- Online Games-Worldwide. Available online: https://www.statista.com/outlook/212/100/online-games/worldwide (accessed on 27 January 2021).
- Si-young, O. Steam Co-Existence Surpassed 25 Million ’New Record’. 2021. Available online: http://it.chosun.com/site/data/html_dir/2021/01/04/2021010400549.html (accessed on 27 January 2021).
- Kim, Y.H.; Yang, S.I.; Kim, H.K. Correlation Analysis between Game Bots and Churn using Access Record. J. Korea Game Soc. 2018, 18, 47–58. [Google Scholar]
- Park, S.; Lee, K. The Gravy Value: A Set of Features for Pinpointing BOT Detection Method. In Proceedings of the International Conference on Information Security Applications, Jeju Island, Korea, 26–28 August 2020; Springer: Berlin/Heidelberg, Germany, 2020; pp. 142–153. [Google Scholar]
- Woo, J.; Kim, H.K. Survey and research direction on online game security. In Proceedings of the Workshop at SIGGRAPH Asia, Singapore, 26–27 November 2012; pp. 19–25. [Google Scholar]
- Melnikov, N.; Schönwälder, J. Cybermetrics: User identification through network flow analysis. In Proceedings of the IFIP International Conference on Autonomous Infrastructure, Management and Security, Zurich, Switzerland, 23–25 June 2010; Springer: Berlin/Heidelberg, Germany, 2010; pp. 167–170. [Google Scholar]
- Charnes, A.; Cooper, W.W.; Rhodes, E. Measuring the efficiency of decision making units. Eur. J. Oper. Res. 1978, 2, 429–444. [Google Scholar] [CrossRef]
- Fitzsimmons, J.A.; Fitzsimmons, M.J. Service Management for Competitive Advantage; McGraw-Hill: New York, NY, USA, 1994. [Google Scholar]
- Banker, R.D.; Charnes, A.; Cooper, W.W. Some models for estimating technical and scale inefficiencies in data envelopment analysis. Manag. Sci. 1984, 30, 1078–1092. [Google Scholar] [CrossRef] [Green Version]
- Curi, C.; Gitto, S.; Mancuso, P. New evidence on the efficiency of Italian airports: A bootstrapped DEA analysis. Socio-Econ. Plan. Sci. 2011, 45, 84–93. [Google Scholar] [CrossRef]
- Cooper, W.W.; Seiford, L.M.; Zhu, J. Handbook on Data Envelopment Analysis; Kluwer Academic: Boston, MA, USA, 2004. [Google Scholar]
- Garcia-Font, V.; Garrigues, C.; Rifà-Pous, H. Difficulties and challenges of anomaly detection in smart cities: A laboratory analysis. Sensors 2018, 18, 3198. [Google Scholar] [CrossRef] [Green Version]
- Garcia-Font, V.; Garrigues, C.; Rifà-Pous, H. A comparative study of anomaly detection techniques for smart city wireless sensor networks. Sensors 2016, 16, 868. [Google Scholar] [CrossRef] [Green Version]
- Bawaneh, M.; Simon, V. Anomaly detection in smart city traffic based on time series analysis. In Proceedings of the 2019 International Conference on Software, Telecommunications and Computer Networks (SoftCOM), Split, Croatia, 19–21 September 2019; pp. 1–6. [Google Scholar]
- Korzhuk, V.; Groznykh, A.; Menshikov, A.; Strecker, M. Identification of attacks against wireless sensor networks based on behaviour analysis. J. Wirel. Mob. Netw. Ubiquitous Comput. Dependable Appl. 2019, 10, 1–21. [Google Scholar]
- Fujita, A.; Itsuki, H.; Matsubara, H. Detecting real money traders in MMORPG by using trading network. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Stanford, CA, USA, 10–14 October 2011. [Google Scholar]
- Kwon, H.; Mohaisen, A.; Woo, J.; Kim, Y.; Lee, E.; Kim, H.K. Crime scene reconstruction: Online gold farming network analysis. IEEE Trans. Inf. Forensics Secur. 2016, 12, 544–556. [Google Scholar] [CrossRef]
- Chung, Y.; Park, C.; Kim, N.; Cho, H.; Yoon, T.; Lee, H.; Lee, J.H. Game bot detection approach based on behavior analysis and consideration of various play styles. ETRI J. 2013, 35, 1058–1067. [Google Scholar] [CrossRef] [Green Version]
- Kang, A.R.; Jeong, S.H.; Mohaisen, A.; Kim, H.K. Multimodal game bot detection using user behavioral characteristics. SpringerPlus 2016, 5, 523. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Oh, J.; Borbora, Z.H.; Sharma, D.; Srivastava, J. Bot detection based on social interactions in MMORPGs. In Proceedings of the 2013 International Conference on Social Computing, Alexandria, VA, USA, 8–14 September 2013; pp. 536–543. [Google Scholar]
- Kang, A.R.; Kim, H.K.; Woo, J. Chatting pattern based game BOT detection: Do they talk like us? KSII Trans. Internet Inf. Syst. 2012, 6, 2866–2879. [Google Scholar] [CrossRef]
- Kang, A.R.; Woo, J.; Park, J.; Kim, H.K. Online game bot detection based on party-play log analysis. Comput. Math. Appl. 2013, 65, 1384–1395. [Google Scholar] [CrossRef]
- Lee, J.; Lim, J.; Cho, W.; Kim, H.K. I know what the BOTs did yesterday: Full action sequence analysis using Naïve Bayesian algorithm. In Proceedings of the 2013 12th Annual Workshop on Network and Systems Support for Games (NetGames), Denver, CO, USA, 9–10 December 2013; pp. 1–2. [Google Scholar]
- Xu, J.; Luo, Y.; Tao, J.; Fan, C.; Zhao, Z.; Lu, J. NGUARD+ An Attention-based Game Bot Detection Framework via Player Behavior Sequences. ACM Trans. Knowl. Discov. Data TKDD 2020, 14, 1–24. [Google Scholar]
- Platzer, C. Sequence-based bot detection in massive multiplayer online games. In Proceedings of the 2011 8th International Conference on Information, Communications & Signal Processing, Singapore, 13–16 December 2011; pp. 1–5. [Google Scholar]
- Thawonmas, R.; Kashifuji, Y.; Chen, K.T. Detection of MMORPG bots based on behavior analysis. In Proceedings of the 2008 International Conference on Advances in Computer Entertainment Technology, Yokohama, Japan, 3–5 December 2008; pp. 91–94. [Google Scholar]
- Mishima, Y.; Fukuda, K.; Esaki, H. An analysis of players and bots behaviors in MMORPG. In Proceedings of the 2013 IEEE 27th International Conference on Advanced Information Networking and Applications (AINA), Barcelona, Spain, 25–28 March 2013; pp. 870–876. [Google Scholar]
- Chen, K.T.; Liao, A.; Pao, H.K.K.; Chu, H.H. Game bot detection based on avatar trajectory. In Proceedings of the International Conference on Entertainment Computing, Pittsburgh, PA, USA, 25–27 September2008; Springer: Berlin/Heidelberg, Germany, 2008; pp. 94–105. [Google Scholar]
- Yu, S.Y.; Hammerla, N.; Yan, J.; Andras, P. A statistical aimbot detection method for online FPS games. In Proceedings of the 2012 International Joint Conference on Neural Networks (IJCNN), Brisbane, Australia, 10–15 June 2012; pp. 1–8. [Google Scholar]
- Yu, S.Y.; Hammerla, N.; Yan, J.; Andras, P. Aimbot detection in online fps games using a heuristic method based on distribution comparison matrix. In Proceedings of the International Conference on Neural Information Processing, Doha, Qatar, 12–15 November 2012; Springer: Berlin/Heidelberg, Germany, 2012; pp. 654–661. [Google Scholar]
- Han, M.L.; Park, J.K.; Kim, H.K. Online game bot detection in fps game. In Proceedings of the 18th Asia Pacific Symposium on Intelligent and Evolutionary Systems; Springer: Berlin/Heidelberg, Germany, 2015; Volume 2, pp. 479–491. [Google Scholar]
- Chen, K.T.; Hong, L.W. User identification based on game-play activity patterns. In Proceedings of the 6th ACM SIGCOMM Workshop on Network and System Support for Games, Melbourne, Australia, 19–20 September 2007; pp. 7–12. [Google Scholar]
- Shin, H.K.; Lee, W.; Yun, J.H.; Kim, H. {HAI} 1.0: HIL-based Augmented {ICS} Security Dataset. In Proceedings of the 13th {USENIX} Workshop on Cyber Security Experimentation and Test ({CSET} 20), Boston, MA, USA, 10 August 2020. [Google Scholar]
- Raith, A.; Rouse, P.; Seiford, L.M. Benchmarking Using Data Envelopment Analysis: Application to Stores of a Post and Banking Business. In Multiple Criteria Decision Making and Aiding; Springer: Berlin/Heidelberg, Germany, 2019; pp. 1–39. [Google Scholar]
- Saito, T.; Rehmsmeier, M. The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS ONE 2015, 10, e0118432. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Tafiadis, D.; Chronopoulos, S.K.; Helidoni, M.E.; Kosma, E.I.; Voniati, L.; Papadopoulos, P.; Murry, T.; Ziavra, N.; Velegrakis, G.A. Checking for voice disorders without clinical intervention: The Greek and global VHI thresholds for voice disordered patients. Sci. Rep. 2019, 9, 1–9. [Google Scholar] [CrossRef] [PubMed]
Original Data (80%) | New Data (20%) | |||||||
---|---|---|---|---|---|---|---|---|
DMU (Feature List) | Feature Set Size | Complexity | Performance | Efficiency | Feature Set Size | Complexity | Performance | Efficiency |
(’max’) | 1 | 0.002334118 | 0.997300786 | 1 | 1 | 0.002334118 | 0.997579382 | 1 |
(’min’) | 1 | 0.001845837 | 0.996949073 | 1 | 1 | 0.001845837 | 0.997436628 | 1 |
(’similar’) | 1 | 0.141670704 | 0.997534922 | 1 | 1 | 0.141670704 | 0.997435897 | 0.999856121 |
(’max’, ’similar’) | 2 | 0.144004822 | 0.997653684 | 1 | 2 | 0.144004822 | 0.998006266 | 1 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Park, S.; Lee, K. Improved Mitigation of Cyber Threats in IIoT for Smart Cities: A New-Era Approach and Scheme. Sensors 2021, 21, 1976. https://doi.org/10.3390/s21061976
Park S, Lee K. Improved Mitigation of Cyber Threats in IIoT for Smart Cities: A New-Era Approach and Scheme. Sensors. 2021; 21(6):1976. https://doi.org/10.3390/s21061976
Chicago/Turabian StylePark, Semi, and Kyungho Lee. 2021. "Improved Mitigation of Cyber Threats in IIoT for Smart Cities: A New-Era Approach and Scheme" Sensors 21, no. 6: 1976. https://doi.org/10.3390/s21061976
APA StylePark, S., & Lee, K. (2021). Improved Mitigation of Cyber Threats in IIoT for Smart Cities: A New-Era Approach and Scheme. Sensors, 21(6), 1976. https://doi.org/10.3390/s21061976