Next Article in Journal
A Tale of Many Networks: Splitting and Merging of Chord-like Overlays in Partitioned Networks
Previous Article in Journal
An AI-Driven Framework for Integrated Security and Privacy in Internet of Things Using Quantum-Resistant Blockchain
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Enhanced Peer-to-Peer Botnet Detection Using Differential Evolution for Optimized Feature Selection

1
Biomedical Sensors & Systems Lab, University of Memphis, Memphis, TN 38152, USA
2
Department of Computer Science and Information Technology, Cotton University, Guwahati 781001, India
3
Department of Information Technology, Gauhati University, Guwahati 781014, India
4
Faculty of Computer Technology, Assam Down Town University, Guwahati 781026, India
5
Department of Information Technology, Assam Skill University, Mangaldoi 784125, India
6
Electrical and Computer Engineering Department, University of Memphis, Memphis, TN 38152, USA
*
Author to whom correspondence should be addressed.
Future Internet 2025, 17(6), 247; https://doi.org/10.3390/fi17060247
Submission received: 23 April 2025 / Revised: 23 May 2025 / Accepted: 28 May 2025 / Published: 30 May 2025

Abstract

With the growing prevalence of cybercrime, botnets have emerged as a significant threat, infiltrating an increasing number of legitimate computers annually. Challenges arising for organizations, educational institutions, and individuals as a result of botnet attacks include distributed denial of service (DDoS) attacks, phishing attacks, and extortion attacks, generation of spam, and identity theft. The stealthy nature of botnets, characterized by constant alterations in network structures, attack methodologies, and data transmission patterns, poses a growing difficulty in their detection. This paper introduces an innovative strategy for mitigating botnet threats. Employing differential evolution, we propose a feature selection approach that enhances the ability to discern peer-to-peer (P2P) botnet traffic amidst evolving cyber threats. Differential evolution is a population-based meta-heuristic technique which can be applied to nonlinear and non-differentiable optimization problems owing to its fast convergence and use of few control parameters. Apart from that, an ensemble learning algorithm is also employed to support and enhance the detection phase, providing a robust defense against the dynamic and sophisticated nature of modern P2P botnets. The results demonstrate that our model achieves 99.99% accuracy, 99.49% precision, 98.98% recall, and 99.23% F1-score, which outperform the state-of-the-art P2P detection approaches.

1. Introduction

Botnets persist as a significant threat to computing resources worldwide. They are one of the most hazardous types of malware, efficiently used by cyber criminals as a platform to launch various malicious activities such as distributed denial of service (DDoS) attacks, identity theft, sending spam, phishing attacks, etc. [1]. The term “botnet” came from a combination of the two words “robot network” [2]. It is a large network of compromised computers which are interconnected via the internet and controlled by one attacker known as the botmaster [3]. Since the botmaster has large computing resources at their disposal, botnet attacks can be massive. In fact, cybersecurity threats are one of the major challenges for countries worldwide after digitization. According to a report [4], over 3800 major corporations worldwide encountered data breaches in 2019, leading to significant repercussions. Another study [5] reveals a sharp surge in botnet attacks during the first half of 2021, with their frequency increasing by 41%. Concurrently, a separate report [5] highlights a substantial rise in the percentage of organizations detecting botnet activity, increasing from 35% to 51% during a similar time frame. Since organizations worldwide are negatively influenced by botnet attacks, there is a vital need to detect such activities. Hence, there is a necessity to develop an enhanced botnet detection framework.
Botnets are classified as centralized and decentralized botnets [6]. Centralized botnets operate through a specific Command and Control (C&C) server. The botmaster utilizes this centralized hub to communicate with all the bots under their influence. Centralized botnets basically use IRC and HTTP protocols for communication. This architecture offers many advantages to the botmaster. It offers consistent control and coordination between the botmaster and the bots, simplifying monitoring for the botmaster. Nevertheless, it comes with a vulnerability—a single point of failure. Detecting and blocking the C&C server exposes a critical weakness. With the server blocked, the botmaster loses communication with the other bots, leading to the collapse of the entire botnet. On the other hand, decentralized botnets, also known as peer-to-peer (P2P), do not have a designated C&C server. Initially, a single bot receives the message from the C&C server. This bot then takes on the responsibility of relaying the message to peer bots, creating a chain of transmission across the network. Various communication protocols, such as Overnet, Kademlia, etc., are employed for this purpose [6,7].
For botnet detection, researchers are mainly focusing on machine-learning-based botnet detection models because of their high effectiveness [8,9]. Most of these detection models rely on feature selection, where distinct feature sets are derived from the existing high-dimensional dataset. A high-dimensional real-world dataset consists of a large number of features. But all of the features may not be essential or beneficial. Some features might be redundant and irrelevant. Feature selection removes redundant, irrelevant, and misleading features in order to obtain the most critical subset representing the best solution, thereby increasing the overall performance, accuracy, and learning speed of any machine-learning model [10]. Hence, feature selection becomes crucial. The underlying concept of feature selection is to thoughtfully choose a relevant subset of features from the initial dataset. This reduces the dimensionality of the dataset, enhancing the overall efficiency of the learning model.
Feature selection is a combinatorial optimization problem. Combinatorial optimization problems involve a finite set of potential solutions. An optimal solution to such problems is typically found by examining all feasible solutions within the search space. However, this exhaustive approach is often impractical, particularly when dealing with a vast search space. Consequently, numerous metaheuristic algorithms have been developed and adapted to address these challenges [11]. Metaheuristic algorithms are stochastic optimization methods designed to enhance the effectiveness of underlying heuristic techniques. These approaches effectively address the challenge of local optima and have proven successful in solving a diverse range of optimization problems [11,12]. While metaheuristic-based feature selection plays a crucial role in the development of various machine-learning models, it has received comparatively less attention in the literature on botnet detection.
This paper introduces an enhanced P2P botnet detection model using differential evolution-based feature selection and supervised machine-learning algorithms including ensemble learning, which can efficiently identify P2P bot communications by considering the following key questions:
(1)
How to enhance feature selection using evolutionary metaheuristic algorithm such as differential evolution.
(2)
How to design an enhanced learning model for P2P botnet detection using various supervised machine-learning algorithms.
(3)
How to enhance the performance of the proposed model by using the majority voting ensemble learning algorithm.
(4)
How to validate the proposed model in a dataset containing real-world botnet flows.
(5)
How to evaluate the performance of the proposed model using metrics such as accuracy, precision, recall, F1-score, and confusion matrix.
The rest of this paper is arranged as follows: Section 2 covers the related work, Section 3 covers the proposed approach, Section 4 contains the experiment discussions, Section 5 contains results, Section 6 contains comparison, and Section 7 contains conclusions and scope for future research.

2. Related Work

Some related works that have used metaheuristic-based techniques for feature selection (FS) and machine-learning-based botnet detection are presented in this section. Based on the category of feature selection algorithms applied, we group all the state-of -the-art research works into four categories. In the first group, we discuss research works that have used swarm- and evolution-based metaheuristics for FS. In the second group, we discuss those research works that have used evolution-based metaheuristics for FS. In the third group, we discuss research works that have used human-behavior-based metaheuristics for FS. In the fourth group, we discuss those research works that have used filter methods for FS.
Lin et al. [13] proposed a botnet detection method combining the Artificial Fish Swarm Algorithm (AFSA) for feature selection and Support Vector Machine (SVM) for classification. They compared this model with a GA-SVM system and found that AFSA yielded a superior performance, achieving 100% detection under a 10-fold cross-validation scheme. However, their experiments were conducted in a simulated LAN environment, which limits applicability to real-world, diverse network conditions. Moreover, their work does not target P2P botnets specifically, nor does it incorporate ensemble learning, which can enhance generalization. In contrast, our work focuses on detecting P2P botnets using differential evolution for feature selection and a majority voting ensemble of classifiers, applied on a real-world botnet dataset (CTU-13). Liao et al. [14] investigated botnet detection using two metaheuristic algorithms—Particle Swarm Optimization (PSO) and Genetic Algorithm (GA)—for feature selection, combined with a backpropagation neural network for classification. Their model achieved an average detection accuracy of 95.09%. The authors also introduced a custom feature based on packet transmission regularity. While the work highlights the effectiveness of metaheuristic-driven feature selection, it lacks clarity on the dataset used and does not target P2P botnets, which are harder to detect due to their decentralized architecture. Additionally, the use of a single neural network classifier may limit generalization. In contrast, our approach employs differential evolution for feature selection and integrates six supervised classifiers in a majority voting ensemble, evaluated on a real-world P2P botnet dataset, thereby offering improved robustness and relevance. Asadi et al. [15] proposed a hybrid botnet detection system, BD-PSO-V, that integrates Particle Swarm Optimization (PSO) for feature selection with an ensemble voting-based classification system comprising a Deep Neural Network (DNN), Support Vector Machine (SVM), and decision tree. Their model was evaluated on the ISOT and Bot-IoT datasets and achieved a high accuracy (up to 99.63%). While their work effectively demonstrates the advantage of hybrid ensemble approaches, it focuses primarily on IoT botnets and does not specifically address the detection of P2P botnets, which have more complex and decentralized architectures. Moreover, the computational demands of PSO and the limited scope of classifier diversity may constrain generalizability. In contrast, our proposed method used differential evolution (DE) for efficient feature selection and employed a broader ensemble of six classifiers under a majority voting system, specifically targeting P2P botnet detection using a real-world dataset (CTU-13 Scenario 12).
Alejandre et al. [16] proposed a botnet detection framework focused on identifying connections during the Command and Control (C&C) phase of botnet activity. They employed a Genetic Algorithm (GA) for selecting optimal features and used a C4.5 decision tree classifier to distinguish botnet traffic. Their model achieved a high detection rate (up to 99.46%) using datasets from ISOT and ISCX. While their focus on C&C traffic is important, the work does not address P2P botnets, which do not have centralized control structure and are inherently more difficult to detect. Additionally, the use of a single classifier may limit adaptability to varying network patterns. In contrast, our work focuses on detecting P2P botnets using differential evolution (DE) for efficient feature selection and a diverse ensemble of classifiers under a majority voting strategy, validated on real-world botnet data (CTU-13 Scenario 12). Alhijaj et al. [8] proposed a botnet detection model that integrates the Genetic Algorithm (GA) for feature selection with a decision tree classifier for traffic classification. Their approach aimed to find the optimal feature subset that enhances classification performance. Using the UNSW-NB15 and CICIDS2017 datasets, they demonstrated improved results over standard decision tree models across metrics such as precision, F1-score, and detection rate. However, their study does not address the detection of P2P botnets, which are decentralized and more elusive than conventional botnets or intrusions. Furthermore, the use of only one classifier may limit adaptability in real-world environments. In contrast, our work uses differential evolution for more efficient feature selection and applies a majority voting ensemble with six diverse classifiers, evaluated on a real-world P2P botnet dataset (CTU-13) to improve both accuracy and robustness.
Nejad et al. [17] presented a real-time botnet detection model that utilizes the World Competitive Contests (WCC) algorithm for feature selection and Support Vector Machine (SVM) for classification. Their approach achieved an accuracy of 95%, outperforming other comparative methods. While the focus on real-time detection is commendable, the study does not specify the type of botnet traffic, and it relies on a single classifier, which may limit robustness. Additionally, the novel WCC algorithm is not extensively benchmarked against more established metaheuristics. In contrast, our work focuses on the detection of P2P botnets, a more evasive and challenging class, using differential evolution (DE) for efficient feature selection and a majority voting ensemble of six classifiers, which collectively improve accuracy and resilience under real-world conditions.
Yallamanda et al. [18] proposed a machine-learning-based model for detecting both botnet and DDoS attacks using Support Vector Machine (SVM) and Naïve Bayes (NB) classifiers. Evaluated on the CICIDS2017 dataset, the results indicated that SVM performed better than NB. However, their approach lacks a feature selection component and focuses solely on traditional classifiers, which may limit performance on more complex and stealthy threats. Additionally, their study does not address P2P botnets, nor does it explore ensemble learning or comprehensive performance metrics. In contrast, our work focuses on P2P botnet detection using differential evolution for feature selection and a diverse ensemble of classifiers, validated with detailed performance analysis on real-world traffic. Joshi et al. [19] conducted an empirical study using five filter-based feature selection techniques—Univariate Selection, RFE, PCA, Correlation Matrix, and Feature Importance—combined with four classifiers including SVM, Logistic Regression, KNN, and decision trees. Their experiments, conducted on the CTU-13 dataset, found that KNN consistently outperformed other classifiers across different FS techniques. While their work offers useful insights into the interaction between feature selection and classifier performance, it is limited to filter methods, which do not consider feature interdependencies. The study does not explore metaheuristic or optimization-based FS techniques, nor does it address P2P botnet detection, which presents more complex detection challenges. Our work addresses these gaps by using differential evolution (DE) for feature selection and a majority voting ensemble of six classifiers, specifically tailored for P2P botnet detection on CTU-13 Scenario 12. Ismail et al. [20] proposed the BADS framework for botnet detection over encrypted channels using a structured six-phase methodology. They employed Information Gain as a filter-based feature selection method and used the NIMS and MCFP datasets for experimentation. While the study is notable for addressing encrypted traffic and presenting a modular detection pipeline, it relies on a single FS approach and classifier, limiting its adaptability to more dynamic and evasive botnets such as those with P2P architectures. In contrast, our work focuses on P2P botnet detection using a differential evolution-based feature selection strategy and a majority voting ensemble of six classifiers, enhancing detection performance and robustness in decentralized botnet scenarios. Muhammad et al. [21] proposed an early-stage botnet detection model that utilizes filter-based feature selection methods (such as PCA and Information Gain) and evaluates the classification performance using Logistic Regression, Multilayer Perceptron, Support Vector Machine, and Random Forest. Using the Cyber Clean Centre (CCC) dataset, which contains IRC and HTTP-based botnet traces, their model achieved a detection accuracy of 99%. While the focus on early detection is valuable, the approach does not address the challenges posed by P2P botnets, which lack centralized control and are harder to detect. Moreover, the use of filter-based FS methods may overlook critical feature dependencies. In contrast, our study targets P2P botnet detection using differential evolution (DE) for feature selection and a majority voting ensemble of six classifiers, tested on real-world decentralized botnet traffic (CTU-13 Scenario 12) for enhanced robustness and relevance. Joshi et al. [22] proposed a multiclass botnet detection framework using Artificial Neural Networks (ANNs), trained on a dataset comprising seven different botnet classes. Their study employed various filter-based feature selection techniques and evaluated the model on the MCFP dataset, achieving a classification accuracy of 99.04%. While the shift toward multiclass detection is noteworthy, this study does not clearly address P2P botnets, which are more difficult to detect due to their decentralized nature. Additionally, the exclusive reliance on ANN and filter-based FS methods may limit adaptability to unseen or complex attack patterns. In contrast, our study specifically targets P2P botnets using a differential evolution-based feature selection strategy and a majority voting ensemble of six classifiers, evaluated on the real-world CTU-13 Scenario 12 dataset, offering both higher robustness and practical relevance.
To summarize, while prior works have contributed significantly to botnet detection using various filter-based and metaheuristic feature selection techniques alongside diverse classifiers, several critical gaps remain unaddressed. The comprehensive summary of existing work on machine learning based botnet detection with their advantages and disadvantages are presented in Table 1. Most studies focus on centralized botnet types and rely on limited classifier diversity or simulated environments. Moreover, filter-based feature selection dominates the landscape, often overlooking feature interdependencies and the adaptability required for detecting more elusive peer-to-peer (P2P) botnets. In contrast, our proposed approach introduces differential evolution (DE) as an efficient evolutionary strategy for feature selection, which better captures complex feature interactions. Furthermore, we enhance robustness by employing a majority voting ensemble of six diverse supervised classifiers. Most importantly, our method is specifically tailored for P2P botnet detection and is rigorously evaluated using the real-world CTU-13 Scenario 12 dataset, thereby offering a novel, scalable, and practical solution that addresses the limitations found in the existing literature.

3. Proposed Approach

This section discusses the proposed approach for the classification and detection of P2P botnet flows and also discusses their theoretical description.

3.1. Model Construction

Often, it has been seen that in a network, communication among the bot nodes is less than the benign nodes. The bot nodes communicate infrequently to evade detection. Thus, if we use network packet passing through a router of a network, our method has to execute a large volume of data to detect bot node communication. This is not feasible in a real-world network scenario. Therefore, we use network flow data in our method to detect bot nodes in a network.
Network flow data aim to identify botnet flows based on the information extracted from network conversations between hosts. Flow data, as described by Kemp et al. [23], encapsulate IP traffic details in the form of features. A flow summarizes the communication between hosts and is characterized primarily by a quintuple attribute set, including the source IP, destination IP, source port, destination port, and transport protocol. In addition to these essential attributes, flow data may encompass various other features like flow duration, flow direction, total bytes transferred, and more. The advantage of using flow data lies in their capability to be analyzed efficiently using machine-learning techniques to detect botnet flows. Notably, the analysis of flow data does not involve the inspection of the payload or content of the data. As a result, detection can occur even when the data are encrypted. Furthermore, flow data analysis remains independent of structure and protocol, enhancing its versatility in botnet detection procedures [6].
This network flow data are parsed as input into our proposed detection approach. The detection approach starts with feature selection in the input dataset. Here, the differential evolution (DE) algorithm is implemented to select the optimal feature set from the input. Selected feature sets are then applied to six machine-learning models (base learners) which are as follows: Gaussian Naïve Bayes (NB), K-Nearest Neighbors (KNN), Linear Discriminant Analysis (LDA), decision tree (DT), Random Forest (RF), and Support Vector Machine (SVM). To enhance the results produced by the classification models, we have used the majority voting ensemble learning algorithm. The system workflow of our proposed botnet detection approach is shown in Figure 1.

3.2. Feature Selection

Feature selection is the process of identifying the most relevant subset of features from a given dataset [24]. In today’s context, real-world datasets are often abundant, containing a large number of features. However, the sheer quantity of features does not imply that every feature is essential or beneficial. Some features may be redundant or irrelevant. A dataset containing noisy, redundant, and irrelevant information can lead to overfitting, diminishing the overall performance, accuracy, and learning speed of any machine-learning model [10]. Feature selection reduces the computational cost by reducing the dimensionality of data. Therefore, feature selection is crucial. Feature selection involves choosing a subset of features from the original set without modifying them, thus preserving the original features’ meaningful interpretations. Traditionally, feature selection techniques are categorized into three categories: filter method, wrapper method, and embedded methods [24]. Apart from these algorithms, metaheuristic algorithms can also be used for feature selection. Metaheuristic algorithms are inspired by natural phenomena and are crafted to tackle intricate optimization problems [25]. They belong to a category of optimization algorithms crafted to discover approximate solutions for challenging optimization problems. These algorithms prove especially beneficial when addressing complex problems where obtaining exact solutions is either impractical or computationally demanding. Serving as high-level strategies, metaheuristics direct the exploration and exploitation of solution spaces, enabling efficient searches for near-optimal solutions. In our proposed work, we have employed a differential evolution algorithm for feature selection, which is an evolutionary based metaheuristic algorithm [26,27]. Evolutionary algorithms are based on Darwin’s theory of evolution. Some algorithms of this category are Genetic Algorithm (GA), Evolutionary Programming (EP), Evolution Strategies (ESs), Genetic Programming (GP), and differential evolution (DE). Previous studies have shown that differential evolution (DE) achieves a better performance than traditional FS approaches in other fields [28,29,30].

Differential Evolution

Differential evolution (DE) is a metaheuristic algorithm used for solving various optimization problems where the objective function may be nonlinear, non-differentiable, or noisy. Feature selection is a combinatorial optimization problem. DE is a metaheuristic algorithm based on populations, designed to locate the global minimum of a multivariate function. This evolutionary computing algorithm initiates with an initial set of candidate solutions and systematically updates them through iterations [27]. DE is applied to multidimensional real-valued functions without relying on the gradient of the optimization problem. This lack of dependence on gradients distinguishes DE from classical optimization methods like gradient descent and quasi-Newton methods, which necessitate differentiability. Consequently, DE is suitable for optimization problems that are non-differentiable, non-continuous, noisy, subject to changes over time, and various other challenging scenarios [27]. DE is a stochastic direct search method that explores the solution space without relying on derivative information, making it well suited for optimization tasks [28]. It draws inspiration from Darwin’s theory of evolution and natural selection. Demonstrating its effectiveness across diverse domains, the DE algorithm orchestrates the evolution of a population of candidate solutions through iterative processes of mutation, crossover, and selection. This ongoing evolution, transitioning from one generation to the next, guarantees that individuals with superior qualities persist within the population, while weaker ones are systematically phased out in each iteration [29].
Based on the reviews [28,29,30,31,32], the advantages of DE for feature selection are described below:
  • DE is simple, straightforward, and easy to implement compared to other evolutionary algorithms.
  • Despite its simplicity, DE exhibits a better performance.
  • The number of control parameters in DE is much lower.
  • DE is known for its fast convergence and efficient exploitation of the search space.
  • DE can be used for nonlinear and non-differentiable optimization problems.
The workings of the DE algorithm are depicted by the following flow chart (Figure 2) and Algorithm 1.
Algorithm 1. The operation of the differential evolution algorithm.
1. Initialize: Generate an initial random population of individuals across the entire search space.
2. Repeat until the maximum number of generations is reached:
   For each individual in the population:
      (i) Perform mutation: Create a mutant vector.
      (ii) Perform recombination (crossover): Combine the mutant vector with the target vector to create a trial vector.
      (iii) Perform selection: Compare the trial vector with the target vector and select the one with the better fitness.
3. Return the best individual found.
DE has three basic operators: mutation, crossover, and selection.
Mutation: DE generates new candidate solutions by perturbing the existing population. This is typically achieved by selecting three distinct individuals (parents) from the current population and combining them to create a new solution (offspring). During this process, an offspring or mutant vector is generated for every target vector using the following Equation (1):
V i , g + 1 = X r 1 , g + F X r 2 , g X r 3 , g
where g represents the generation or iteration, V i is the new solution vector or mutant vector (i = 1, 2, 3, …., NP), and r1, r2, and r3 are three distinct individuals selected randomly from the population such that r1, r2, r3 ϵ [1, NP] and r1 ≠ r2 ≠ r3 ≠ i. NP is the population size. F is a user-defined scaling factor or mutation parameter that amplifies the difference vector and lies within [0, 1]. This process of generating a new individual is known as mutation.
Crossover: In order to enhance the diversity of the perturbed population, a crossover or recombination operator is used. During this phase, a crossover operator is applied between the parent vector or target vector X i and the corresponding mutant vector V i to generate a trial vector W i j as in the following Equation (2):
W i j = V i j             if   r a n d C R   or   j = jr   and X i j             otherwise
where i ϵ (1, 2,…,NP), j ϵ (1, 2,…,D), rand () is a uniformly distributed random number ϵ [0, 1], CR is the crossover rate ϵ [0, 1], and jr and is a randomly chosen index to ensure that at least one component from the mutant vector is copied to the trial vector.
Selection: In the selection phase, a decision is taken as to whether to include the trial vector wij in the next generation g + 1. The trial vector wij is compared with the target vector xij using a greedy criterion. If the trial vector has a better fitness value than the target, it replaces the target in the population. This step ensures that the population evolves over generations towards better solutions.

3.3. Supervised Machine-Learning Classifiers

The six supervised machine-learning algorithms and ensemble learning algorithm used in our experiments are discussed below.

3.3.1. Gaussian Naïve Bayes (NB)

The Gaussian Naïve Bayes algorithm is a supervised ML algorithm which is based on the Bayes theorem. It is simple, computationally efficient, and suitable for high-dimensional datasets.
It is an intuitive classification approach which measures the likelihood of an event based on prior knowledge of the conditions that facilitated the occurrence of such an event [11]. In the sphere of machine learning, the Naïve Bayesian method works under the principle that every feature contributes independently to the occurrence of an event. To be precise, the Naïve Bayes classifier estimates the probabilities for all classes (values) associated with a target feature and selects the one with the highest probability. The algorithm can be expressed as Equation (3):
P ( A / B ) = ( P ( B / A ) P ( A ) ) / ( P ( B ) )
where A and B are occurrences and P(B) ≠ 0.
  • P(A/B) is the proportion of instances of a specified class given the observed features.
  • P(B/A) is the likelihood of observing the features given the class.
  • P(A) is the prior probability of the class.
  • P(B) is the probability of observing the features.

3.3.2. K-Nearest Neighbors (KNN)

KNN is very easily understood by beginners and very simple to implement. KNN is a supervised machine-learning algorithm based on classification as well as regression. KNN works well for complex and non-linear datasets. It is a non-parametric and instance-based algorithm, which is based on the fact that it does not impose any stringent assumptions on the data distribution and rather utilizes the similarity of the instances to make predictions [33]. KNN utilizes a distance metric such as a Euclidean distance measure to measure the similarity between data points [6]. However, distance metrics such as Manhattan or Minkowski may also be utilized. To make a prediction for a new data point, KNN identifies the K-nearest neighbors in the training set based on the chosen distance metric. It operates by computing the Euclidean distance (or any other chosen metric) between data points. The categorization of an object is determined by the majority vote among its K-nearest neighbors, considering entities from various classes [7]. The Euclidean distance between two points (A, B) in Euclidean n-space can be expressed as Equation (4):
d A , B = x = 1 n ( B x A x ) 2

3.3.3. Linear Discriminant Analysis (LDA)

LDA is a type of supervised and classification-based machine-learning algorithm. It is particularly advantageous in scenarios where the objective is to identify a linear combination of features for the optimal separation of two or more classes. LDA is most effective when the aim is to increase class separability, especially when the classes have similar covariance. The reason for its popularity in many fields can be related to the fact that it is straightforward and easy to understand. LDA works under two assumptions: first, the features are always normally distributed; second, the feature-covariance matrices are identical across all of the classes. The approach seeks to establish a hyperplane that separates the classes as far away as possible while at the same time minimizing the intra-class variance [34].

3.3.4. Decision Tree (DT)

The DT classifier is one well-known supervised machine-learning method used for regression and classification. It is favored due to its great performance and simplicity, especially in a highly interpretable scenario. The method builds a hierarchical structure that looks like a tree such that every non-leaf node represents a decision based on a particular feature, every branch represents the outcome of this decision, and every terminal node represents the final classification label or value [33]. A decision tree is constructed by partitioning the training data into sub-samples based on the most distinguishing feature. As new data come in, the data attributes are traced from the root of the tree, progressing through various nodes, until reaching a leaf node that signifies the classification of the data [35]. In particular, decision trees are easy to understand and use, rendering them valuable for elucidating decision-making processes. On top of that, they are able to model complex nonlinear relationships in the data, and they operate without imposing assumptions about the underlying data distribution.

3.3.5. Random Forest (RF)

The Random Forest classifier is essentially an ensemble classifier which attempts to utilize a large number of decision trees in order to predict with greater accuracy. This method can be used for both classification and regression purposes and is considered to be robust, versatile, and efficient for dealing with complex datasets. It is applied in different fields where accuracy in predictions holds paramount importance. An RF consists of a number of decision trees, in which every tree is trained on a random sample of the training data [6]. Using a technique called bagging, this method trains each tree on a new sample of the data (with replacement) taken from the original dataset. At each DT node, a random subset of features is considered for splitting, introducing diversity among the individual trees. In classification tasks, the ultimate prediction is determined through a majority vote among the predictions generated by individual trees. Through this ensemble approach, Random Forest effectively mitigates overfitting concerns often associated with standalone decision trees.

3.3.6. Support Vector Machine (SVM)

The Support Vector Machine (SVM) is a very popular supervised machine-learning algorithm that can be used for both regression and classification tasks. The SVM is acknowledged as being flexible and effective to do linear and non-linear classification. This algorithm is considered to be quite useful, particularly when working in high dimensional spaces, resulting in its application in many fields. The main strength of SVM is that it can find the optimal hyperplane that separates the data sample of different classes in the feature space. Whether in two dimensions where it manifests as a line or in higher dimensions as a hyperplane, SVM’s proficiency extends to handling non-linear relationships. By utilizing kernel functions such as polynomial or radial basis functions, SVM employs the kernel to transform the input space into a higher-dimensional space, facilitating the identification of linear separation within that space [33].

3.4. Ensemble Learning Algorithm (Majority Voting)

Ensemble learning is a machine-learning approach where predictions made by multiple models are utilized to improve overall performance and generalization. The core concept is to leverage the diversity inherent to individual models, thereby generating predictions that are not only robust but also more accurate. The effectiveness of ensemble methods lies in the diversity among constituent models. Diverse models, even if individually weak, can collectively provide a more accurate and stable prediction [11]. Majority voting is a simple yet powerful ensemble algorithm used in classification tasks. Majority voting requires multiple base models, each trained on a subset of the data. Each individual model in the ensemble independently makes a classification decision for a given input. The final prediction is determined by a majority vote among the individual models. The class that receives the most votes is chosen as the ensemble’s prediction [6,36,37,38].

4. Experiments

This section describes the dataset used, parameter tuning of DE, all classification algorithms, and the majority voting algorithm. This section also explains all of the the evaluation matrices used in this work.

4.1. Dataset

Our experiments utilized the CTU-13 botnet dataset, which is a collection of botnet traffic that was assembled by CTU University, Czech Republic, in 2011. This is a publicly available dataset which contains 13 botnet scenarios or captures of different botnet samples summarized in 13 bidirectional NetFlow files, which comprises actual botnet traffic, offering a valuable resource for scrutinizing real botnet behavior and gaining insights into the diverse strategies employed by botnet operators [39]. Out of these 13 bidirectional NetFlow files, we have selected the 12th scenario for our experiments because only the 12th scenario contains P2P bot nodes’ communication as well as benign nodes’ communication. The first reason for selecting this dataset is that it is publicly available, whereas there are some datasets which are not publicly accessible. Another reason is that there are certain datasets that comprise bot flows originating from a single bot node, while others feature multiple bot nodes without inter-node communication. However, the CTU-13 (Scenario 12) dataset contains three P2P bot nodes along with benign nodes and also incorporates the communication flow among bot nodes.
While the CTU-13 Scenario 12 dataset originates from 2011, it continues to serve as a valuable benchmark for botnet research—particularly in the context of P2P botnet detection. This scenario is among the few publicly available datasets that include real-world P2P botnet communication with multiple infected hosts and inter-bot interactions. These characteristics are essential for studying decentralized botnet behavior. Nevertheless, we acknowledge that botnet communication patterns evolve, and future work will involve validating the proposed feature selection and classification framework on newer datasets or real-time traffic to further assess generalizability and resilience to emerging threats.
The features related to the network traffic for Scenario 12 are shown in Table 2. Table 2 represents information like size, duration, number of packets, number of flows, number of bots, and bot family [35].
This dataset is labelled and comprises bidirectional NetFlows. NetFlow is a network protocol developed by Cisco that facilitates the collection and monitoring of IP traffic information. The CTU-13 (Scenario 12) dataset is specifically a bidirectional NetFlow file and is characterized by 15 features [6]. Table 3 describes various features of the dataset used in our experiments.

4.2. Parameter Tuning

This section discusses the experiments performed for parameter tuning of all of the algorithms used.

4.2.1. Parameter Tuning of Differential Evolution Algorithm

DE is an evolutionary metaheuristic optimization algorithm, with multiple parameters that influence its performance. The selection of these parameters plays a crucial role in determining the algorithm’s efficiency and convergence speed. These parameters can be optimized using methodologies such as grid search, random search, or by using some advanced optimization techniques to discover values that effectively address the specifics of the optimization problem at hand. Python (version 3.12) offers a range of libraries for implementing evolutionary algorithms such as SciPy, DEAP, etc. In our experiments, we have used python’s DEAP (Distributed Evolutionary Algorithms in Python) library. DEAP is a multi-purpose library for evolution strategies, and it has a number of modules adapted to various kinds of evolutionary strategies including DE. This library offers a high degree of customization and flexibility, making it a preferred choice if we wish to tailor our evolutionary algorithms to specific needs or experiment with different aspects of the algorithm’s behavior [40]. DEAP is a powerful library for building custom evolutionary strategies for specific purposes. In our studies, however, the DE parameters [41] that were used according to DEAP’s Simple function are as follows: “ngen”, “popsize”, “cxpb”, and “mutpb”.
Number of generations (“ngen”) is a controlling factor used to determine how many generations or iterations to carry out within the algorithm.
Population size (“popsize”) is a controlling factor used to control the number of the candidate solutions that are maintained in each generation.
Crossover probability (“cxpb”) is a control parameter that determines the likelihood of performing crossover (recombination) operations on the individuals in the population.
Mutation probability (“mutpb”) is also a control parameter that governs the chances of mutations happening in individual candidate solutions within the population.
In order to find the best value for these parameters, we have experimented with a random search algorithm. We have taken 10 values (V1 to V10) for each parameter as shown in Table 4.
To make a selection for the most optimal values of the DE parameters, a random search algorithm was run 50 times. Table 5 shows the fluctuations of “ngen” selected values during 50 runs. Table 5 reveals that V8 (90) is the value that was mostly selected for the parameter “ngen”.
Table 6 shows the fluctuations of “popsize” values selected during 50 runs. Table 6 reveals that V9 (100) is the value that was mostly selected for the parameter “popsize”.
Table 7 shows the fluctuations of “cxpb” values selected during 50 runs. Table 7 reveals that V1 (0.1) is the value that was mostly selected for the parameter “cxpb”.
Table 8 shows the fluctuations of selected “mutpb” value. Table 8 reveals that V10 (0.3) is the value that was mostly selected for the parameter “mutpb”.
The selected parameters of DE that are used for feature selection are given in Table 9.

4.2.2. Parameter Tuning of Classification Algorithms

For our experiments, we have used six classification algorithms, namely NB, LDA, SVM, KNN, DT, and RF. To implement these algorithms, we used the Scikit-learn python library. To select the best values for the parameters of these algorithms, we used the randomSearchCV algorithm.
Table 10, Table 11, Table 12, Table 13, Table 14 and Table 15 present the key parameters along with their optimal values for various classification algorithms. Specifically, Table 10 details the parameters for the Gaussian Naïve Bayes algorithm [42], Table 11 for the K-Nearest Neighbors (KNN) algorithm [43], Table 12 for the Linear Discriminant Analysis (LDA) algorithm [44], Table 13 for the decision tree (DT) algorithm [45], Table 14 for the Random Forest (RF) algorithm [46], and Table 15 for the Support Vector Machine (SVM) algorithm [47].

4.2.3. Parameter Tuning of Majority Voting Ensemble Learning Algorithm

Table 16 presents the key parameters and their optimal values for the majority voting ensemble algorithm [48], which combines multiple base classifiers to improve the overall predictive performance. The algorithm utilizes six individual estimators—LDA, KNN, DT, RF, NB, and SVM—as the constituent models. The voting strategy employed is soft voting, which considers the predicted class probabilities from each model rather than just the final class labels. Additionally, the weights assigned to each estimator are specified as [1, 2, 2, 2, 1, 1], indicating a higher influence of KNN, DT, and RF in the final ensemble decision, thereby reflecting their stronger individual performance or contribution in the ensemble framework.
The weights [1, 2, 2, 2, 1, 1] assigned to LDA, KNN, DT, RF, NB, and SVM, respectively, were obtained using RandomizedSearchCV, which explored multiple combinations of weights across five-fold cross-validation to identify the configuration with the best predictive performance. This ensures a systematic, data-driven approach to weighting rather than relying on heuristic assumptions.

4.3. Evaluation Metrics

The following evaluation matrices have been used in our experiments:
Confusion Matrix: A confusion matrix is a very useful tool for evaluating classification models. It is particularly beneficial for handling imbalanced datasets and provides a more comprehensive evaluation than accuracy alone. It offers a comprehensive analysis of a classification algorithm’s performance by presenting a detailed summary of true positive, true negative, false positive, and false negative predictions. The key components of a confusion matrix are as follows:
True Positive (TP): TPs are occurrences that are correctly predicted as positive by the classification model.
True Negative (TN): TNs are occurrences that are correctly predicted as negative by the classification model.
False Positive (FP): FPs are occurrences that are incorrectly predicted as positive by the classification model.
False Negative (FN): FNs are occurrences that are incorrectly predicted as negative by the classification model.
Accuracy: Accuracy is a fundamental performance metric that measures the overall correctness of predictions made by a classification model. It is the ratio of correctly classified instances, both positive and negative, to the total number of instances in the dataset. It is calculated as Equation (5):
A c c u r a c y = ( T P + T N ) ( T P + T N + F P + F N )    
Precision: Precision is a performance metric that measures the accuracy of the positive predictions made by a classification model. In other words, it is the ratio of true positive instances, among all instances predicted as positive. It is calculated as Equation (6):
  P r e c i s i o n = T P T P + F P      
Recall (Sensitivity): Recall is also known as sensitivity or the true positive rate. This performance metric measures the ability of a classification model to correctly identify all relevant instances from a dataset. In other words, it is the ratio of true positive instances among all actual positive instances. It is calculated as Equation (7):
  R e c a l l = T P T P + F N      
F1-Score: The F1-score is a performance metric that combines both precision and recall into a single value. It is the harmonic mean of precision and recall and provides a balanced measure of performance of a classification model. It is particularly useful when there is an uneven class distribution or when the cost of false positives and false negatives needs to be balanced. It is calculated as Equation (8):
F 1 = 2     P r e c i s o n R e c a l l P r e c i s o n + R e c a l l      

5. Results

To reduce the dimensionality of the dataset, one of the state-of-the-art evolutionary algorithms known as differential evolution (DE) is used. Due to the inherent nature of stochastic optimization algorithms like DE, it produces different results each time it is run. This algorithm involves an element of randomness, and this stochastic nature can yield diverse outcomes in separate runs, even under optimal functioning conditions. The factors given below describes variations observed in the results of stochastic optimization algorithms like DE.
  • DE starts with a randomly generated initial population of candidate solutions. The variations among these initial solutions can influence diverse paths in the optimization process.
  • DE involves stochastic operations such as mutation and crossover. These stochastic processes can result in different offspring or solutions at each generation.
  • With each generation, the mutation operation introduces random perturbations to the solutions. The unpredictability in these perturbations can lead the optimization process to explore diverse regions of the search space in each iteration.
  • DE employs heuristics to direct the search, and the interaction of these heuristics with randomness can result in varied decisions across different runs.
Therefore, to mitigate the effects of randomness and to determine the most optimal features, we run the algorithm 50 times. In each run, the DE algorithm produces a diverse set of optimal features. If the number of times a feature is selected is high, that means the particular feature is very important and very relevant for classification. Thus, those features with high counts are deemed critical for effective botnet detection. The 12th scenario of the CTU-13 dataset consists of 14 features (F1 to F14), excluding the target or label, as described in Table 3. Table 17 shows the count of all of the features that were selected during 50 runs, where it is possible that more than one feature is selected in each run. Table 17 reveals that feature 4 (SrcAddr) was the mostly selected feature.
The repeated use of SrcAddr (source IP address) shows how useful it is in classifying P2P botnet traffic. In networks infected with botnets, compromised hosts (bots) participate in recurrent and decentralized peer-to-peer communication. This leads to anomalous behavior where specific source IPs initiate an excessive number of connections, sometimes to different destinations or within very short intervals. These patterns can significantly differ from typical host behavior in legitimate traffic, making SrcAddr a valuable feature for distinguishing infected nodes.
Sport (source port) was the second most frequently selected feature in DE runs due to its operational significance. Botnet communications may involve the use of non-standard or fixed source ports for consistent peer discovery or command exchange. The appearance of uncommon or repetitive source ports across different flows can signal the presence of malware or automated behavior. From a domain perspective, both SrcAddr and Sport provide granular, connection-level insights that are highly effective for detecting distributed and stealthy P2P botnet activities.
In the experiments, it has been found that all of the classifiers and the ensemble learning algorithm exhibit different levels of performance when different feature subsets are applied. In order to find out the optimal result, the proposed approach was tested with different feature subsets. The initial subset encompasses the two most optimal features, followed by the gradual creation of subsequent subsets through the incremental addition of less significant features, up to the incorporation of the least significant one. Table 18 demonstrates the results of the majority voting ensemble algorithm using all of the feature subsets.
By analyzing the table above, we can draw the following conclusions:
  • Accuracy is consistently high across all feature sets, ranging between 99.97% and 99.99%. The accuracy remains nearly unchanged, even as the number of features increases.
  • Precision is a key metric that measures the proportion of true positives out of all positive predictions. A feature set with two features gives the highest precision at 99.49, which decreases slightly as more features are included but generally remains high.
  • Recall measures how well the model identifies true positives out of all actual positives. The recall is highest with the two-feature set (98.98%) and tends to decrease as more features are added, bottoming out at 95.54% with seven features.
  • The F1-score balances precision and recall, and its values are high across all feature sets. The highest F1-score (99.23%) is observed with two features, which balances both precision and recall well.
  • The true positives are highest with feature sets ranging from 9 to 10 features (2086 TPs).
  • The false negatives (missed detections) are lowest when using two features (22 FNs), which is desirable as it indicates fewer missed detections of positive cases.
  • The false positives (incorrect positive classifications) are slightly higher with feature sets that have fewer features (such as 2 features with 11 FPs) but remain very close across most feature sets.
Based on the F1-score (which balances precision and recall), two features appear to represent the optimal feature set, as this provides the highest F1-score (99.23%); the highest recall (98.98%), which means the model identifies more true positives; the highest precision (99.49%), reducing the number of false positives; and the lowest number of false negatives (22 FNs). Thus, using two features gives the best balance between precision and recall, making it the most efficient feature set for this task.
The following table (Table 19) demonstrates the accuracy matrix of the model using all of the feature subsets.
The below figures (Figure 3 and Figure 4) display the accuracy and precision of the six classification algorithms and majority voting ensemble learning algorithm. The following table (Table 20) demonstrates the precision matrix of the model using all of the feature subsets.
The following table (Table 21) demonstrates the recall matrix of the model using all of the feature subsets.
Figure 5 shows the recall vs. number of features for different models for a peer-to-peer botnet detection system.
The following table (Table 22) demonstrates the F1-score matrix of the model using all of the feature subsets.
The figures above (Figure 3, Figure 4, Figure 5 and Figure 6) illustrate the performance of six classification algorithms (NB, KNN, DT, LDA, RF, and SVM) and the ensemble learning algorithm (mv) across all of the feature subsets.
In order to enhance the performance of the proposed method, we employed the majority voting algorithm. This involved taking the results obtained from the base classifiers mentioned earlier (as detailed in Table 18) and utilizing them as input for the majority voting algorithm. The output of this ensemble approach, depicted in Table 18, represents the final result of our methodology.
Our method attained an accuracy of 99.99% (Table 23), indicating that the overall predictions made by the classification model are highly accurate. While accuracy is crucial, especially in balanced datasets, it may not fully represent model performance in imbalanced class distributions. Therefore, we also evaluated our model using precision, recall, and F1-score metrics and we generated a confusion matrix as well. Our method achieved a precision of 99.49%, ensuring that a large proportion of instances predicted as positive are indeed relevant. With a recall value of 98.98%, our model effectively minimizes false negatives, correctly identifying a significant portion of positive instances. Furthermore, the F1-score, at 99.23%, signifies a strong balance between precision and recall, highlighting the robustness of our model’s performance.

6. Comparison

In this section, we present a comparison of results of our approach with the approaches described in the Related Work Section. Table 24 shows a comparison of our approach with the approaches discussed in the related work.
As shown in the above table (Table 24), no other work has displayed their result using accuracy, precision, recall, and F1-score, except Joshi et al. [22]. Some methods have displayed their result using only an accuracy performance metric, which is not enough to check the performance of a method. Some methods did not mention whether their dataset contained P2P bots or not. Some methods use IRC datasets, where P2P bot communication is not available. But our method is validated on a publicly available P2P benchmark dataset and it outperforms all other methods by achieving 99.99% acc, 99.49% pre, 98.98% rec (slightly less than other methods), and 99.23% F1-score.

7. Conclusions and Future Works

This paper introduces a novel botnet detection model leveraging the differential evolution algorithm, which is an evolutionary metaheuristic. Metaheuristic algorithms are general-purpose optimizers which can effectively handle a spectrum of complex problems, across various domains. They excel in tackling problems that are computationally complex and often involve a large solution space. They are particularly valuable for solving NP-hard (non-deterministic polynomial time hard) problems.
Our differential-evolution-based proposed model excels in selecting an optimal subset of features, effectively discerning botnet traffic from normal patterns. The outcomes demonstrate a marked enhancement in accuracy, precision, recall, and F1-score, achieved with a reduced set of features. For future endeavors, refining the model’s scalability and exploring its applicability across diverse datasets stand as promising avenues to further elevate its effectiveness and broaden its utility in real-world scenarios.

Author Contributions

Conceptualization, S.B., V.D., D.D., U.B. and M.J.S.; methodology, S.B., V.D. and U.B.; software, S.B., V.D., D.D. and U.B.; validation, U.B. and M.J.S.; formal analysis, S.B., V.D., D.D. and U.B.; investigation, M.J.S.; resources, V.D. and M.J.S.; data curation, S.B., V.D., D.D. and U.B.; writing—original draft preparation, S.B., V.D., D.D. and U.B.; writing—review and editing, M.J.S.; visualization, S.B., V.D., D.D. and U.B.; supervision, M.J.S.; project administration, U.B. and M.J.S. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Data Availability Statement

Required data is presented in the manuscript. Full data will be shared upon request.

Acknowledgments

The authors thank the Biomedical Sensors & Systems Lab, University of Memphis, Memphis, TN 38152, USA, for supporting this research and the article processing charges.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Oyelakin, A.M.; Jimoh, R.G. A survey of feature extraction and feature selection techniques used in machine learning-based botnet detection schemes. VAWKUM Trans. Comput. Sci. 2021, 9, 1–15. [Google Scholar] [CrossRef]
  2. Feily, M.; Shahrestani, A.; Ramadass, S. A survey of botnet and botnet detection. In Proceedings of the Third International Conference on Emerging Security Information, Systems and Technologies, Athens, Greece, 18–23 June 2009; pp. 268–273. [Google Scholar]
  3. Miller, S.; Busby-Earle, C. The role of machine learning in botnet detection. In Proceedings of the 11th International Conference for Internet Technology and Secured Transactions, Barcelona, Spain, 5–7 December 2016; pp. 359–364. [Google Scholar]
  4. Negera, W.G.; Schwenker, F.; Debelee, T.G.; Melaku, H.M.; Ayano, Y.M. Review of Botnet Attack Detection in SDN-Enabled IoT Using Machine Learning. Sensors 2022, 22, 9837. [Google Scholar] [CrossRef]
  5. Owen, H.; Zarrin, J.; Pour, S.M. A survey on botnets, issues, threats, methods, detection and prevention. J. Cybersecur. Priv. 2022, 2, 74–88. [Google Scholar] [CrossRef]
  6. Baruah, S.; Borah, D.J.; Deka, V. Detection of Peer-to-Peer Botnet Using Machine Learning Techniques and Ensemble Learning Algorithm. Int. J. Inf. Secur. Priv. 2023, 17, 1–16. [Google Scholar] [CrossRef]
  7. Pokhrel, S.; Abbas, R.; Aryal, B. IoT Security: Botnet Detection in IoT Using Machine Learning. arXiv 2021, arXiv:2104.02231. Available online: https://arxiv.org/pdf/2104.02231 (accessed on 27 May 2025).
  8. Alhijaj, T.B.; Hameed, S.M.; Bara’a, A.A. A decision tree-aware genetic algorithm for botnet detection. Iraqi J. Sci. 2021, 62, 2454–2462. [Google Scholar] [CrossRef]
  9. Shinan, K.; Alsubhi, K.; Ashraf, M.U. Botsward: Centrality measures for graph-based bot detection using machine learning. Comput. Mater. Contin. 2023, 74, 693–714. [Google Scholar] [CrossRef]
  10. Pabboju, G.R. A host-based P2P host identification approach with flow-based detection of P2P bots. OSF Preprints 2022. [Google Scholar] [CrossRef]
  11. Said, G.A.E.A.; Mahmoud, A.M.; El-Horbaty, E.M. A comparative study of meta-heuristic algorithms for solving quadratic assignment problem. Int. J. Adv. Comput. Sci. Appl. 2014, 5. [Google Scholar] [CrossRef]
  12. Sharma, M.; Kaur, P. A comprehensive analysis of nature-inspired meta-heuristic techniques for feature selection problem. Arch. Comput. Methods Eng. 2020, 28, 1103–1127. [Google Scholar] [CrossRef]
  13. Lin, K.C.; Chen, S.Y.; Hung, J.C. Botnet detection using support vector machines with artificial fish swarm algorithm. J. Appl. Math. 2014, 2014, 986428. [Google Scholar] [CrossRef]
  14. Liao, J.L.; Lin, K.C.; Hsu, J.Y. Botnet detection and feature analysis using backpropagation neural network with bio-inspired algorithms. Int. J. Cogn. Perform. Support 2018, 1, 132–142. [Google Scholar] [CrossRef]
  15. Asadi, M.; Jamali, M.A.J.; Parsa, S.; Majidnezhad, V. Detecting botnet by using particle swarm optimization algorithm based on voting system. Futur. Gener. Comput. Syst. 2020, 107, 95–111. [Google Scholar] [CrossRef]
  16. Alejandre, F.V.; Cortés, N.C.; Anaya, E.A. Feature selection to detect botnets using machine learning algorithms. In Proceedings of the CONIELECOMP, Cholula, Mexico, 22–24 February 2017; IEEE: New York, NY, USA, 2017; pp. 1–7. [Google Scholar]
  17. Farahmand-Nejad, A.; Noferesti, S. A real-time botnet detection model based on an efficient wrapper feature selection method. Int. J. Secur. Netw. 2020, 15, 36. [Google Scholar] [CrossRef]
  18. Yallamanda, R.B.; Gunnam, R.D.; Prasanna. Detection of Botnet and DDoS Using Network Traffic Analysis and Machine Learning Algorithms. RS. (Not Specified). 2023. Available online: https://www.researchsquare.com/article/rs-3397184/v1 (accessed on 27 May 2025).
  19. Joshi, C.; Bharti, V.; Ranjan, R.K. Analysis of feature selection methods for P2P botnet detection. Adv. Comput. Data Sci. 2020, 4, 272–282. [Google Scholar]
  20. Ismail, Z.; Jantan, A.; Yusoff, M.N. A Framework for Detecting Botnet Command and Control Communication over an Encrypted Channel. Int. J. Adv. Comput. Sci. Appl. 2020, 11, 1. [Google Scholar] [CrossRef]
  21. Muhammad, A.; Asad, M.; Javed, A.R. Robust early stage botnet detection using machine learning. In Proceedings of the 2020 International Conference on Cyber Warfare and Security (ICCWS), Islamabad, Pakistan, 20–21 October 2020; IEEE: New York, NY, USA, 2020; pp. 1–6. [Google Scholar]
  22. Joshi, C.; Ranjan, R.K.; Bharti, V. ANN based Multi-Class classification of P2P Botnet. Int. J. Comput. Digit. Syst. 2022, 11, 1319–1325. [Google Scholar] [CrossRef]
  23. Kemp, C.; Calvert, C.; Khoshgoftaar, T. Utilizing netflow data to detect slow read attacks. In Proceedings of the IEEE International Conference on Information Reuse and Integration, Salt Lake City, UT, USA, 6–9 July 2018; pp. 108–116. [Google Scholar]
  24. Jović, A.; Brkić, K.; Bogunović, N. A review of feature selection methods with applications. In Proceedings of the IEEE 38th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), Opatija, Croatia, 25–29 May 2015; pp. 1200–1205. [Google Scholar]
  25. Bianchi, L.; Dorigo, M.; Gambardella, L.M.; Gutjahr, W.J. A survey on metaheuristics for stochastic combinatorial optimization. Nat. Comput. 2009, 8, 239–287. [Google Scholar] [CrossRef]
  26. Differential Evolution. Available online: https://en.wikipedia.org/wiki/Differential_evolution (accessed on 27 December 2023).
  27. Storn, R.; Price, K.V. Differential Evolution—A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces. J. Global Optim. 1997, 11, 341–359. [Google Scholar] [CrossRef]
  28. Singh, D.; Kumar, V.; Vaishali; Kaur, M. Classification of COVID-19 patients from chest CT images using multi-objective differential evolution–based convolutional neural networks. Eur. J. Clin. Microbiol. Infect. Dis. 2020, 39, 1379–1389. [Google Scholar] [CrossRef]
  29. Kok, K.Y.; Rajendran, P. Differential-Evolution Control Parameter Optimization for Unmanned Aerial Vehicle Path Planning. PLoS ONE 2016, 11, e0150558. [Google Scholar] [CrossRef] [PubMed]
  30. Wang, J.; Xue, B.; Gao, X.; Zhang, M. A differential evolution approach to feature selection and instance selection. In PRICAI 2016: Trends in Artificial Intelligence, Proceedings of the14th Pacific Rim International Conference on Artificial Intelligence, Phuket, Thailand, 22–26 August 2016; Lecture Notes in Computer Science; Springer: Cham, Switzerland, 2016; pp. 588–602. [Google Scholar]
  31. Das, S.; Suganthan, P.N. Differential evolution: A survey of the state-of-the-art. IEEE Trans. Evol. Comput. 2011, 15, 4–31. [Google Scholar] [CrossRef]
  32. Alshamkhany, M.; Alshamkhany, W.; Mansour, M.; Dhou, S.; Aloul, F. Botnet Attack Detection using Machine Learning. In Proceedings of the IEEE International Conference on Innovations in Information Technology, Al Ain, United Arab Emirates, 17–18 November 2020. [Google Scholar]
  33. Atangana, R.; Tchiotsop, D.; Kenne, G.; Chanel, L. EEG signal classification using LDA and MLP classifier. Health Informat. Int. J. 2020, 9, 14–32. [Google Scholar] [CrossRef]
  34. Ryu, S. Comparison of Machine Learning Algorithms and Their Ensembles for Botnet Detection. Ph.D. Thesis, Purdue University, West Lafayette, IN, USA, 2018. [Google Scholar]
  35. García, S.; Grill, M.; Stiborek, J.; Zunino, A. An empirical comparison of botnet detection methods. Comput. Secur. 2014, 45, 100–123. [Google Scholar] [CrossRef]
  36. Dogan, A.; Birant, D. A weighted majority voting ensemble approach for classification. In Proceedings of the 2019 4th International Conference on Computer Science and Engineering (UBMK), Samsun, Turkey, 11–15 September 2019; IEEE: New York, NY, USA, 2019; pp. 1–6. [Google Scholar]
  37. Maz, Y.A.; Anbar, M.; Manickam, S.; Rihan, S.D.A.; Alabsi, B.A.; Dorgham, O.M. Majority voting ensemble classifier for detecting keylogging attack on internet of things. IEEE Access 2024, 12, 19860–19871. [Google Scholar] [CrossRef]
  38. Altulaihan, E.; Almaiah, M.A.; Aljughaiman, A. Anomaly Detection IDS for Detecting DoS Attacks in IoT Networks Based on Machine Learning Algorithms. Sensors 2024, 24, 713. [Google Scholar] [CrossRef]
  39. Delplace, A.; Hermoso, S.; Anandita, K. Cyber Attack Detection thanks to Machine Learning Algorithms. arXiv 2020, arXiv:2001.06309. [Google Scholar]
  40. Rainville, F.D.; Fortin, F.; Gardner, M.; Parizeau, M.; Gagné, C. DEAP: A python framework for evolutionary algorithms. In Proceedings of the Annual Conference on Genetic and Evolutionary Computation, Philadelphia, PA, USA, 7–11 July 2012. [Google Scholar]
  41. Algorithms—DEAP 1.4.1 Documentation. Available online: https://deap.readthedocs.io/en/master/api/algo.html (accessed on 2 February 2024).
  42. sklearn.naive_bayes.GaussianNB. Available online: https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.GaussianNB.html (accessed on 2 February 2024).
  43. sklearn.neighbors.KNeighborsClassifier. Available online: https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html (accessed on 2 February 2024).
  44. sklearn.discriminant_analysis.LinearDiscriminantAnalysis. Available online: https://scikit-learn.org/stable/modules/generated/sklearn.discriminant_analysis.LinearDiscriminantAnalysis.html (accessed on 2 February 2024).
  45. sklearn.tree.DecisionTreeClassifier. Available online: https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html (accessed on 2 February 2024).
  46. sklearn.ensemble.RandomForestClassifier. Available online: https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html (accessed on 2 February 2024).
  47. sklearn.svm.SVC. Available online: https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html (accessed on 2 February 2024).
  48. sklearn.ensemble.VotingClassifier. Available online: https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.VotingClassifier.html (accessed on 2 February 2024).
Figure 1. System workflow of proposed peer-to-peer botnet detection system.
Figure 1. System workflow of proposed peer-to-peer botnet detection system.
Futureinternet 17 00247 g001
Figure 2. Flow chart of differential evolution for peer-to-peer botnet detection system.
Figure 2. Flow chart of differential evolution for peer-to-peer botnet detection system.
Futureinternet 17 00247 g002
Figure 3. Graph showing the accuracy of the six classification algorithms and majority voting ensemble learning algorithm.
Figure 3. Graph showing the accuracy of the six classification algorithms and majority voting ensemble learning algorithm.
Futureinternet 17 00247 g003
Figure 4. Graph showing precision vs. number of features for different models for peer-to-peer botnet detection system.
Figure 4. Graph showing precision vs. number of features for different models for peer-to-peer botnet detection system.
Futureinternet 17 00247 g004
Figure 5. Recall of the six classification algorithms and majority voting.
Figure 5. Recall of the six classification algorithms and majority voting.
Futureinternet 17 00247 g005
Figure 6. F1-score vs. number features for different models for peer-to-peer botnet detection system.
Figure 6. F1-score vs. number features for different models for peer-to-peer botnet detection system.
Futureinternet 17 00247 g006
Table 1. Comprehensive summary of existing work on ML-based botnet detection with their advantages and disadvantages.
Table 1. Comprehensive summary of existing work on ML-based botnet detection with their advantages and disadvantages.
Sl. No.AuthorBotnet Type DetectedDataset UsedCategory of FS AlgorithmFS Algorithm UsedClassification AlgorithmPerformance MatrixDisadvantageAdvantage
1[13]IRCTestbedSwarm- and
evolution-based MH
AFSA, GASVMAcc—100%IRC botnet is used. Only accuracy is calculated for performance evaluation.Metaheuristic algorithm is used for FS.
2[14]IRCTestbedSwarm- and
evolution-based MH
PSO, GABPNAcc—95.09%IRC botnet is used. Only accuracy is calculated for performance evaluation.Metaheuristic algorithm is used for FS.
3[15]P2PISOTSwarm intelligence-based MHPSOMajority votingAcc—99.64%, Re—99.82%Precision and F1 are not calculated.Metaheuristic algorithm is used for FS.
4[16]P2P and IRCISOT and ISCXEvolution-based MHGADecision tree (C4.5)Re—99.46%Accuracy, precision, and F1 are not calculated.Metaheuristic algorithm is used for FS.
5[8]Not mentionedUNSW—NB15 and CICIDS2017Evolution-based MHGADecision treePre—99.17%, Re—96.85%Botnet type detected is not mentioned. Accuracy and F1 values are not provided.Metaheuristic algorithm is used for FS.
6[17]Not mentionedN-BaIoTHuman-behavior-based MHWCCSVMAcc—95%, Pre—95%,
Re—95%
IRC botnet is detected.Metaheuristic algorithm is used for FS.
7[18]Not mentionedCCIDS2017Filter methodPCA, LDANB, SVMAcc—99.68%Detection performed for centralized or decentralized botnet is not known.High accuracy.
8[19]P2PCTU-13 Filter methodPCA, Univariate Selection, RFE, Feature Importance, Correlation Heat MapSVM, logistic regression, KNN, decision treeAcc—99%,
AUC—99%,
AR—99%
Lots of FS algorithms are compared but all belong to the filter methods.High accuracy. P2P detection is performed.
9[20]P2PISOT, MCFP and NIMSFilter methodInformation GainSVM, Naïve Bayes, K-Nearest Neighbor, Bayesian Network, Ada Boost, Random Forest, J48, PART and Ripper Not mentionedInformation about evaluation matrices is not given.FS is conducted and three different datasets are used.
10[21]Not mentionedCyber Clean Center (CCC) datasetFilter methodPCA and Information GainLogistic Regression, SVM, Random Forest and Multilayer Perceptron Acc—97.8%
TPR—97%
FPR—0.7%
Detection conducted for centralized or decentralized botnet is not known.High accuracy.
11[22]P2PMCPFFilter methodRecursive Feature Selection, PCA, Correlation Matrix, Univariate Selection, Feature ImportanceANNAcc—99.04%
Pre—99.35%
Re—99.35%
F1—99.21%
Lots of FS algorithms are compared but all belong to the same category.High accuracy. P2P detection is conducted.
Table 2. Network traffic details of Scenario 12 of CTU-13 dataset.
Table 2. Network traffic details of Scenario 12 of CTU-13 dataset.
ScenarioDuration (hrs)Number of PacketsNumber of NetFlow RecordsSize (GB)Bot FamilyNumber of Bot
121.2113,212,2681,262,7908.3NSIS.ay3
Table 3. Feature descriptions of Scenario 12 of the CTU-13 dataset.
Table 3. Feature descriptions of Scenario 12 of the CTU-13 dataset.
Feature NumberFeature NameDescription
F1StartTimeDate of the flow start (from 2011/08/10 10:02:43 to 2011/08/19 11:45:43)
F2DurDuration of the communication in seconds
F3ProtoTransport protocol used
F4SrcAddrIP address of the source
F5SportSource port
F6DirDirection of the flow
F7DstAddrIP address of the destination
F8DportDestination port
F9StateTransaction state
F10sTosSource TOS byte value
F11dTosDestination TOS byte value
F12TotPktsTotal number of transaction packets
F13TotBytesTotal number of transaction bytes
F14SrcBytesTotal number of transaction bytes from the source
F15Label/targetLabel made of “flow=” followed by a short description (source/destination-malware/application)
Table 4. Values for differential evolution parameters used in the experiments for peer-to-peer botnet detection system.
Table 4. Values for differential evolution parameters used in the experiments for peer-to-peer botnet detection system.
Parameter NameV1V2V3V4V5V6V7V8V9V10
ngen2030405060708090100120
popsize2030405060708090100120
cxpb0.10.20.30.40.50.60.70.80.91.0
mutpb0.030.040.050.060.070.080.090.10.20.3
Table 5. Count of selected “ngen” value for the experiments of peer-to-peer botnet detection system.
Table 5. Count of selected “ngen” value for the experiments of peer-to-peer botnet detection system.
Count of Selected “ngen” Value
V1V2V3V4V5V6V7V8V9V10
2445564947
Table 6. Count of selected “popsize” value for the experiments of botnet detection system.
Table 6. Count of selected “popsize” value for the experiments of botnet detection system.
Count of Selected “Popsize” Value
V1V2V3V4V5V6V7V8V9V10
2566665374
Table 7. Count of selected “cxpb” value for the experiments of peer-to-peer botnet detection.
Table 7. Count of selected “cxpb” value for the experiments of peer-to-peer botnet detection.
Count of Selected “cxpb” Value
V1V2V3V4V5V6V7V8V9V10
10344468362
Table 8. Count of selected “mutpb” value for the experiments of peer-to-peer botnet detection.
Table 8. Count of selected “mutpb” value for the experiments of peer-to-peer botnet detection.
Count of Selected “mutpb” Value
V1V2V3V4V5V6V7V8V9V10
7472357829
Table 9. Optimal values of differential evolution parameters for the experiments of botnet detection.
Table 9. Optimal values of differential evolution parameters for the experiments of botnet detection.
Parameter NameOptimal Value
Ngen90
Popsize100
Cxpb0.1
Mutpb0.3
Table 10. Key parameters and optimal values of Gaussian Naïve Bayes algorithm for peer-to-peer botnet detection system.
Table 10. Key parameters and optimal values of Gaussian Naïve Bayes algorithm for peer-to-peer botnet detection system.
Parameter NameOptimal Value
var_smoothing0.1
priorsNone
Table 11. Key parameters and optimal values of KNN algorithm for peer-to-peer botnet detection system.
Table 11. Key parameters and optimal values of KNN algorithm for peer-to-peer botnet detection system.
Parameter NameOptimal Value
weightsDistance
p1 (manhattan_distance)
n_neighbors3
algorithmAuto
Table 12. Key parameters and optimal values of LDA algorithm for peer-to-peer botnet detection system.
Table 12. Key parameters and optimal values of LDA algorithm for peer-to-peer botnet detection system.
Parameter NameOptimal Value
tol0.001
store_covarianceTrue
solverEigen
shrinkage0.1
priorsNone
n_components1
Table 13. Key parameters and optimal values of DT algorithm for peer-to-peer botnet detection system.
Table 13. Key parameters and optimal values of DT algorithm for peer-to-peer botnet detection system.
Parameter NameOptimal Value
splitterBest
random_state42
min_samples_split7
min_samples_leaf9
max_featuresNone
max_depth10
criterionEntropy
Table 14. Key parameters and optimal values of RF algorithm for peer-to-peer botnet detection system.
Table 14. Key parameters and optimal values of RF algorithm for peer-to-peer botnet detection system.
Parameter NameOptimal Value
n_estimators50
min_samples_split5
min_samples_leaf6
max_featuresSqrt
max_depth15
bootstrapTrue
Table 15. Key parameters and optimal values of SVM algorithm for peer-to-peer botnet detection system.
Table 15. Key parameters and optimal values of SVM algorithm for peer-to-peer botnet detection system.
Parameter NameOptimal Value
tol0.0001
random_state42
shrinkingTrue
probabilityFalse
max_iter100
kernelLinear
gamma1000.0
degree1
oef00.19999999996
class_weightNone
C100.0
Table 16. Key parameters and optimal values of majority voting algorithm for peer-to-peer botnet detection system.
Table 16. Key parameters and optimal values of majority voting algorithm for peer-to-peer botnet detection system.
Parameter NameOptimal Value
Estimatorlda, knn, dt, rf, nb, svm
VotingSoft
Weights[1, 2, 2, 2, 1, 1]
Table 17. Count of selected features for peer-to-peer botnet detection system.
Table 17. Count of selected features for peer-to-peer botnet detection system.
Count of Selected Features for Peer-to-Peer Botnet Detection System
F1F2F3F4F5F6F7F8F9F10F11F12F13F14
42718114933221064
Table 18. Majority voting ensemble result using all feature subsets for peer-to-peer botnet detection.
Table 18. Majority voting ensemble result using all feature subsets for peer-to-peer botnet detection.
No. of FeaturesAccuracyPrecisionRecallF1-ScoreTPTNFPFN
299.9999.4998.9899.232132322,7521122
399.9999.3098.6098.952124322,7481530
499.9899.6296.8098.192085322,755869
599.9899.5696.8098.162085322,754969
699.9799.5296.1597.802071322,7531083
799.9799.5695.5497.512058322,754996
899.9799.5795.6897.592061322,754993
999.9899.5796.8498.192086322,754968
1099.9899.3496.8498.12086322,7501368
1199.9899.4896.8098.122085322,7521169
1299.9799.3396.5697.932080322,7491474
1399.9799.3396.4397.862077322,7491477
1499.9799.3896.6197.982081322,7501373
Table 19. Accuracy matrix of the model using all feature subsets for peer-to-peer botnet detection.
Table 19. Accuracy matrix of the model using all feature subsets for peer-to-peer botnet detection.
Number of Features
Model234567891011121314
LDA99.3499.3499.2899.1299.1299.3399.3299.3299.3099.2599.2699.2699.26
KNN99.9899.9799.9499.9499.8899.8899.8899.8899.8499.8499.8499.8499.84
DT99.9899.9899.9899.9899.9899.9799.9799.9799.9799.9799.9799.9799.97
RF99.9999.9999.9999.9999.9999.9999.9899.9999.9999.9999.9899.9899.98
NB99.3899.3494.2694.4188.5286.9986.9943.8451.6552.6953.0253.0253.00
SVM99.3499.3899.6099.6099.3499.3499.3499.3499.3499.3499.3499.3499.34
MV99.9999.9999.9899.9899.9799.9799.9799.9899.9899.9899.9799.9799.97
Table 20. Precision matrix of the model using all feature subsets.
Table 20. Precision matrix of the model using all feature subsets.
Number of Features
Model234567891011121314
LDA000.510.990.992.414.174.179.096.425.575.565.56
KNN97.3197.0894.7794.7790.6091.1291.1191.2287.4587.4587.3787.3787.37
DT98.1297.9898.0297.8897.8397.6197.7097.8397.4397.3997.2697.0397.39
RF99.5399.3099.5899.5899.6799.6799.7699.8199.8199.7699.8699.7699.81
NB002.904.054.073.613.611.161.341.371.381.381.38
SVM0079.2778.89000000000
MV99.4999.3099.6299.5699.5299.5699.5799.5799.3499.4899.3399.3399.38
Table 21. Recall matrix of the model using all feature subsets for peer-to-peer botnet detection.
Table 21. Recall matrix of the model using all feature subsets for peer-to-peer botnet detection.
Number of Features
Model234567891011121314
LDA000.050.320.320.050.090.090.600.970.740.740.74
KNN99.3098.7095.8295.8291.3290.9590.9591.1888.3088.3088.0288.0288.02
DT99.3099.2198.8098.7998.4898.5198.4798.3398.5698.5698.7098.7098.70
RF99.3099.2198.9898.6598.6598.1997.6898.2497.9698.1497.6397.6397.73
NB0023.5432.7372.2872.5672.5699.0399.0799.1299.0799.0799.07
SVM0054.6954.83000000000
MV98.9898.6096.8096.8096.1595.5495.6896.8496.8496.8096.5696.4396.61
Table 22. F1-score matrix of the different models using all feature subsets for peer-to-peer botnet detection system.
Table 22. F1-score matrix of the different models using all feature subsets for peer-to-peer botnet detection system.
Number of Features
Model234567891011121314
LDA000.090.490.490.090.180.181.131.691.311.311.31
KNN98.3097.8995.2995.2990.9691.0391.0391.2087.8787.8787.7087.7087.70
DT98.7098.5998.4098.3498.1598.0698.0898.0897.9997.9797.9797.8698.04
RF99.4199.2699.2899.1199.1698.9298.7199.0298.8898.9498.7398.6998.76
NB005.167.217.706.896.892.282.642.702.722.722.72
SVM0064.7364.69000000000
MV99.2398.9598.1998.1697.8097.5197.5998.1998.198.1297.9397.8697.98
Table 23. Result of the proposed approach for peer-to-peer botnet detection system.
Table 23. Result of the proposed approach for peer-to-peer botnet detection system.
Accuracy (%)Precision (%)Recall (%)F1-Score (%)TPTNFPFN
99.9999.4998.9899.232132322,7521122
Table 24. Comparison of our approach with the approaches presented in related work.
Table 24. Comparison of our approach with the approaches presented in related work.
Sl. No.AuthorBotnet Type DetectedAccuracyPrecisionRecallF1
1[13]IRC100%Not mentionedNot mentionedNot mentioned
2[14]IRC95.09%Not mentionedNot mentionedNot mentioned
3[15]P2P99.64%Not mentioned99.82%Not mentioned
4[16]P2P and IRCNot mentionedNot mentioned99.46%Not mentioned
5[8]Not mentionedNot mentioned99.17%96.85%Not mentioned
6[17]Not mentioned95%95%95%Not mentioned
7[18]Not mentioned99.68%Not mentionedNot mentionedNot mentioned
8[19]P2P99%Not mentionedNot mentionedNot mentioned
9[20]P2PNot mentionedNot mentionedNot mentionedNot mentioned
10[21]Not mentioned97.8%Not mentioned97%Not mentioned
11[22]P2P99.04%99.35%99.35%99.21%
12Our approachP2P99.99%99.49%98.98%99.23%
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Baruah, S.; Deka, V.; Das, D.; Barman, U.; Saikia, M.J. Enhanced Peer-to-Peer Botnet Detection Using Differential Evolution for Optimized Feature Selection. Future Internet 2025, 17, 247. https://doi.org/10.3390/fi17060247

AMA Style

Baruah S, Deka V, Das D, Barman U, Saikia MJ. Enhanced Peer-to-Peer Botnet Detection Using Differential Evolution for Optimized Feature Selection. Future Internet. 2025; 17(6):247. https://doi.org/10.3390/fi17060247

Chicago/Turabian Style

Baruah, Sangita, Vaskar Deka, Dulumani Das, Utpal Barman, and Manob Jyoti Saikia. 2025. "Enhanced Peer-to-Peer Botnet Detection Using Differential Evolution for Optimized Feature Selection" Future Internet 17, no. 6: 247. https://doi.org/10.3390/fi17060247

APA Style

Baruah, S., Deka, V., Das, D., Barman, U., & Saikia, M. J. (2025). Enhanced Peer-to-Peer Botnet Detection Using Differential Evolution for Optimized Feature Selection. Future Internet, 17(6), 247. https://doi.org/10.3390/fi17060247

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop