You are currently viewing a new version of our website. To view the old version click .
Electronics
  • Article
  • Open Access

9 March 2022

Effectiveness Evaluation of Different IDSs Using Integrated Fuzzy MCDM Model

,
,
,
,
,
,
,
and
1
Department of Computer Science, College of Computers and Information Technology, Taif University, P.O. Box 11099, Taif 21944, Saudi Arabia
2
Department of Information Technology, Babasaheb Bhimrao Ambedkar University, Lucknow 226025, Uttar Pradesh, India
3
Department of Information Technology, College of Computers and Information Technology, Taif University, P.O. Box 11099, Taif 21944, Saudi Arabia
4
Department of Computer Science and Engineering, Babu Banarasi Das University, Lucknow 226028, Uttar Pradesh, India
This article belongs to the Special Issue Advances on Networks and Cyber Security

Abstract

Cyber-attacks are becoming progressively complicated; hence, the functional issues of intrusion-detection systems (IDSs) present ever-growing challenges. Failing to detect intrusions may jeopardize the trustworthiness of security services, such as privacy preservation, authenticity, and accessibility. To fight these risks, different organizations nowadays use a variety of approaches, techniques, and technologies to safeguard the systems’ credibility. Establishing policies and procedures, raising user awareness, implementing firewall and verification systems, controlling system access, and building computer-issue management groups are all examples of safeguarding methods. There is a lack of sufficient emphasis on the effectiveness of intrusion-detection systems. In enterprises, IDS is used to analyze the potentially dangerous activities taking place within the technological settings. The selection of efficient IDS is a challenging task for organizations. This research evaluates the impact of five popular IDSs for their efficiency and effectiveness in information security. The authors used the fuzzy analytical hierarchy process (AHP) and fuzzy technique for order performance by similarity to ideal solution (TOPSIS)-based integrated multi-criteria decision-making (MCDM) methodology to evaluate the efficacy of the popular IDSs. The findings of this research suggest that most of the IDSs appear to be highly potential tools. Even though Snort is extensively deployed, Suricata has a substantial advantage over Snort. Suricata uses multi-threading functionality in comparison to Snort to boost the processing performance.

1. Introduction

Communication networks are an integral part of our lives in the digital age. They are a privilege to the digital world. They introduce the rest of the world relatively close to all of us. The challenge of intrusion became much more prevalent with the network. Hackers are always present in the virtual environment. Business organizations could be vulnerable to cyber-attacks. It is necessary to take significant action to analyze network attacks and return them to normal. As a result, intrusion-detection systems (IDS) are important in a network security solution. IDS is a solution that detects network attacks. IDS assists anyone in detecting network traffic. IDS can further initiate an immediate alert. This will assist the IT workforce in dealing with such challenges. However, it would not stop a threat from impacting system applications. It analyzes daily network activities by using the guidelines. The security management system receives an alert through IDS in case any malicious activity is detected over the network. As a result, it is easier for the security expert to identify such suspected activities. The traffic information is moved in bulk and then analyzed for any unusual behavior in the data. The team explores threats by using known attack signatures, as well as trends [1,2,3,4,5,6].
IDS is a mechanism that detects discrepancies in capturing attackers before they cause serious damage to the network system. On the user’s computer, a host-based IDS is installed. The network is monitored by a network-based IDS. During the normal assignment, IDSs search for signatures from attack patterns. These abnormalities are reported to the knowledge base and, after that, analyzed at the guidelines and application layers based on the policy and procedures. IDS could be used as software or as a network security device. Since the IDSs only need to identify the risks, these systems are positioned outside of band on the communications infrastructure, indicating that the transmitter and the recipient of data do not communicate in real-time. IDS systems commonly use a Test Access Point (TAP) or Switch Port Analyzer (SPAN) port for the analysis of a copy of the Inline Traffic Stream, so that IDS does not affect the performance of the Inline Network. This has been established initially by IDS, since the required intrusion analysis at the time cannot be conducted at a pace that is consistent with the elements on the network infrastructure’s direct communication line [7,8,9,10]. Figure 1 shows the general architecture of IDS.
Figure 1. General architecture of intrusion-detection system (IDS).
An efficient IDS and mitigation systems are important for an organization’s normal system functioning. Traditional defense technology has become increasingly inefficient as a result of attacker tactics, such as obfuscation techniques, metamorphism, and polymorphism, which increase malware’s resistance. Discrepancies are detected by the IDSs to capture attackers that can cause significant harm to the organization. They could be a network or even a host. Due to new attacks developing almost daily, IDSs are crucial in identifying and responding to potential system intrusions. IDSs should change and continually adjust to all of these new threats and assault techniques. The challenge on which scholars have been researching for decades is how to construct effective, efficient, and responsive IDSs [11,12,13,14,15].
The evaluation of IDSs is a popular and challenging task in research at present. Choosing an efficient IDS to secure a business network must not be undertaken lightly, rapidly, or without a thorough grasp of the technology, solutions, and potential consequences. The decision-making process can be broken down into several stages, which include determining the requirement, gaining a broad awareness of IDSs, more in-depth knowledge of the network, and also determining policy and processes by evaluating various IDS solutions [16]. This paper evaluates the effectiveness of different IDSs, using the integrated fuzzy analytical hierarchy process (AHP) technique for order performance by similarity to the ideal solution (TOPSIS) model.
The remainder of this paper is structured as follows: The study outlines relevant research efforts in this domain in Section 2. Section 3 discusses the overview of several types of popular IDSs. Section 4 discusses the integrated fuzzy AHP-TOPSIS technique. Section 5 of this work contains the findings, comparisons of the findings, and a sensitivity analysis. Section 6 closes with the summary and conclusions.

3. Different Types of Intrusion-Detection Systems

3.1. Zeek

Zeek [22,23] is a network intrusion-detection system that runs on Unix (IDS). Zeek analyzes network traffic and recognizes intrusion attempts, depending on the type and substance of the traffic. Zeek IDS was previously known as Bro. Zeek identifies intrusions by matching network traffic to rules that describe undesirable events. These rules may define activities (for example, certain hosts interacting to specific services), what actions warrant alerting (for example, attempts to a particular number of distinct hosts constitute a “scan”), or signatures characterizing known attacks or exposure to security issues. If Zeek discovers anything of interest, the direction could be to either write a log entry or run an operating system function. Zeek is designed for high-speed (Gbps) and increased intrusion detection. Zeek IDS can perform competently while operating on commercially accessible PC hardware by intelligently exploiting packet filtering methods, and so can provide an economically efficient way of monitoring a site’s Internet communication.

3.2. Suricata

Suricata was designed in 2010 by the OISF (Open Information Security Foundation), with funding from the US Department of Homeland Security [24]. Suricata’s design is quite close to snort’s, except that, instead of using a single thread to process packets, Suricata uses many threads [25]. This enables Suricata’s distinctive feature, which is to maximize the capacity to obtain packets. Snort was single-threaded; therefore, when packets exceeded the capacity to obtain bandwidth, Snort disregarded them. Thus, multi-threading is a useful capability of Suricata. Suricata has many detection algorithm threads.

3.3. Security Onion

Security Onion [26] is a Linux-based intrusion-detection approach that includes several IDS that are both host-oriented (HIDS) and network-oriented (NIDS). Security Onion can collect and analyze an extensive range of data. This comprises information on the host, connection, session, resource, alerting, and standards. Security Onion can be deployed as a solo implementation with a server and detector, or with a master server and many detectors that allow the platform to be expanded as needed. Numerous gateways and tools are present for system management and information analysis, including Sguil, Snorby, Squert, and Enterprise Log Search and Archive (ELSA). It offers host-based identification in the form of OSSEC HIDS, as well as network-based detection via Snort, Suricata, and Zeek NIDS. Security Onion is particularly customizable because it may be set as a master server with numerous sensors or as an independent or hybrid installation. The information gathered by Security Onion is saved in log files and also in a Sguil database, which includes a new interface for recording and analysis.

3.4. OSSEC

OSSEC is an open-source intrusion-detection system built by Daniel B. Cid, who sold the product to Trend Micro in 2008 [27]; however, the project remained a free and open source. The most recent stable release is 2.9.3. It is made up of numerous services and modules, each of which has its own distinct set of intrusion-detection capabilities. HIDS has numerous elements, and OSSEC combines them all to provide certain fundamental advantages. OSSEC ensures that security compliance standards are met. Many consumers, primarily corporate clients, demand that the companies with whom they do business have particular security compliances, including Payment Card Industry Data Security Standard (PCI DSS), Health Insurance Portability and Accountability Act (HIPAA), and so on. Analyzing logs and evaluating them for suspicious activity is a one-way OSSEC that helps firms comply with various security standards [28].

3.5. Snort

Snort fills an essential “ecological niche” in network security by serving as a cross-platform lightweight network intrusion-detection program that can be used to monitor tiny TCP/IP networks, as well as discover a wide range of abnormal network traffic and also explicit cyberattacks. It can offer controllers adequate information to make informed judgments on how to proceed in the event of suspicious behavior. Snort could also be quickly implemented to cover any gaps in a network’s protection measures, such as when a big threat occurs and corporate security companies take their time releasing new attack identification signatures. Snort is beneficial when it is not cost-effective to install commercial NIDS sensors. Current commercialized intrusion-detection systems cost thousands or tens of thousands, or perhaps even hundreds of thousands of dollars in extreme situations [29,30]. Snort was created to encounter the requirements of a typical compact network intrusion-detection system. It has progressed into a compact, versatile, and high-performing technology that is used on both large and small networks around the globe. It has met its initial design aims and is a perfectly capable solution to corporate intrusion-detection systems in locations where installing standard production systems would be prohibitively expensive.
Table 1 shows the comparison of the five popular intrusion-detection methods.
Table 1. Comparison of the five intrusion-detection methods.

4. Methods

4.1. Identification of Evaluation Criteria and Alternatives

MCDM is a discipline of combinatorial optimization in which the alternatives are evaluated to identify the best alternative that meets a set of various and frequently contradictory parameters. MCDM is a critical component of the decision-making principle and operational investigation. It is frequently regarded as trustworthy. It is a set of strategies and techniques for integrating various and contradictory parameters into a decision-making process. Furthermore, MCDM might be regarded as a systematic approach for evaluating and selecting between possibilities. It seeks to divide an issue into smaller components, analyze and evaluate each part, and then aggregate those components to determine the appropriate reasonable alternative from a range of options based on a stated set of parameters. In uncertain, unclear, fuzzy, or risky contexts, MCDM tries to help decision-makers to tackle contradictory real-world statistical and/or qualitative subjective multi-criteria challenges, and to select best-fit options from a group of options. Figure 2 shows the hierarchy for the assessment of some popular intrusion-detection systems (IDS) in this research study. Based on an analysis of the relevant literature, as well as insight from seventy-seven security experts, the four considerable factors at level one, as well associated sub-factors at level two, in the current method that make a significant contribution to the assessment of multiple IDSs were evidently identified and designed. The four main criteria to evaluate the different popular IDSs are Types, Audit source location, Targets, and Protected system denoted as M1, M2, M3, and M4 respectively. The significant criterion Types contain the different types of IDSs. The IDSs may be open-source, closed-source, or freeware, denoted by M11, M12, and M13, respectively. Different IDSs are classified by the type of input data they evaluate at the audit source location. Audit procedures on a host log file, network packets, application log files, or sensor alerts denoted by M21, M22, M23, and M24, respectively, produced by other IDSs can all be used as input data. The targets’ criterion shows the IDS capability to detect potential attacks against specific targets. These targets may be application, network, or host, as denoted by M31, M32, and M33, respectively. The protected system criteria show the approach of intrusion detection. It can be HIDS, NIDS, or hybrid types denoted by M41, M42, and M43, respectively. The five alternatives, namely Zeek, Suricata, Security Onion, OSSEC, and Snort, are represented by S1, S2, S3, S4, and S5, respectively.
Figure 2. Hierarchy for the effectiveness evaluation of different IDSs.

4.2. Fuzzy AHP-TOPSIS Methodology

Problems with decision-making are likely a consequence of putting too much emphasis on analogical reasoning that has worked previously. When attempting to make a judgment or decision, researchers use a predictive model, which is a heuristic algorithm or guiding principle. These strategies can help in making better decisions by reducing our cognitive strain, and they can also lead to mistakes. However, AHP is unable to resolve the fundamental uncertainty and inaccuracies in a decision-maker’s response to genuine statistical information. Because the real world is so indistinct, researchers noted that experts have combined the fuzzy theory with AHP to investigate obscure real-world problems [31,32,33]. Furthermore, while the AHP method is typically based on a highly volatile scale of decisions, the fuzzy AHP does have deficiencies [23,24]. As a result, a combined AHP and TOPSIS fuzzy method is a special process that could aid in the efficacious evaluation of options. Moreover, the fuzzy AHP-TOPSIS technique is as follows:
Fuzzy AHP: Fuzzy AHP is a popular methodology for resolving difficult selection challenges. Every complicated topic can be investigated by using remarkable categorized ranges of objectives, i.e., hierarchy. With the help of fuzzy AHP, the problem is differentiated into a tree form to describe it. Figure 2 shows how to make a tree shape. This tree shape was created with the help of experts’ opinions [34]. The triangular fuzzy number (TFN) is then constructed from the hierarchical structure. A pair-wise comparison of each group of categorized goals is critical because of the impact of one criterion on other criteria.
Transforming linguistic numbers into crisp numbers, as well as TFN, is the next step. The TFN is used in this research, and it ranges from 0 to 1 [35]. The quantitative simplification of TFN membership functions, as well as their capabilities to handle with fuzzy data, is driving this implementation [23]. In addition, linguistic values are categorized as equally important or weakly important, and crisp values are categorized as 1, 2, …, 9. Furthermore, if the membership functions of a fuzzy number M on F are recognized, it is referred to as TFN:
µ a   ( x ) = a [ 0 , 1 ]
µ a ( x ) = { x m i l l m i l x [ l , m i ] x m i u u m i u x [ m i , u ]
In the triangular membership function, l, mi, and u represent the lower, middle, and upper limits, respectively. A TFN is shown in Figure 3.
Figure 3. Triangular fuzzy numbers.
A TFN can be written as (l, mi, u). Specialists assigned numerical scores to the elements that influence the values using a measure that is highlighted in Table 2.
Table 2. TFN scale.
The numerical methods (3–6) are used to convert numeric values into TFNs that are identified as (lij, miij, and uij), where lij is lower significance, miij is middle significance, and uij is uppermost significance events. Moreover, TFN (ɳij) is represented as follows:
Φ i j = ( l i j ,   m i i j ,   u i j )
where l i j   m i i j   u i j
l i j = m i n ( J i j d )
m i i j = ( J i j 1 , J i j 2 , J i j 3 ) 1 x
a n d       u i j = m a x ( J i j d )
In Equations (3)–(6), Jijd specifies the proportional position of the values among two elements that is quantified by specialist d, where i and j indicate a pair of elements being decided by specialists. Φij is calculated according to the geometric mean of a specialist’s observations for a particular assessment. The geometric mean is capable of appropriately joining and demonstrating the consent of specialists and designates the lowest and highest marks, respectively, for the relative consequence among the two elements. Additionally, Equations (7)–(9) provide the joint TFN values. Take into account the two TFNs, M1 = (l1, mi1, u1) and M2 = (l2, mi2, u2). The procedures of actions on them are as follows:
( l 1 ,   m i 1 ,   u 1 ) + ( l 2 ,   m i 2 ,   u 2 ) = ( l 1 + l 2 ,   m i 1 + m i 2 ,   u 2 + u 2 )
( l 1 ,   m i 1 ,   u 1 ) × ( l 2 ,   m i 2 ,   u 2 ) = ( l 1 × l 2 ,   m i 1 × m i 2 ,   u 1 × u 2 )
( l 1 , m i 1 , u 1 ) 1 = ( 1 u 1 , 1 m i 1 , 1 l 1 )
With the support of the equation, a fuzzified pair-wise comparative matrix in the state of n × n matrix is formed after acquiring the TFN variables for every pair of comparisons (10).
A d ˜ = [ k ˜ 11 d k ˜ 12 d     k ˜ 1 n d k ˜ 21 d k ˜ 22 d     k ˜ 2 n d         k ˜ n 1 d k ˜ n 2 d k ˜ n n d ]
where k i j k ˜ symbolizes the dth decision-makers’ priority of the ith standards over the jth criteria. If more than one responsible party is prevalent, the equation is used to calculate the average of every judgment-maker’s priorities (11).
k ˜ i j = d = 1 d . k ˜ i j d
The next step is to use the equation to modify the pair-wise comparative matrixes for all elements in the hierarchical order based on the averaged priorities (12).
A ˜ = k 11 ˜     k 1 n ˜         k n 1 ˜     k ˜ n n  
The fuzzy geometrical mean and fuzzy weights of each element are then described by using the geometrical mean method, as demonstrated in Equation (13).
p ˜ i = ( j = 1 n . k ˜ i j ) 1 n ,   i = 1 ,   2 ,   3   , n
The next step is to use the equation to calculate the identified element’s fuzzy weight (14).
w ˜ i = p ˜ i ( p ˜ 1 p ˜ 2 p ˜ 3     p ˜ n ) 1
Equations are also used to determine the average and normalized weight criteria (15–16).
M i = w ˜ 1 w ˜ 2     w ˜ n n
N r i = M i M 1 M 2     M n
Additionally, the Centre of Area (COA) method is used to estimate the BNP (best non-fuzzy performance) number of the fuzzy weights of each quantity with the support of Equation (17).
B N P w D 1 = [ ( u w 1 l w 1   ) + ( m i w 1 l w 1   ) ] 3 + l w 1
Fuzzy TOPSIS: TOPSIS sees multi-standard decision-making problems, with m options as a geometrical configuration with m points inside the n-dimensional problem area. The strategy used in this article for TOPSIS is principally founded on the notion that a designated possibility is the relatively short and farthest range from the positive-ideal solution, as well as the negative-ideal solution, correspondingly, for best possible and lowest ideal solutions [36]. Specialists have a difficult time assigning a particular performance rating to an alternative based on criteria. TOPSIS uses fuzzy numbers rather than precise numerals to demonstrate the relative importance of criteria in order to be consistent with the actual fuzzy situation. Furthermore, the fuzzy AHP-TOPSIS method is most suitable for handling group decision-making issues in fuzzy settings. The following is the fuzzy AHP-TOPSIS methodology:
The first step is to determine the weights of the evaluation criteria. With the support of equations, this research utilized fuzzy AHP to arrive at fuzzy choice weights (1–16). Moreover, with support of Equation (18), as well as Table 3, the researcher designed the fuzzy decision matrix and selected the best linguistic aspects as criteria possibilities.
C 1       C n K ˜ = A 1 A m [ x ˜ 11 x ˜ 1 n x ˜ m 1 x ˜ m n ]
where x ˜ i j = 1 D ( x ˜ i j 1     x ˜ i j d     x ˜ i j D ) , and x ˜ i j d is the assessment of the alternative Ai in terms of factor Cj effectiveness estimated by the dth specialist, as well as x ˜ i j d = ( l i j d , m i i j d ,   u i j d ).
Table 3. Linguistic scales for the rating.
The subsequent step is to use an expression to normalize the fuzzy decision matrix (19). The normalized fuzzy assessment matrix denoted by P ˜ is represented as follows.
P ˜ = [ p ˜ i j ] m × n
After that, using equations, one may complete the normalizing procedure by using Equation (20).
p ˜ i j = ( l i j u j + , m i i j u j + , u i j u j + ) ,   u j + = m a x { u i j ,   i = 1 ,   2 ,   3 , n }
Otherwise, we can establish the highest anticipated level, u j + , and j = 1, 2, …, n is equivalent to 1; or else, the worst is 0. Furthermore, the normalized p ˜ i j remains to be TFNs. For TFNs, the normalization procedure can be achieved in an analogous way. The subjective fuzzy normalized assessment matrix ( Q ˜ ) is measured with the support of Equation (21).
Q ˜ = [ q ˜ i j ] m × n i = 1 , 2 ,   , m ;   j = 1 ,   2 ,   3   , n
where q ˜ i j = p ˜ i j w ˜ i j also, at that point, describes the Fuzzy Positive-Ideal Solution (FPIS), as well as Fuzzy Negative-Ideal Solution (FNIS). The subjective normalized fuzzy assessment matrix specifies that the components q ˜ i j are normalized positive TFN and their choices fit in to the closed interval [0, 1]. Afterward, we can designate the FPIS A+ (aspiration levels) and FNIS A (the worst levels), as presented in Equations (22) and (23).
A + = ( q ˜ 1 ,       * q ˜ j ,       * q ˜ n , * )
A = ( q ˜ 1 ,       * q ˜ j ,       * q ˜ n , * )
where q ˜ 1 * = ( 1 , 1 , 1 )     w ˜ i j = ( L w j , M w j , H w j ) and q ˜ i j = ( 0 , 0 , 0 ) , j = 1 ,   2 ,   3   , n . For computing the space of every alternative solution from FPIS, as well as FNIS, the spaces ( d ˜ i + a n d   d ˜ i ) of every alternative solution from A+ and A can be assessed with the area compensation procedure, as presented in Equations (24) and (25).
d ˜ i + = j = 1 n . d ( q ˜ i j , q ˜ i j * ) i = 1 , 2 ,   , m ;   j = 1 , 2 , 3   , n
d ˜ i = j = 1 n . d ( q ˜ i j , q ˜ i j * ) i = 1 , 2 ,   , m ;   j = 1 , 2 , 3   , n
The next stage is to identify the closeness coefficients (absolute gaps’ degree) and construct the alternative solutions for achieving the aspiration degrees in each element. Ying-Chyi Chou et al. recommended that C C ˜ i is cleared to estimate the point of the fuzzy gap based on the fuzzy closeness coefficients to increase the alternatives solution [37]. Once the d ˜ i + and d ˜ i of every alternative solution is assessed, the comparisons to the ideal clarification are computed. This procedure resolves the equation’s similarity to the best choice (26).
C C ˜ i = k ˜ i k ˜ i + + k ˜ i = 1 k ˜ i + k ˜ i + + k i   ,   i = 1 , 2 ,   , m
where k ˜ i k ˜ i + + k ˜ i is demarcated as the fuzzy satisfaction amount in the ith alternative solution, and k ˜ i + k ˜ i + + k ˜ i is the fuzzy-gap amount in the ith alternative solution. Based on the ranks of alternatives, the solution is accomplished. The subsequent step is to evaluate the various IDSs by using their contributing qualities.

5. Numerical Data Analysis

5.1. Statistical Findings

It should be necessary to evaluate the effectiveness of different intrusion-detection systems [33,34]. A condition that is hard to ascertain is unsuitable for the job. Prohibitively expensive evaluations still would not yield the anticipated outcomes. The gap of cost-benefit analysis is advantageous toward a better assessment, as long as it includes the necessary standards for reliability and bias. A common method for estimating effectiveness is to evaluate the IDS numerous times and then determine a “significance level of measurement”. An independent evaluation is better than one that is biased. Measuring bias, on the other hand, is challenging. In this research, we used an integrated fuzzy AHP-TOPSIS based methodology to evaluate the different popular IDSs. First, seventy-seven researchers and cybersecurity specialists with different IDS experiences were consulted for each parameter set and relevant technologies. These 77 decision-makers are composed of 30 cybersecurity specialists with more than 12 years of experience and 47 researchers from various security organizations with 10 years of IDS research experience. The specialists were required to submit and assess their viewpoints in a collaborative online setting, and they were given information on the degree of the variables in relation to the various groups, as well as linguistic standards. To begin, the decision-maker creates a pair-wise comparison matrix containing the parameters. The decisions are considered valid, since the consistency ratio (CR) is less than 0.1 or near to it. Table 4 demonstrates the combined fuzzify pair-wise assessment matrix at the initial first level. The fuzzy-aggregated pair-wise comparison matrix at the second level for Types, Audit Source Location, Targets, and Protected System is presented in Table 5, Table 6, Table 7 and Table 8. For every second-level phase, the global weights were calculated. The statistical findings were tabularized in Table 9, Table 10, Table 11, Table 12 and Table 13. Table 14 shows the overall weights and ranking of different factors. Table 15 shows the evaluators’ subjective intelligence findings in linguistic terminology, and Table 16 shows the fuzzy-decision matrix with normalized decisions. Furthermore, Table 17 demonstrates the fuzzy-decision matrix with weighted normalization. In addition, with the support of the hierarchical structure, Table 18 and Figure 4 illustrate the comprehensive and the ultimate relative closeness of the different alternatives.
Table 4. Level 1 fuzzify consolidated pair-wise comparative matrix.
Table 5. Pair-wise fuzzy multi-criteria decision comparative matrix for types.
Table 6. Pair-wise fuzzy multi-criteria decision comparative matrix for audit source location.
Table 7. Pair-wise fuzzy multi-criteria decision comparative matrix for targets.
Table 8. Pair-wise fuzzy multi-criteria decision comparative matrix for protected system.
Table 9. Pair-wise comparative matrix with defuzzification and local weight of parameters at level 1.
Table 10. Defuzzified pair-wise comparative matrix with parameter local weight for types.
Table 11. Defuzzified pair-wise comparative matrix with parameter local weight for audit source location.
Table 12. Defuzzified pair-wise comparative matrix with parameter local weight for targets.
Table 13. Defuzzified pair-wise comparative matrix with parameter local weight for protected system.
Table 14. Overall weights and level of different factors.
Table 15. Evaluators’ subjective intelligence findings in linguistic terminology.
Table 16. Fuzzy decision matrix with normalized decisions.
Table 17. Fuzzy decision matrix with weighted normalization.
Table 18. Coefficients of closeness to the aspired level throughout the alternatives.
Figure 4. Satisfaction Degree of CC-i.
Depending upon the value of satisfaction degree shown in Figure 4, it is concluded that the initial ranking of the efficient IDS using the integrated fuzzy AHP-TOPSIS approach is S2 > S5 > S3 > S4 > S1 (\>“means\higher to”). Therefore, S2, which is Suricata, is considered the preferable and effective IDS.

5.2. Comparative Analysis

The use of multiple techniques on almost the same data delivers contradictory conclusions. The investigators use a variety of complementary ways to test the correctness of the research outcome. In this investigation, the researcher applied a hybrid fuzzy AHP-TOPSIS-based approach to evaluate the effectiveness of different alternatives. The data-acquisition and -assessment process for that sample in fuzzy AHP-TOPSIS is similar to the traditional AHP-TOPSIS approach. As a result, fuzzification and defuzzification are required for the fuzzy-AHP-TOPSIS. Furthermore, the data for fuzzy AHP-TOPSIS are captured in their initial statistical values and afterward transformed into fuzzy value sets. The outcomes obtained through the traditional method were well correlated with those obtained by using the fuzzy procedure. The comparison analysis’ results were not highly different and varied from each other, but the accuracy of the outcomes was varied. The traditional approaches for the effectiveness evaluation of different IDSs are insufficient for working with the inaccurate or ambiguous quality of linguistic evaluations. Therefore, integrated fuzzy multi-criteria decision-making strategies are developed to tackle this challenge. Moreover, the fuzzy approach confirms the findings of the classic strategy, thus enhancing the validity of the ranking among the five methods. Table 19 and Figure 5 show the comparison of outcomes with the fuzzy and traditional AHP-TOPSIS approach.
Table 19. Comparison of the results of classical and fuzzy AHP-TOPSIS methods.
Figure 5. Comparison of the results with traditional approach.

5.3. Sensitivity Analysis

The responsiveness was evaluated by altering the parameters to test the validity of the gathered data. During the same statistical analysis, the sensitivity of the obtained weights (factors involved) was evaluated. Throughout the investigation, at the final (second) step, 13 variables were picked so that sensitivities can be investigated by using 12 experiments. In each trial, the degree of satisfaction (CC-i) was found by adjusting the weights of every parameter, while the weight of the other element remained stable, using the Fuzzy-AHP-TOPSIS approach. Reported consequences are shown in Table 20 and Figure 6.
Table 20. Sensitivity analysis.
Figure 6. Sensitivity analysis.
Table 18 represents the real weights of this research investigation in the first row. Taking the performances into account, we see that the alternative (S2) has a high level of satisfaction (CC-i). Twelve experiments were carried out. The results reveal that, after twelve tests, Alternative-2 (S2) consistently retains a high level of satisfaction (CC-i). The performance results demonstrate that the rankings of the alternatives are weight-dependent.

6. Conclusions

In today’s modern environment, network security is significant for small, as well as large, organizations. Modern security challenges that are increasingly complex and advanced are always being developed. Online security issues are becoming more prevalent, and an IDS can assist in defending your organization from malicious activities. An IDS simply analyzes network traffic and notifies the system administrator if any suspicious activity is detected. This paper discusses the five most widely used intrusion-detection systems. Thereafter, the functionality of these five popularly used intrusion-detection systems, including Zeek, Suricata, Security Onion, OSSEC, and Snort, are effectively compared with the help of a hybrid fuzzy-logic-based approach and discussed in this study. All of the IDSs appear to be highly potential tools. Even though Snort is extensively deployed, it is important to note that Suricata is the new-generation multi-threaded application with a broader range of features. Suricata also has a significant benefit over Snort in that it does not need many processes to handle an upsurge in network traffic. The type of connection or attack to be monitored will determine the implementation of a multispectral IDS. The recommendation to network administrators is based on a generic network design and frequent network obstructions; however, this may vary depending on the network infrastructure of the business entity. The target for future research work is to deploy similar research with other MCDM models. The real-world dataset would be reviewed, and final assessments would be easily measured without the assistance of any other third entity.

Author Contributions

Conceptualization, H.A., M.T.J.A., A.A. (Abdullah Alharbi), W.A. and M.A.; methodology, H.A., M.T.J.A., A.A. (Abdullah Alharbi), A.A. (Alka Agrawal) and R.K.; software, M.T.J.A., A.A. (Alka Agrawal) and R.K.; validation, M.T.J.A., A.A. (Abdullah Alharbi), A.A. (Alka Agrawal) and R.K.; formal analysis, H.A., M.T.J.A., A.A. (Alka Agrawal) and R.A.K.; investigation, H.A., M.T.J.A., A.A. (Abdullah Alharbi), W.A., M.A., D.P., A.A. (Alka Agrawal), R.K. and R.A.K.; resources, H.A., M.T.J.A., A.A. (Abdullah Alharbi), W.A. and M.A.; data curation, M.T.J.A., A.A. (Abdullah Alharbi), A.A. (Alka Agrawal) and R.K.; writing—original draft preparation, M.T.J.A., A.A. (Abdullah Alharbi), W.A., M.A., D.P., A.A. (Alka Agrawal) and R.A.K.; writing—review and editing, M.T.J.A., A.A. (Abdullah Alharbi), W.A., D.P., A.A. (Alka Agrawal) and M.A.; visualization, M.T.J.A., A.A. (Abdullah Alharbi), W.A., M.A., D.P. and R.A.K.; supervision, H.A., M.T.J.A., A.A. (Alka Agrawal) and R.A.K.; project administration, H.A., A.A. (Abdullah Alharbi), W.A. and R.A.K.; funding acquisition, H.A., A.A. (Abdullah Alharbi), W.A. and R.A.K. All authors have read and agreed to the published version of the manuscript.

Funding

The project was funded by Researchers Supporting project number (TURSP-2020/306), Taif University, Taif, Saudi Arabia.

Acknowledgments

This research was supported by Taif University Researchers Supporting Project number (TURSP-2020/306), Taif University, Taif, Saudi Arabia.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Sen, S. A survey of intrusion detection systems using evolutionary computation. In Bio-Inspired Computation in Telecommunications; Morgan Kaufmann: Burlington, MA, USA, 2015; pp. 73–94. [Google Scholar]
  2. Ansari, M.T.J.; Pandey, D.; Alenezi, M. STORE: Security threat oriented requirements engineering methodology. J. King Saud Univ.-Comput. Inf. Sci. 2018, 34, 191–203. [Google Scholar] [CrossRef]
  3. Ali, M.; Shiaeles, S.; Bendiab, G.; Ghita, B. MALGRA: Machine learning and N-gram malware feature extraction and detection system. Electronics 2020, 9, 1777. [Google Scholar] [CrossRef]
  4. Swann, M.; Rose, J.; Bendiab, G.; Shiaeles, S.; Savage, N. A comparative study of traffic generators: Applicability for malware detection testbeds. J. Internet Technol. Secur. Trans. 2021, 8, 705–713. [Google Scholar] [CrossRef]
  5. Shiaeles, S.N.; Papadaki, M. FHSD: An improved IP spoof detection method for web DDoS attacks. Comput. J. 2015, 58, 892–903. [Google Scholar] [CrossRef] [Green Version]
  6. Ansari, T.J.; Pandey, D. An Integration of Threat Modeling with Attack Pattern and Misuse Case for Effective Security Requirement Elicitation. Int. J. Adv. Res. Comput. Sci. 2017, 8, 16–20. [Google Scholar]
  7. Ansari, M.T.J.; Baz, A.; Alhakami, H.; Alhakami, W.; Kumar, R.; Khan, R.A. P-STORE: Extension of STORE methodology to elicit privacy requirements. Arab. J. Sci. Eng. 2021, 46, 8287–8310. [Google Scholar] [CrossRef]
  8. Alosaimi, W.; Ansari, M.T.J.; Alharbi, A.; Alyami, H.; Seh, A.H.; Pandey, A.K.; Agrawal, A.; Khan, R.A. Evaluating the Impact of Different Symmetrical Models of Ambient Assisted Living Systems. Symmetry 2021, 13, 450. [Google Scholar] [CrossRef]
  9. Ji, H.; Wang, Y.; Qin, H.; Wang, Y.; Li, H. Comparative performance evaluation of intrusion detection methods for in-vehicle networks. IEEE Access 2018, 6, 37523–37532. [Google Scholar] [CrossRef]
  10. Magán-Carrión, R.; Urda, D.; Díaz-Cano, I.; Dorronsoro, B. Towards a reliable comparison and evaluation of network intrusion detection systems based on machine learning approaches. Appl. Sci. 2020, 10, 1775. [Google Scholar] [CrossRef] [Green Version]
  11. Hussein, S.M. Performance evaluation of intrusion detection system using anomaly and signature based algorithms to reduction false alarm rate and detect unknown attacks. In Proceedings of the 2016 International Conference on Computational Science and Computational Intelligence (CSCI), Las Vegas, NV, USA, 15–17 December 2016; pp. 1064–1069. [Google Scholar]
  12. Nikolopoulos, K.; Punia, S.; Schäfers, A.; Tsinopoulos, C.; Vasilakis, C. Forecasting and planning during a pandemic: COVID-19 growth rates, supply chain disruptions, and governmental decisions. Eur. J. Oper. Res. 2021, 290, 99–115. [Google Scholar] [CrossRef]
  13. Alhakami, W.; Binmahfoudh, A.; Baz, A.; Alhakami, H.; Ansari, M.T.J.; Khan, R.A. Atrocious Impinging of COVID-19 Pandemic on Software Development Industries. Comput. Syst. Sci. Eng. 2021, 36, 323–338. [Google Scholar] [CrossRef]
  14. Ansari, M.T.J.; Khan, N.A. Worldwide COVID-19 Vaccines Sentiment Analysis through Twitter Content. Electron. J. Gen. Med. 2021, 18, em329. [Google Scholar] [CrossRef]
  15. Attaallah, A.; Ahmad, M.; Ansari, M.T.J.; Pandey, A.K.; Kumar, R.; Khan, R.A. Device security assessment of Internet of healthcare things. Intell. Autom. Soft Comput. 2020, 27, 593–603. [Google Scholar] [CrossRef]
  16. Upendran, V.; Gopinath, R. Feature selection based on multi-criteria decision making for intrusion detection system. Management 2021. [Google Scholar] [CrossRef]
  17. Hu, Q.; Yu, S.Y.; Asghar, M.R. Analysing performance issues of open-source intrusion detection systems in high-speed networks. J. Inf. Secur. Appl. 2020, 51, 102426. [Google Scholar] [CrossRef]
  18. Imoize, A.L.; Oyedare, T.; Otuokere, M.E.; Shetty, S. Software intrusion detection evaluation system: A cost-based evaluation of intrusion detection capability. Commun. Netw. 2018, 10, 211–229. [Google Scholar] [CrossRef] [Green Version]
  19. Saber, M.; Belkasmi, M.G.; Chadli, S.; Emharraf, M.; El Farissi, I. Implementation and Performance Evaluation of Intrusion Detection Systems under high-speed networks. In Proceedings of the 2nd International Conference on Big Data, Cloud and Applications, Tetouan, Morocco, 29–30 March 2017; pp. 1–6. [Google Scholar]
  20. Shiaeles, S.N.; Katos, V.; Karakos, A.S.; Papadopoulos, B.K. Real time DDoS detection using fuzzy estimators. Comput. Secur. 2012, 31, 782–790. [Google Scholar] [CrossRef] [Green Version]
  21. Schrötter, M.; Scheffler, T.; Schnor, B. Evaluation of Intrusion Detection Systems in IPv6 Networks. In Proceedings of the 16th International Joint Conference on e-Business and Telecommunications (ICETE 2019), Prague, Czech Republic, 26–28 July 2019; pp. 408–416. [Google Scholar]
  22. Haas, S.; Sommer, R.; Fischer, M. Zeek-osquery: Host-network correlation for advanced monitoring and intrusion detection. In IFIP International Conference on ICT Systems Security and Privacy Protection; Springer: Cham, Switzerland, 2020; pp. 248–262. [Google Scholar]
  23. Paxson, V. Bro: A system for detecting network intruders in real-time. Comput. Netw. 1999, 31, 2435–2463. [Google Scholar] [CrossRef]
  24. Garcia-Teodoro, P.; Diaz-Verdejo, J.; Maciá-Fernández, G.; Vázquez, E. Anomaly-based network intrusion detection: Techniques, systems and challenges. Comput. Secur. 2009, 28, 18–28. [Google Scholar] [CrossRef]
  25. Park, W.; Ahn, S. Performance comparison and detection analysis in snort and suricata environment. Wirel. Pers. Commun. 2017, 94, 241–252. [Google Scholar] [CrossRef]
  26. Burks, D. Security Onion. 2012. Available online: Securityonion.blogspot.com (accessed on 17 January 2022).
  27. Cid, D.B. Log Analysis Using OSSEC. 2017. Available online: http://www.academia.edu/8343225/Log_Analysis_using_OSSEC (accessed on 17 January 2022).
  28. Anafcheh, A. Intrusion Detection with OSSEC. 2018. Available online: https://www.theseus.fi/bitstream/handle/10024/150030/ali-anafcheh-thesis.pdf?sequence=1 (accessed on 17 January 2022).
  29. Roesch, M. Snort: Lightweight intrusion detection for networks. In Proceedings of the LISA ’99: 13th Systems Administration Conference, Seattle, WA, USA, 7–12 November 1999; Volume 99, pp. 229–238. [Google Scholar]
  30. Saaty, T.L. How to make a decision: The analytic hierarchy process. Eur. J. Oper. Res. 1990, 48, 9–26. [Google Scholar] [CrossRef]
  31. Hwang, C.L.; Yoon, K. Methods for multiple attribute decision making. In Multiple Attribute Decision Making; Springer: Berlin/Heidelberg, Germany, 1981; pp. 58–191. [Google Scholar]
  32. Chen, S.J.; Hwang, C.L. Fuzzy multiple attribute decision making methods. Fuzzy Mult. Attrib. Decis. Mak. 1992, 375, 289–486. [Google Scholar]
  33. Ansari, M.T.J.; Al-Zahrani, F.A.; Pandey, D.; Agrawal, A. A fuzzy TOPSIS based analysis toward selection of effective security requirements engineering approach for trustworthy healthcare software development. BMC Med. Inform. Decis. Mak. 2020, 20, 236. [Google Scholar] [CrossRef] [PubMed]
  34. Rose, J.R.; Swann, M.; Bendiab, G.; Shiaeles, S.; Kolokotronis, N. Intrusion Detection using Network Traffic Profiling and Machine Learning for IoT. In Proceedings of the 2021 IEEE 7th International Conference on Network Softwarization (NetSoft), Tokyo, Japan, 28 June–2 July 2021; pp. 409–415. [Google Scholar] [CrossRef]
  35. Kumar, R.; Khan, A.I.; Abushark, Y.B.; Alam, M.M.; Agrawal, A.; Khan, R.A. An integrated approach of fuzzy logic, AHP and TOPSIS for estimating usable-security of web applications. IEEE Access 2020, 8, 50944–50957. [Google Scholar] [CrossRef]
  36. Kumar, R.; Khan, A.I.; Abushark, Y.B.; Alam, M.M.; Agrawal, A.; Khan, R.A. A knowledge-based integrated system of hesitant fuzzy set, ahp and topsis for evaluating security-durability of web applications. IEEE Access 2020, 8, 48870–48885. [Google Scholar] [CrossRef]
  37. Abushark, Y.B.; Khan, A.I.; Alsolami, F.J.; Almalawi, A.; Alam, M.M.; Agrawal, A.; Kumar, R.; Khan, R.A. Usability Evaluation through Fuzzy AHP-TOPSIS Approach: Security Requirement Perspective. CMC-Comput. Mater. Contin. 2021, 68, 1203–1218. [Google Scholar] [CrossRef]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.