A Machine Learning-Based Hybrid Encryption Approach for Securing Messages in Software-Defined Networking

Abstract

The security of a network is based on the foundation of confidentiality, integrity, and availability, often referred to as the CIA triad. The privacy of data over a network, maintained by confidentiality, has long been one of the major issues in network settings. With the decoupling of the data plane and control plane in the software-defined networking (SDN) environment, this challenge is significantly amplified. This paper aims to address the challenges of confidentiality in SDN by introducing a genetic algorithm-based hybrid encryption network policy to secure messages across the network. The proposed approach achieved an average entropy of 0.989, revealing a significant improvement in the strength of the encryption with the hybrid mechanism. However, the method exhibited processing overhead, significantly increasing the transmission time for encrypted messages compared to unencrypted transmission. Compared to standalone AES, DES, and RSA, this approach shows better encryption randomness, but a trade-off between security and network performance is evident in the absence of load-balancing techniques.

1. Introduction

The evolving and developing landscape of network security is significantly shaped by the emergence of software-defined networking (SDN), which decouples the data and control planes, enabling dynamic network reconfiguration. Although doing so provides flexibility, scalability, and efficiency, the programmability of SDN also presents different challenges for security [1,2,3,4], especially in the face of the rising deployment of SDN over critical infrastructures and the increasing number of IoT applications. These trends underscore the urgent need for advanced encryption methods capable of addressing the unique demands of modern SDN environments.

While centralized control in SDN allows for agile management and adaptation to evolving security needs, it also centralizes the attack surface, making the data plane a prime target for adversaries and increasing vulnerabilities to targeted breaches [3,5,6]. The data plane remains susceptible to packet interception, which could compromise data confidentiality and integrity [2]. Encryption is crucial in secure data transmission to avoid unauthorized access and tampering [7]. However, the dynamic nature of SDN and its diverse requirements for a variety of modern applications make it difficult for classical encryption to meet standard demands reliably in SDN environments [8]. Traditional cryptographic techniques may not be sufficient to overcome all aspects of complexity in SDN [9]. There are also method-dependent aspects that may prove to be resource-intensive, face scalability challenges, and be exposed to broad attack vectors targeting the data plane; they further rely on service providers, and this reliance might diminish the network security [10]. These limitations create a significant gap in developing secure and efficient encryption methods suitable for the dynamic settings of SDN.

This study addresses the limitations created by primary threats, including external attackers that intercept packets in the data plane and adversaries that employ statistical linear approximation and differential techniques to recover information about the underlying cryptographic structure. However, our approach does not explicitly address advanced persistent threats (APTs) or quantum attacks. While APTs or quantum attacks represent significant threats, they require special countermeasures such as post-quantum cryptography and AI-driven anomaly detection, which lie outside the scope of this work. Furthermore, APTs usually explore the vulnerabilities in SDN controllers rather than their encryption mechanisms. The study also does not address attacks that exploit vulnerabilities in the hardware implementations of SDN devices or DDoS or DoS attacks that target SDN controllers or vulnerabilities in control plane protocols.

This study presents a comprehensive analysis of five critical domains: encryption methodologies and network security within traditional networks, encryption and security protocols tailored for SDN, policy implementation strategies within SDN, genetic algorithm-based encryption mechanisms, and data flow management in SDN environments. Building on these insights, we propose an innovative encryption scheme that leverages genetic algorithms (GAs) in conjunction with RSA to enhance payload encryption as a policy to uphold the confidentiality of messages, specifically within the SDN environment.

The novelty and contribution of the proposed method lie in its deployment of a genetic algorithm to provide efficient and secure encryption as a network-wide policy, thereby reducing the need for service provider-dependent or paid solutions. Furthermore, it achieves strong resistance to multiple vectors of cryptanalytic attacks by optimizing multi-objective functions that balance ciphertext length, unpredictability, and decryption accuracy. The application of evolutionary concepts with multiple criteria for key generation improves resilience to reverse-engineering attempts, whereas the dynamic evolution of session-specific keys combined with RSA provides strong security, even in the presence of partial system exposure. The contributions of this study are summarized below:

• GA-based hybrid encryption approach as a network-wide policy is implemented.
• A comparative performance analysis with AES, DES, and RSA in terms of cryptographic strength is carried out.
• A discussion of the trade-off between security and network performance and strategies are presented.

The rest of this paper is organized as follows: Section 2 provides the background and related work in the proposed study domain. The overall research is designed along with the methodology in Section 3. Section 4 provides the detailed experimental analysis with results, experiments, and evaluations, while Section 5 concludes this paper with limitations to be addressed in future work.

2. Background and the Related Work

2.1. Background

The integration of encryption techniques into SDN has attracted significant attention due to the increasing demand for robust security frameworks [11]. The evolution of digital communication requires advanced encryption techniques. The decoupling of the control and data planes in SDN provides centralized network management, scalability, and flexibility. However, the inherent architecture introduces vulnerabilities, which require robust encryption mechanisms to protect data integrity and confidentiality.

Traditional algorithms such as the advanced encryption standard (AES), the data encryption standard (DES), and Rivest–Shamir–Adleman (RSA), have limitations. AES and DES are vulnerable to brute-force attacks when these algorithms are implemented as standalone encryption techniques, and RSA is impractical for large datasets due to their complexity [12,13]. Hybrid encryption, which combines the strengths of both symmetric and asymmetric cryptography, is more suitable for high-security applications because it balances security and efficiency.

Naseemuddin Mohammad [14] reviewed encryption techniques, access control, governance, compliance, and threat mitigation in multi-cloud environments. The author highlighted methodologies such as secure multiparty computation and ISO/IEC 27001 certification for regulatory compliance and cyber risk mitigation.

Atdoga et al. [15] highlighted that encryption practices are motivated by certain regulatory frameworks. On one hand, the U.S., with its federal and state laws, finds some common balance between security and privacy, influenced by the standards and sector-specific regulations established by the National Institute of Standards and Technology (NIST). On the other hand, the European Union insists through general data protection regulation (GDPR) that privacy is a fundamental right protected by the European Union Agency for Cybersecurity (ENISA) standards.

Fauri et al. [16] illustrated that encryption maintains the confidentiality of data in an industrial control system network, but it creates difficulties in troubleshooting. Hence, industrial control system security should prioritize availability and integrity with an emphasis on authentication and integrity over complete encryption. Alves et al. [17] concluded that combining AES-256 encryption with an ML-based intrusion prevention system (IPS) on open-source programmable logic controller platforms increases protection from cyber threats.

Pothireddy et al. [18] presented an innovative hybrid approach that combines fully homomorphic encryption and secure hash algorithm-3 (SHA-3) to enhance data confidentiality, integrity, and processing efficiency. It tackles key management challenges and vendor limitations in cloud environments.

Yan et al. [19] highlighted the advances in attribute-based searchable encryption for enhancing security, efficiency, and flexibility in encrypted data searching. This includes policy hiding, computational outsourcing, and online/offline mechanisms, while bringing to light the integration of traceability, retractability, and future directions such as quantum attack resilience and parallel computing.

Legacy networking has limitations with respect to the implementation of encryption mechanisms, key management, and resistance to vulnerabilities. Addressing these limitations in a traditional setup is complicated. In contrast, SDN offers a better prospect for implementing robust techniques to counter these challenges by virtue of its dynamic and programmability nature.

2.2. Related Works

Researchers have proposed a variety of solutions, ranging from advanced encryption techniques, security policy frameworks, and the use of genetic algorithms for cryptographic innovations. This section explores these works, focusing on their methodologies, strengths, and limitations. The study of related works is organized into three segments: (a) encryption and network security in SDN, (b) security policy deployment in SDN, and (c) the GA-based encryption approach.

2.2.1. Encryption and Network Security in SDN

Durner and Kellerer [20] examined the alleged impact of transport layer security (TLS) encryption on OpenFlow performance over SDN. In their research, they pointed out the trade-offs between security and performance vis-a-vis TLS adoption and elaborated on its consequences on the packet-in delays across multiple SDN switch models.

Chen et al. [21] presented a new scheme separating multicast control from data transmission to provide higher capacity and robustness for multicast applications over SDN. This scheme used conventional cryptography and not one based on homomorphism, which improved efficiency and reduced the complexity of computations.

In order to secure an SDN system, Ghaly and Abdullah [22] implemented hybrid encryption algorithms, which combined AES, RSA, and hybrid approaches, offering greater security and efficiency than RSA alone.

Alzahrani and Chaudhry [23] proposed an identity-based encryption method for source routing systems enabled by SDN. The authors claimed to improve the security and path verification while keeping the size of the authenticator fixed and supported multiple recipients and parallel links.

Abdi et al. [24] presented a review regarding traditional, AI-based, and moving target defense security methods for addressing challenges and improving security over SDN. The abovementioned studies represent meaningful progress in SDN security. However, they lack insight into how resilient their methods will be against a comparatively wide variety of attacks that involve using cryptanalysis to extract information from the underlying cryptographic structure.

2.2.2. Security Policy Deployment in SDN

Pisharody et al. [25] introduced Brew, a security policy analysis framework for distributed SDN-based cloud environments. It detects and resolves security policy conflicts across several layers so that implementation and effectiveness is consistent.

Dong et al. [26] put forth a new design for an SDN switch that works by discovering users automatically based on their behavior and deployed a personalized network policy. This demonstrates efficient network security management through traffic pattern analysis using k-means clustering for user classification conducted by the SDN controller.

Syed et al. [27] introduced “DEPO”, a framework for evaluating and managing policy deployment in software-defined infrastructure (SDI), utilizing knowledge modeling, data analysis, machine learning, and emulation techniques to assess the impact of service-level policies on SDI orchestration and configuration within test-bed environments.

Ahmed et al. [28] conducted an empirical study that outlines techniques to identify and analyze SDN policy parameters and network topology for enhancing security by preventing adversaries from exploiting data plane vulnerabilities through empirical latency-based approaches and configuration analysis. Qazi et al. [29] introduced “SIMPLE”, an SDN-based solution facilitating efficient middle-box-specific traffic steering validated through implementation using a POX controller.

Although the proposed frameworks offered significant contributions to policy analysis, personalized policy deployment, and traffic management in SDN, they lack a comprehensive approach to integrating encryption as policies and addressing challenges with the interception of packets that occurs during transmission in data planes.

2.2.3. GA-Based Encryption

Sindhuja and Pramela [30] introduced a GA-based symmetric key encryption technique utilizing substitution, genetic crossover, mutation, and key operations, for example, right shift and matrix addition for robust data transmission security, with a focus on simplicity, reliability, and effective encryption and decryption, using two-point crossover techniques.

Naik and Naik [31] introduced a Java implementation of asymmetric key encryption with a genetic algorithm, focused on the importance of the crossover and mutation points for enhancing algorithmic effectiveness in the creation of the asymmetric key pair through a random process and application of a permutation technique for robustness, due to which 36-bit key was able to produce very strong encryption and decryption, and assurance against breaching was used.

Arshad et al. [32] presented an advanced encryption technique using a customized GA with enhanced operators and local intelligence and achieved at least 80% greater efficiency in key generation while maintaining randomness compared to conventional GA methods. The study presented the algorithm customization, key parameter manipulation, binary data conversion, and result comparison. This contributes valuable insights into optimizing cryptographic security protocols.

Shafai et al. [33] proposed a new framework that works with GA encryption to generate multiple encrypted cancelable biometric images. Compared with conventional algorithms, it shows the optical scanning holography algorithm on different biometric databases. It is expected to have better biometric security and better privacy preservation with lower risks of unauthorized access.

Alhassan et al. [34] introduced an advanced audio encryption and decryption technique using an enhanced GA. This integrates fission and fusion into traditional operators for robust keyspace and key parameter sensitivity. This effectively secures audio data while maintaining size and showcasing the method’s success and limitations, suggesting future steganography applications.

Mawla and Khafaji [35] suggested a better method for concealing encrypted messages that makes use of protein motifs. By using genetic algorithms, logic gates, and protein motifs—which had several metrics that outperformed DNA-based algorithms, such as blindness, functional conservation, and capacity—the authors demonstrated encryption and steganography for enhanced data security and confidentiality. These works show how GA-based encryption can be used in both symmetric and asymmetric encryptions. The studies, however, do not explore GA’s integration into SDN, where encryption must consider several criteria in addition to decryption accuracy, and instead focus on specific use cases with a single criterion of maintaining decryption accuracy.

With SDN placing a strong focus on network security and encryption regulations, ideally managed by the SDN controller, the literature review shows that encryption techniques varied with respect to applications and compliance with various requirements. GA was highlighted for the flexibility it brought to the encryption of various data types together with parameter optimization, contributing to security. The implementation of GA-based hybrid encryption as an SDN policy remains under-explored regarding its impact on the encryption strength and effect of network execution.

3. Methodology

This section describes the research design along with the method of implementation of a GA-based hybrid encryption policy in SDN. The experimental and quantitative research techniques used here implement and evaluate a GA-based hybrid encryption solution implemented in SDN. The experiments were conducted in a Mininet-based SDN environment. Mininet offers a robust platform for testing and evaluating SDN security policies by creating a network of virtual switches, hosts, and controllers. This allows for controlled experimentation and systematic assessment of encryption efficiency and performance. The implementation of encryption as a policy process begins with formulating the encryption algorithm. This algorithm is implemented with the POX controller in a way that it enforces the algorithm as a policy over the network, resulting in policy establishment. When the topology designed in Mininet is connected to the controller, and messages are transmitted between hosts, the controller enforces the designed policy. We evaluate the algorithm’s effectiveness and network performance based on the metrics illustrated in Figure 1.

3.1. Development of Hybrid Encryption Technique

The hybrid encryption technique is accomplished by developing an encryption algorithm that uses multiple levels of encryption and decryption. Specifically, text data are encrypted in two steps. The initial encryption uses a GA-based technique, followed by a second layer of encryption utilizing the RSA algorithm, as shown in Figure 2.

GA-Based Encryption: Initially, a random set of keys is generated. Each key will be assessed by encoding and decoding the original message using the XOR function. The fitness criteria for generating a symmetric encryption key include a multi-objective function to balance three major parameters, viz., encoded message length and XOR efficiency, randomness in ciphertext, and key diversity. Keys generated with uniqueness over encryption sessions or packets are rewarded. This prevents the reuse of keys, enhanced forward secrecy, and ensures that if one key is compromised, the other sessions remain secure. In addition, the fitness function verifies that the keys produce ciphertext with a high degree of randomness, which is assessed by minimizing the correlation between the plaintext and ciphertext. While these functions remain a part of the fitness criteria, the selection of keys only occurs once lossless decoding criteria are satisfied. Each key is evaluated based on its ability to encode and decode the original message using the XOR function, thereby maintaining strong obfuscation through XOR operations. The goal is to minimize the length of the encoded message while retaining the true decryption. The sequential process performed over multiple generations of selection, crossover, and mutation organizes the population to find the best key. The key for encoding the message is cryptographic and is further secured by RSA encryption, thus providing extra security. This ensures confidentiality even if the XOR key is compromised, as presented in Algorithm 1 and shown in Figure 2.

Algorithm 1 Genetic algorithm-based encryption process.

1: Initialization:   2: Define population size n.   3: Generate a random population of potential keys $\mathrm{population}=\{k_1,k_2,\dots ,k_n\}$ where $k_i\in \{0,1,\dots ,255\}$.   4: Evaluation:   5: for each key $k_i$ in the population do   6:     Encode and decode the original message M using XOR with $k_i$.   7:     Calculate fitness score $f\left(k_i\right)={\sum }_{j=1}^m\delta (M\left[j\right],D(E(M,k_i),k_i)\left[j\right])$, where: M is the original message of length m bytes. $E(M,k_i)$ is the encoded message using XOR with key $k_i$. $D(E,k_i)$ is the decoded message using XOR with key $k_i$. $\delta (a,b)$ is the Kronecker delta, which is 1 if $a=b$ and 0 otherwise.   8: end for   9: Selection: 10: Calculate the probability of selection $P\left(k_i\right)={\displaystyle \frac{f\left(k_i\right)}{{\sum }_{j=1}^{n}f\left(k_j\right)}}$. 11: Select parents $k_{\mathrm{parent}1}$ and $k_{\mathrm{parent}2}$ based on probabilities $P\left(k_i\right)$. 12: Reproduction (Mating): 13: Perform crossover to generate offspring: c is a random crossover point within the length of the keys. $k_{\mathrm{child}}=(k_{\mathrm{parent}1}[:c]+k_{\mathrm{parent}2}[c:])$. 14: Mutation: 15: for each gene j in $k_{\mathrm{child}}$ do 16:     Mutate gene with probability $p_m$: $k_{\mathrm{mutated}\text{\_}\mathrm{child}}\left[j\right]=\left\{\begin{array}{cc}\mathrm{random}(0,255)\hfill & \mathrm{with}\mathrm{probability}{p}_m\hfill \\ k_{\mathrm{child}}\left[j\right]\hfill & \mathrm{with}\mathrm{probability}1-p_m\hfill \end{array}\right.$. 17: end for 18: Iteration: 19: Repeat the steps of evaluation, selection, reproduction, and mutation for G generations. 20: Convergence: 21: Select the best key $k_{\mathrm{best}}$ after G generations: $k_{\mathrm{best}}=arg{max}_{k_i\in \mathrm{population}}f\left(k_i\right)$.

For this application, genetic operators are parameterized to key generation for secure and efficient encryption. Single-point crossover is used, where a random crossover point within the key length (1–7 bits) combines segments from two parent keys to create an offspring, preserving diversity and high-fitness traits [36]. Mutation replaces a selected bit of the key with a new random value in the range $[0,255]$ at the rate of 10% to strike a balance between exploration and exploitation [37]. This ensures diversity and prevents premature convergence. These operators, applied iteratively over generations, enable the genetic algorithm to balance encryption strength, ciphertext randomness, and efficient message encoding while maintaining the decryption accuracy.

RSA Encryption: RSA encryption is a public key encryption algorithm that uses two keys: a public key for encryption and a private key for decryption. This is based on the mathematical difficulties of factoring the product of two large primes [38,39,40,41]. The integration of the RSA and the GA was designed such that the GA was executed as a pre-encoding step prior to the RSA encryption process, effectively introducing an additional layer of security. For plaintext, GA computes the necessary parameters while maintaining multiple criteria, as explained in the paragraph GA-Based Encryption for key generation and encrypts plaintext, as shown in Algorithm 1. This results in the formation of cryptographic information and structure. Attackers target cryptographic structures using attacks involving cryptanalysis processes [42] to recover keys and original information, necessitating additional security, which is ensured by RSA. The RSA encrypts the resulting cryptographic structure from the GA and stores keys (private and public) as objects that obscure its visibility against threats outside of the controller. Decryption was performed in the reverse order, with RSA executed before the GA, as shown in Figure 2.

3.2. Programming the Controller

The SDN controller (POX controller) was programmed to implement a data encryption policy. The controller centrally managed and orchestrated all network traffic within an SDN environment [43]. Encryption and decryption were integrated directly into the controller. This allowed the application of security policies to be deployed according to dynamic flow rules and policies, thus ensuring that transmission of data took place in encrypted form and were decrypted upon reception. This centralized functionality increased network security and data integrity while enabling the communication of encryption processes without requiring configuration on individual network devices [44]. The functionalities that were considered in order to enable the encryption policy through the SDN controller are explained below:

1. Topology Discovery and Network Configuration: The SDN controller needs to understand the network topology to make informed decisions. In this case, static routing tables can provide this topology information. The steps for topology discovery are as shown in Figure 3. The process in this study began by loading a static routing table into the SDN controller through the manual configuration of information, such as the destination network, next-hop address, and corresponding interfaces. The configurations were then parsed to identify the relationships and connections between different nodes. The parsed routing table was used by the controller to construct a network graph that provides the global view of the network. Once the graph was constructed and the configurations were obtained, they were pushed to the data plane devices through the southbound API (Openflow) that dynamically enforces policies, updates routing information, and manages network behavior [45].

2. Flow Rules for Packet: In SDN, the deployment of flow rules is an important part of configuring network behavior and managing traffic flows across OpenFlow switches. For this study, deployment started with converting the numerical identifiers of switch ports into string format for consistency, ease of reference, and standardization. The switch IDs were verified in routing entries, where the route table stores routing details specific to each switch. For each valid switch ID, the search proceeded to iterate through each entry in the destination IP and output port. For each destination IP, a match criterion was created using of.ofp_match(). The criterion triggered the associated flow rules, and nw_dst() was set to the destination IP. A flow modification message was then initiated with of.ofp_flow_mod() that encapsulated the specifics of flow rules intended for installation on the switch. Upon receiving the modification signals, the SDN controller dispatched the modification to the respective nodes through the southbound API.

3. Packet Handling:

a. At the Switch: Packet handling at the switch began immediately upon the packet’s arrival. At the ingress stage, the switch examined the packet’s header against the flow table entries and flow rules. Upon a match, the switch took the corresponding actions based on the rules. However, if there was no match, the packet was forwarded to the controller, which had a broader view of the network and its configurations. Based on the controller’s settings, it then decided how the packet should be handled and updated the flow rules for similar packets in the switch.

b. At the Controller Side: The packet handling process at the controller began with receiving and parsing the incoming packet from the network. The controller identified the datapath ID (dpid) and the port on which the packet arrived. Initially, the controller checked whether the packet was an Address Resolution Protocol (ARP) packet, an ICMP packet, or a non-IP packet. If so, an independent function handled the ARP packet, and no further action was taken for this packet within the current function. Similarly, an exception was made for all non-IP packets. When the packet was recognized as an IP packet, the controller extracted its IPv4 payload, which contained the source and destination IP addresses. If the IP packet contained a TCP payload, the controller checked the payload byte sequence for a match with the contents of the text file. The controller read the content from the text file, which was later used for encryption–decryption operations. If there was no message to be read, the controller ceased encryption processing on the packet.

For a new source IP address, the controller logs the message and generates a symmetric encryption key using a GA. The message is then encoded with this key and subsequently encrypted using the controller’s public key. The destination IP, encrypted message, and encryption keys are stored in a dictionary for later use. The TCP payload is then updated with the encrypted message, and the packet is patched together. Then, based on the destination IP and dpid, the controller decides the appropriate output port, generates a packet-out message, and sends the rules to the switch to forward the packet to the designated port, as shown in Figure 4.

When the packet reaches the switch at the destination end, the controller retrieves the stored key if the packet has a known source IP address with an existing destination IP in the dictionary. The message will be decrypted using the controller’s private key and then decoded using the stored symmetric key. The decrypted message will be recorded and used to update the TCP segment; subsequently, the packet will be reconstructed. The controller will know the correct output switch port and transmit the packet via the port, as shown in Figure 5. In cases where packets do not meet any of the described conditions, the controller decides upon the correct output port without inspecting the packet itself and simply forwards it. Such forwarding includes both packets that require special handling as well as those that do not, thereby allowing for consistent and complete operation throughout the network.

4. Key Management: The SDN controller centrally manages the keys, allowing for the standardization of security policies across the network. This centralization simplifies the management of distinct encryption and decryption entities, reducing complexity [8]. Through the centralization of key management at the controller level, an integrated and scalable security architecture will be established, ensuring that encryption policies are applied uniformly. This reduces the chances of misconfigurations or vulnerabilities associated with decentralized key management [10]. Although the centralized key management in the SDN controller simplifies administration, it also introduces a single point of failure. If the SDN controller is compromised, the entire encryption framework could be at risk. To mitigate this, redundant controllers with distributed key replication strategies will be deployed. Nevertheless, the study focused on controlled experimentation and systematic evaluation of encryption efficiency; the replication strategy is not within the current scope of the study.

3.3. Encryption Logic Deployment in SDN Environment and Performance Evaluation

The deployment of encryption logic is carried out in an SDN environment using Mininet. The network topology consisted of a central encryption controller that managed policy deployment across the network. Two switches and two hosts are connected to each switch as shown in Figure 6.

The SDN environment is assessed for performance using key metrics of transmission time, instantaneous throughput, average packet loss, and average jitter to diagnose network bottlenecks and to analyze the impact of encryption on overall network performance. In addition, a thorough cryptographic analysis has been conducted in relation to entropy and pattern analysis, involving advanced techniques such as differential and linear cryptanalysis, to evaluate the robustness of the encryption scheme.

4. Results and Analysis

To rigorously evaluate the encryption strength and network performance of the implemented GA-based hybrid encryption system within the SDN, we conducted a series of controlled experiments. The dataset consisted of 31 distinct text messages, each varying in length from 6 to 256 characters. The selection of messages was influenced by the need for controlled testing, imposed by limited resources in the simulation environment. While larger sample sizes could have allowed for more thorough and detailed assessments, the chosen 31 messages represented a diverse range of alphanumeric characters, symbols, and byte distributions. This allowed for meaningful cryptographic analysis. Despite variations in content, the messages followed a similar format, ensuring comparable byte distributions. This design allowed for testing of encryption’s ability to provide randomness and assess performance under diverse data conditions in SDN. The messages were transmitted across the network using the netcat command. This method helped model real-world network traffic and measure performance metrics. The variation in message length and complexity allowed us to check the robustness of the algorithm. This also tested the system’s ability to handle different loads. Logs of encryption and decryption processes were recorded in detail, along with CPU and memory utilization. Instances of packet loss or latency issues were also noted. This systematic analysis provided a clear and broad understanding of the proposed encryption scheme’s effectiveness and its impact on overall network performance.

4.1. Evaluation of Hybrid Encryption Algorithm Strength

The assessment of encryption strength involved an approach that encompassed several advanced analytical techniques. This included entropy analysis to measure the randomness and unpredictability of encrypted data; pattern analysis to scrutinize recurring sequences or structures within the encrypted content; differential cryptanalysis, which is a method for assessing encryption resistance against chosen plaintext attacks; and linear cryptanalysis to examine the susceptibility of encryption algorithms to linear approximations.

4.1.1. Entropy Analysis of Encryption

In cryptography, entropy is recognized as an important measure of the randomness or unpredictability of data. High entropy indicates a greater degree of randomness, ensuring that encrypted data are well secured [46]. Essentially, it refers to how uncertain or non-repetitive the contents of a message are. For example, the message “AAAAAA” has very low entropy due to its highly predictable, repetitive characters, resulting in similar byte distributions. In contrast, the message “G7$k@P!Q” has high entropy because of its complexity and unpredictability in both characters and byte distribution.

In this study, Shannon entropy was calculated for 31 encrypted messages. By computing the entropy for each encrypted message, we aimed to quantify the randomness and unpredictability of the encrypted data. This information is important for programmers, as it helps in the assessment of the algorithm’s performance in obscuring the message contents [47]. In the context of information theory, entropy can be derived mathematically, as shown in Equation (1):

$$H\left(X\right)=-\sum _{i=1}^{n}P\left(x_i\right){log}_{b}P\left(x_i\right)$$

(1)

where the following denotions are used:

-

$H\left(X\right)$ is the entropy of the random variable X.

-

$P\left(x_i\right)$ is the probability of the outcome $x_i$ occurring.

-

n is the total number of possible outcomes for X.

-

${log}_b$ denotes the logarithm to the base b, typically $b=2$ (binary entropy) or $b=e$ (natural entropy).

The entropy results were plotted on a graph to effectively visualize the variations observed in the dataset. Additionally, comprehensive statistical analyses were conducted and key descriptive metrics of the entropy data were calculated to provide a thorough understanding of the underlying patterns and behaviors, as shown in Figure 7.

The entropy values for 31 different messages are shown in Figure 7, and the corresponding statistics are presented in

Table 1. Summary of statistics of entropy values.

Description	Value
Count (No. of Messages)	31
Mean	0.989152
Standard Deviation	0.001774
Minimum Entropy	0.985688
25th Percentile	0.987888
Median	0.989167
75th Percentile	0.989937
Maximum Entropy	0.991961

. The mean entropy is 0.989152, and the median is 0.989167. The mean and median are close to each other. In a perfectly symmetrical distribution centered on the mean, this, along with a low standard deviation of 0.001774, suggests uniform randomness across all contributions. There is no significant deviation from the mean entropy. In cryptographic terms, this indicates the generation of highly unpredictable and random ciphertext patterns due to the applied encryption process. High entropy is desirable to maintain security [48] and shows the effectiveness of encryption algorithms in producing random data resistant to attacks. These consistent entropy values are crucial in ensuring that all encrypted data receive equal security strength [49].

In cryptographic terms, the algorithm’s ability to retain the entropy values at 1 allows each bit to have an equal probability of 0 or 1. This means an attack against the algorithm would not find it that easy to identify patterns or obtain useful information from the ciphertext.

4.1.2. Pattern Analysis of Encryption Algorithm

Pattern analysis of byte occurrences in assessing the strength of encryption provides an indication of how uniformly or evenly the distribution of 0s and 1s appears, describing levels of randomness [50]. The analysis of patterns in cryptography evaluates the frequency and occurrence of byte values or bit sequences within the encoded data. It checks for possible correlations and dependencies between bits to ensure that there are no predictable patterns [51]. An ideal encryption algorithm should yield ciphertext without perceivable differences. It should make both a single bit and a complete byte equally probable to be true or false, irrespective of the plaintext. For instance, an output “1010101010” reveals a definite weakness due to the alternating pattern of 1s and 0s. In contrast, “1101010001” appears random and patternless, indicating a secure encryption. The analysis confirms that a change in one bit does not affect any other bits, maintaining independence between them [51]. Moreover, it identifies vulnerabilities for the cryptographic attacks, catching malignant patterns that expose the weak links or faults in the algorithm. In addition, it evaluates the quality of the PRNGs carried out in the encryption process [51]. The heatmap in Figure 8 renders the pattern of occurrences of bytes at the varied position across a blend of 31 encrypted messages. It provides valuable insight into the behavior and characteristics of the cipher algorithm used. The axes and color gradations on the heatmap reveal patterns of how the algorithm addresses and deals with longer and shorter messages, giving clues about the underlying work of the cryptographic facilities.

The most important feature of the heatmap is the emergence of regions of high-frequency byte clusters, highlighted in red. The clusters are mostly prominent in the ranges of byte positions 48 to 64 and 96 to 128. Active, medium-length messages indicate activity in those areas, thereby affirming that those positions play a crucial role in the encryption process. Moreover, the presence of these clusters across all message lengths shows that there might be some periodic or structural elements in the encryption algorithm affecting these positions, probably in relation to the rounds in the encryption process or in particular stages that induce higher levels of data transformations [52].

On the other hand, only the longest messages use byte positions after 200, thereby showing low-frequency occurrences represented by the blue regions. This indicates that those positions are less frequently used, with shorter messages seldom reaching these byte ranges. The low-frequency distribution also suggests that even in longer messages, these positions might be used sparingly or in a more randomized manner, reflecting the padding or additional data added to complete encryption blocks [53]. Additionally, the visible patterns and high-frequency clusters, as shown in Figure 8, suggest some level of predictability in how the encryption algorithm processes data. This means certain keys having similar characteristics are preferred for several generations, which creates predictable patterns. These predictable patterns can also arise through fixed initialization vectors, repetitive key scheduling, or inherent deterministic features in the algorithm. This inadequacy in randomness manifests in some key segments, causing sparse intervals in the ciphertext distribution. The problem can be mitigated by introducing dynamic mutation rates based on key entropy, providing greater randomness in key evolution. Additionally, introducing a post-processing randomization layer prior to RSA or improving current entropy-based selection criteria during key generation may also help. The non-uniformity in distribution suggests areas where certain plaintext structures have a more significant influence on the encrypted output. This fact is evidently shown in Figure 9.

Figure 9 highlights that some byte values appear more frequently than others in the encrypted messages. The byte value with the highest density is around 0, and it progressively tapers off toward the edges of the plot. The various visibly clustered formations and high-density areas, as shown in Figure 8 and Figure 9, provide evidence of some level of predictability in the encryption algorithm’s data processing. These predictable patterns could be related to fixed initialization vectors, repetitive key scheduling, or deterministic features inherent to the algorithm [51].

In conclusion, based on the observed trends and clusters shown in Figure 8 and Figure 9, important implications for cryptographic security arise. Predictability conveyed by such patterns alludes to possible vulnerabilities that attackers could exploit. These vulnerabilities may include those techniques identified through frequency analysis or pattern recognition that might help to pinpoint weaknesses in the encryption process. The way in which the number of the positions of each byte was contemplated, along with their scatter, possibly suggests that enhanced randomness in the encryption algorithm, followed by less predictability, is needed.

4.1.3. Comparative Statistical Byte Distribution Analysis of Plain- and Ciphertext

The comparison focuses on the statistical distribution of byte values between plaintext and ciphertext. First, it shows that the encryption algorithm can hide the natural patterns of the plaintext. This ensures that the ciphertext appears completely unpredictable. As a result, it is resistant to statistical attacks that exploit predictable data structures. Second, by studying the byte distributions, cryptographers can determine whether the algorithm follows key cryptographic principles. One such principle is the avalanche effect, where small changes in the input cause significant changes in the output [54]. This analysis is important in proving the robustness of the algorithm against various types of cryptanalysis, thus establishing it as suitable for secure data protection applications. Figure 10 illustrates the distribution of plaintext and ciphertext byte values.

The histogram on the left of Figure 10 illustrates the frequency distribution of byte values in the plaintext. The byte values range from 0 to 250, with most being clustered around a specific region. There is a noticeable peak in the histogram corresponding to the value zero. This clearly indicates that this byte value is the most frequently occurring in the plaintext data. The high-frequency density of the zero byte points to data repetition or common zero-padding in the plaintext. More notably, a distribution cluster occurs between byte values 90 and 130, representing certain information patterns or encoding, which are more dominant in the plaintext. Such a non-uniform distribution in plaintext could become a vulnerability if the encryption algorithm fails to effectively diffuse these byte values throughout the ciphertext.

The histogram on the right of Figure 10 illustrates the frequency distribution of byte values in the encrypted messages. The byte values are evenly distributed across the entire range from 0 to 250. Unlike the plaintext, the distribution in the encrypted messages appears much more uniform. There is an absence of peaks; the frequencies of byte values are more evenly distributed across the whole range, indicating general uniformity. This equal distribution means that each byte value occurs nearly the same number of times, demonstrating a good level of diffusion. This kind of uniform distribution is a desirable property in cryptographic algorithms, indicating that the encryption process uniformly disperses the plaintext byte values across the entire range, thereby resisting cryptanalysis tests and attacks [55]. This uniformity results directly from the avalanche effect, where a slight change in the plaintext leads to a big, unpredictable change in the ciphertext, resulting in high diffusion [56]. This also makes it more difficult for an attacker to find a correlation between the two.

4.1.4. Differential Cryptanalysis

Differential cryptanalysis is a method for assessing the strength and security of cryptographic algorithms by analyzing the effect of plaintext differences on the corresponding ciphertext output differences [57]. The technique aims to identify patterns or correlations with byte differences that may exist within the entire encryption operation. These patterns could potentially reveal the internal workings of the cipher or even the secret key itself. The difference value in the plaintexts was compared to the difference value in the ciphertexts for the differential analysis, as shown in Figure 11.

The histogram illustration on the left of Figure 11 depicts the frequency distribution of differences within the plaintext value range of 0 to 80. Important observations include a noticeable peak at the 0 mark. This indicates that several plaintext pairs are either identical or have no difference at all. Byte differences near values 20 and 30 also appear frequently. There is a deep dip in the frequency distribution within the range of byte value differences from 40 to 60. Once again, there are few or almost no occurrences within this range. Another slight clustering of differences appears around the 60–80 range. The peak at 0 suggests that a significant number of plaintext pairs are identical, which may imply a high occurrence of repetitive data. Furthermore, clusters around 20 and 30 indicate regions where certain differences are more prevalent, possibly revealing weaknesses or patterns in the plaintext structure. This histogram implies that certain types of differences between plaintext pairs are more common than others.

The histogram on the right of Figure 11 displays the frequency distribution of the corresponding ciphertext byte values’ differences across a much wider range from 0 to 250. Among the various observations, one can note that there is a pronounced peak at 0, showing that while plaintexts may be different, some pairs of ciphertexts remained identical. The distribution appears to be much more dispersed than that of the plaintext difference, showing several peaks along the range. The observed byte difference value frequently appears at intervals of around 50, 100, 150, and 250. The wider and varied distribution indicates a higher level of diffusion in the ciphertext for plaintext differences in byte values. More evidence can further be substantiated through the analysis by creating a density plot of the byte values’ difference for plaintext and ciphertext, as demonstrated in Figure 12.

The density plot presented in Figure 12 indicates that it achieves a peak near a zero difference score. It suggests that many plaintext pairs differ slightly or not at all from one another. Similar pairs are either almost identical or differ slightly, as indicated by the peak close to zero. This may be the result of repeated data or particular test circumstances. There is a smaller peak at a difference value of 20, indicating that there is another prevalent difference amongst the pairs of plaintext. The secondary peak around the difference value of 20 indicates another prevailing difference, which may be due to a certain structural or patterned nature in plaintext. The distribution of plaintext differences is relatively narrow, mainly concentrated between −20 and 100, with very few instances toward longer differences. Such observations suggest that, in general, differences among plaintext pairs are minor and occur around particular values, letting certain patterns or characteristics emerge from the plaintext data. On the contrary, as observations shift to those that compare distinct ciphertext pairs, it becomes apparent that the density plots for the ciphertext differences were spread wider than their counterparts, rendering such plots clearly to be in the wider range of values. The broad range of ciphertext differences from −100 to 300 indicates that the encryption algorithm must be performing notably well. This effective diffusion of differences in the plaintext is entirely guaranteed to enhance security by reducing any patterns or correlations in the ciphertext.

4.1.5. Linear Cryptanalysis

Linear cryptanalysis is a critical technique for evaluating the strength of encryption algorithms. It systematically identifies potential linear relationships between plaintext, ciphertext, and the key. This method assumes that by refining the algorithm design, the security can be enhanced, making it more resistant to attacks aimed at decrypting the encrypted data [57]. By guaranteeing that the relationship between plaintext, ciphertext, and key is complicated and non-linear, this technique is a measure of how well an algorithm gives confusion and diffusion features [58]. In this experiment, the linear cryptanalysis was performed through the study of bit correlation and distribution of correlation coefficients, which were visualized through various plots so that a clear analytical conclusion could be drawn.

The correlation coefficients of bit pairs have been plotted on a scatter plot in Figure 13, providing linear relationships that exist between the bits of the encrypted messages. Most of the points in the scatter plot presented in Figure 13 coalesce around a coefficient of correlation close to zero, showing that most bit pairs lacked significant linear relationships. This dense clustering in the middle is, in fact, a positive sign of the encryption algorithm producing bit sequences close to random, giving no predictable relationship to work with. An encryption system would ideally pass with a minimum degree of correlation, as any form of predictability could present an exposure of the key or plaintext. However, scatter plots extend the correlation coefficients, ranging from approximately −0.8 to 0.8. The scatter plot shows that although most bit pairings appear uncorrelated, some pairs exhibit positive or negative correlation. In the plots, a few of the clusters plotted are positive, with correlation coefficients greater than 0.4.

These clusters indicate certain linear relationships within particular bit pairs. Changes in one bit are predictably mirrored by changes in the other. Positive correlations here might indicate a structural weakness in the encryption algorithm, which may arise from low diffusion or linear transformations that do not adequately obscure plaintext relationships. Similar clusters of datasets have also shown very strong correlations below −0.4. In these cases, the alteration in one bit would be expected to bring about an alteration in the opposite direction in the other bit. Negative correlations are just as significant as positive correlations, given that they will put together a systematic relationship. It could be true for linear cryptanalysis purposes. The presence of both positive and negative correlations suggests that the encryption algorithm may have deterministic behaviors that linear cryptanalysis techniques could leverage [59]. The presence of linear approximation in the algorithm could also be observed through the distribution of correlation coefficients between the bit pairs, as shown in Figure 14.

The histogram in Figure 14 is centered around a correlation coefficient of zero, indicating minimal correlation among most bit pairs. This reflects the encryption algorithm’s randomness, a key trait for security. Correlation coefficients range from −0.8 to 0.8, with most values between −0.2 and 0.2. A value close to zero implies that there are no significant linear correlations that would not support linear cryptanalysis. The density falls off smoothly away from zero. However, a couple of outliers are worth considering for possible enhancement. This type of behavior is visibly illustrated in Figure 15. This shows that the near concentration of correlation coefficients around the zero mark reflects a good unpredictability of outputs duly supported by high central density; such high central density reflects the absence of clear patterns in the encrypted data, a condition for information confidentiality and integrity.

To conclude, analyses of the hybrid encrypting algorithm through different methods show that the encryption algorithm gives out random-like outputs with most of the expected factors near a normal distribution of central density, symmetric at about zero. But with outliers, it shows areas requiring further improvement in security. Amid the usual audits, it may need other evolutions in its diffusion and non-linear transformations. This way, the masks stay robust against cryptographic attacks targeting their data.

4.2. Network Parameter Evaluation

The network parameters, such as delay, jitter, instantaneous throughput, bytes transmitted in the payload, and total bytes transmitted, were measured. This allowed for the analysis of the algorithm’s efficiency and the network’s responsiveness to the encryption policy.

4.2.1. Study of Transmission Time in SDN Environment

To measure the delay in this test accurately, the transmission time of 31 messages of varying character length was precisely recorded as illustrated in Figure 16 and

Table 2. Descriptive statistics of the transmission time.

Description	Value (s)
Number of Messages	31
Mean	1.065513
Standard Deviation	0.057506
Minimum (Min)	1.002
25th Percentile (25%)	1.0135
Median (50%)	1.0416
75th Percentile (75%)	1.101
Maximum (Max)	1.2304

. Messages were transmitted from host h1 to host h4, which were intentionally bridged by several switches in the SDN. This experiment provided valuable insights into the effects of message length and switch interaction on transmission time. In addition, commands were employed to calculate the time of transmission for non-IP packets from h1 to h4. This was a necessary step since it bypassed encryption processing so that the underlying transmission speed in the SDN could be comprehended in a more open way. From the combination of the outcomes of the messages sent and the non-IP packets, an overall view of the efficiency of the SDN in transferring different types and sizes of data was obtained.

The transmission time of the IP packets carrying message payloads to be encrypted is, on average, approximately 1.0655 s. A standard deviation of 0.0575 s indicates moderate variability around the mean. The range is from 1.002 s to 1.2304 s, giving a range of 0.2284 s. The range indicates that, even though the transmission times are different, they are different within a reasonably small range. The median transmission time, at 1.0416 s, is slightly lower than the mean, indicating a slight right skew in the data distribution.

For non-IP packets, the transmission time for 31 packets that do not require encryption was taken into account, as shown in Figure 17 and

Table 3. Descriptive statistics of transmission time for non-IP Packets.

Description	Value
Number of Packets	31
Mean	24.979 ms
Standard Deviation	0.898 ms
Minimum (Min)	23.571 ms
25th Percentile (25%)	24.452 ms
Median (50%)	24.869 ms
75th Percentile (75%)	25.517 ms
Maximum (Max)	26.783 ms

. The average transmission time is significantly lower at 0.024979 s. The standard deviation is just 0.000898 s, reflecting very low variability. Minimum and maximum transmission times are 0.023571 s and 0.026783 s, respectively, with a narrow range of 0.003212 s. Median transmission time is near the mean at 0.024869 s, reflecting symmetry in data distribution. The percentiles also reflect this uniformity. The low standard deviation and narrow IQR indicate that most non-IP packets have little or no deviation from the average transmission time, demonstrating their consistency in the network.

Comparing the two types of packets, it is clear that non-IP packets transmit at a lower and constant time than IP packets. The transmission time for non-IP packets is significantly lower at 0.024979 s than for IP packets at 1.0655 s. This difference indicates the effect of encryption processing on transmission times. The computational cost of encryption is shown by the longer and fluctuating transmission times for IP packets, highlighting a basic trade-off between security and performance that may be examined through

Table 4. Descriptive statistics for encryption and decryption latency times.

Statistic	Encryption Latency Time (s)	Decryption Latency Time (s)
Number of Data	31	31
Mean	0.5342	0.3194
Standard Deviation	0.0393	0.0231
Minimum (min)	0.4895	0.2937
Maximum (max)	0.6399	0.3864

.

As observed from Table 4, the average encryption latency is 0.5342 s while the average decryption latency is 0.3194 s, as expected due to the increased complexity of encryption. This has more computation steps and requires time for convergence to a solution. For comparison with the transmission delays as shown in Table 2 and Table 3, it can be observed that the bulk of the transmission delay is contributed by encryption and decryption overhead. Therefore, to reduce the transmission delay, the proposed method needs to be fine-tuned with approaches for minimization in delay to meet the demand for specific delay-critical applications.

4.2.2. Study of the Jitter in SDN

The jitter is the term used to describe the irregularity in the timing of data packet arrivals and is a key parameter for network performance analysis, especially for maintaining QoS in time-critical applications, such as VoIP and video conference, where predictions in packet delivery are needed to ensure service quality. High jitter levels indicate irregular delays in packet transmission, often caused by network congestion, varying routes, or differing processing times [60]. The calculated jitter for the multiple message packets is shown in Figure 18 and the statistics are presented in

Table 5. Statistical overview of jitter in the network.

Statistic	Value (ms)
Data Count	31
Mean Jitter	50.026
Standard Deviation	26.851
Range	17.7–164.9
25th Percentile (Q1)	32.800
Median (Q2)	47.313
75th Percentile (Q3)	57.363

.

The average jitter of 0.050 s (50 ms) indicates that packets arrive later than their expected times. This delay is caused by network conditions and processing times due to the encryption process. Although most packets arrive close to their intended times, one packet is consistently delayed. This slight delay affects the network’s overall performance. For real-time critical services such as video conferencing or VoIP, the acceptable level of jitter is below 30 ms [61]. However, for network control, routing updates, and non-real-time-sensitive data transfers such as message transfer [62], the jitter is manageable around 200 ms [63]. With an average jitter of 50 ms and a maximum jitter of 164.9 ms, the current encryption method employed as a policy may not be suitable for applications with strict latency requirements, such as real-time multimedia or IoT control. However, the policy may still be relevant for tasks such as secure data or message transfer, where jitter is not a primary concern. To mitigate the issues related to latency-critical tasks, approaches such as optimization of packet scheduling, minimization of computational overhead [64], or traffic classification-based adaptive encryption techniques shall be employed.

4.2.3. Study of Bytes Transmitted in Payload and Transmission Time

We investigated the impact of bytes transmitted after encryption on message transmission time. For each encrypted message, payload size after encryption and the total transmission time were recorded, as shown in Figure 19. The statistics are shown in

Table 6. Statistical summary of bytes transmitted and transmission time.

Metric	Bytes Transmitted	Transmission Time (s)
Number of Data	31	31
Mean	179.61 bytes	1.0655
Standard Deviation	133.46 bytes	0.0575
Min	48 bytes	1.0020
25th Percentile	96 bytes	1.0135
Median	144 bytes	1.0416
75th Percentile	204 bytes	1.1010
Max	816 bytes	1.2304

. This dataset was used to analyze how encryption affects message size and network performance by comparing transmission time and encrypted message size with correlating size and transmission time.

From Table 6 and Figure 19, it is evident that the distributions of bytes transmitted and transmission time are right-skewed. In the case of bytes transmitted, the mean (179.61 bytes) is more than the median (144 bytes). This indicates that a few large sizes in the payloads are pulling the mean upwards, and there are predominantly lower payload sizes. The trend is the same for transmission time. Regarding the spread and variability, transmitted bytes have a greater standard deviation (133.46 bytes), indicating that there is a wide dispersion from the mean and significant variability in payload sizes. In most cases, transmission times are more stable and constant compared to the sizes of the payload and correlation coefficient between transmitted bytes and transmission time, as shown in Figure 20.

The scatter plot in Figure 20 visually represents the correlation between bytes transmitted and transmission time. Most of the data points are clustered together, with a regular trend of larger payload sizes taking longer to transmit. The trend line representing the correlation has a definite upward slope. The graphical representation of the positive correlation emphasizes the statistical computation of the direct relationship between the variables. The strong positive correlation coefficient value of nearly 1 indicates that with the increase in payload size, transmission time also increases proportionally.

4.2.4. Study on Instantaneous Throughput

The throughput for each instance when messages were sent was calculated, and the statistical summary of the instantaneous throughput in bytes per second was noted. This reveals the statistics as shown in

Table 7. Statistical overview of data throughput.

Statistic	Value (bytes/s)
Number of Observations	31
Mean	226.83
Standard Deviation	103.29
Minimum	114.77
25th Percentile (Q1)	161.68
Median (Q2)	203.98
75th Percentile (Q3)	247.82
Maximum	717.65

and Figure 21.

The average instantaneous throughput of 226.83 bytes per second, with peaks of up to 717.65 bytes per second, represents the encryption system’s performance under current conditions. The mean instantaneous throughput (226.83 bytes per second) is higher than the median (203.98 bytes per second), indicating a right-skewed distribution which suggests that while most of the network transmissions achieve throughput values around the lower end of the range, there are a few instances where the network achieves significantly higher throughput. Although this throughput with the current method is sufficient for control plane operations, like flow rule management and policy enforcement, it is significantly below the data plane requirements in modern SDN environments, which typically require throughput ranging from 100 Mbps to 10 Gbps [65]. It highlights the necessity for exploring approaches to be deployed alongside the proposed method to enhance throughput. To properly understand how the throughput behaves as the network transmission scales up, it is important to study its behavior against the bytes of data transmitted in the instance [66]. For this, the correlation between bytes transmitted and instantaneous throughput has been studied, as shown in Figure 22.

Figure 22 depicts a scatter plot for bytes received against instantaneous throughput. There is a correlation coefficient value of 0.86, indicating a positive linear and direct relationship with a good fit between the variables, which means that more data are being processed with increasing throughput in proportion ratio. The graph clearly shows that data are being transferred efficiently with an increasing transmission volume. In networking terms, it implies that the currently designed network infrastructure is efficient with optimal bandwidth and resources to cope with higher data rates and can efficiently scale up with high throughput as the amount of data transmission increases [67] without noticeable diminution in performance.

In conclusion, from the observed range of throughput and trend by correlation, it can be deduced that although the system has encouraging signs with regard to efficient scaling at its current limit, enhancing throughput is required for better operation. To ensure performance as the network scales towards higher data rates and to enhance and avoid possible bottlenecks caused by throughput, optimization through hardware acceleration and parallel and distributed processing approaches need to be explored.

4.3. Evaluation of System Performance

To evaluate system performance, monitoring tools were employed to gather detailed insights into the system’s behavior during the execution of the emulation process. The primary metrics of interest are CPU utilization and network utilization. These parameters were meticulously monitored to understand resource usage and assess system performance under the given conditions.

Figure 23 represents the CPU utilization of 12 cores for a period of one minute. Every colored line indicates the activity of an individual core with the corresponding percentage used indicated in the legend. The data indicate that overall CPU utilization ranges from 0 (0%) to slightly over 20%, which means low to moderate workload levels across the system. Upon examining each core usage individually, utilization observations suggest CPU 1 and CPU 2 with relatively higher peaks of 5.0% and 10.2%, respectively. CPUs such as CPU 3, CPU 5, CPU 6, CPU 8, CPU 12, and CPU 4 are characterized by minimal usage, reaching 1.0% only. CPU 7 and CPU 11 have no significant usage, reading consistently at 0.0% usage. In contrast, CPU 9 and CPU 10 both have much higher activity levels, with their peak usage rates of 18.2% and 14.9%, respectively, indicating that these processors are carrying a greater share of the load.

The unevenness in the distribution of CPU load as seen on the graph indicates that the GA-based hybrid encryption mechanism puts different amounts of workloads on different CPU cores and other programs and processes running in the background, as shown in Figure 23, in stark contrast to the unevenly higher utilization of CPU9 and CPU10 compared with most of the other cores that have near-zero activity reported. These observations suggest that, while the system is not overloaded, there exist certain phases where performance can be improved. For improving performances, optimizing the scheduling of loads and fixing any potential bottlenecks in the heavily loaded cores can enhance the overall system performance and efficiency [68].

The network utilization graph, as shown in Figure 24, provides a detailed view of the data transmission rates over a 30 s interval during the emulation process of the GA-based hybrid encryption while keeping all other tasks that utilize the network off. Figure 24 considers both incoming (blue line) and outgoing (red line) network traffic; this is displayed along the y-axis in KiB/s with respect to time taken along the x-axis. Analysis of the plot indicates obvious trends and effects to the network performance of the system during emulation runtime. Peaks in network usage at certain intervals show phases of intensive data transfer.

The initial small spike is observed around the 50-s mark with subsequent spikes at 30 s and 20 s. The biggest spike of network usage is observed towards the end of the time being monitored, around the 10-s mark, where the rate of incoming data reaches a peak at approximately 30 KiB/s. The rate of outgoing data reaches a peak at approximately 6 KiB/s. These spikes are designed to align specific phases of the GA-based hybrid encryption process involving high data interchange intensity, such as population update, fitness calculations, and outcome dissemination. These peaks and spiking activity at different times also show that the network is not always active, wasting available bandwidth [69]. This can cause network lag or congestion, particularly if the network infrastructure is not optimized to handle such an abrupt spike in data transmission.

In the SDN environment, comprehensive performance evaluation requires power and energy consumption analysis. While the actual energy consumption with emulation process cannot be observed and requires an entirely different approach [70], the power/energy consumption during emulation can be monitored across devices based on their respective process IDs and resource allocation [71]. Observations and calculations were systematically recorded for each message transmission, ensuring precise tracking of resource usage. The summarized results of these observations are presented in

Table 8. Summary statistics of power and energy consumption during emulation.

Statistic	Total Power (W)	Energy (Joules)	Energy (Wh)
Number of Data	31	31	31
Average	10.044148	10.702168	0.002973
Standard Deviation	0.000608	0.577575	0.000160
Min	10.043039	10.065002	0.002796
Max	10.044945	12.359285	0.003433

.

The study of power and energy in the SDN environment highlights stable power uses, with an average of $10.004$ W, and minimal variation, on a processor with a per-core-TDP of $5.42$ W. Figure 23, Table 8 and per-core-TDP indicate an imbalanced load distribution but show a consistent pattern of resource utilization [72]. The imbalance in load distribution is more visible with energy consumption, where variability better reflects the need for some specific optimization. Although the finding implies stable power levels and predictable performance for the current SDN environment, conclusive results can only be obtained after power and energy analysis is conducted per node with different experiments performed on a large network, which is not within the scope of this study.

4.4. Comparative Entropy and Pattern Analysis with Standalone Legacy Encryption

As observed in

Table 9. Entropy comparison of different encryption methods.

Statistic	Hybrid Encryption	AES	DES	RSA
Count	31	31	31	22
Mean Entropy	0.98915	0.98752	0.98842	0.89580
Standard Deviation	0.00177	0.00272	0.00313	0.00254
Minimum Entropy	0.98568	0.98231	0.98141	0.89068
25% Entropy	0.98788	0.98634	0.98685	0.89450
Median (50%) Entropy	0.98916	0.98723	0.98834	0.89576
75% Entropy	0.98993	0.98869	0.99110	0.89749
Maximum Entropy	0.99196	0.99382	0.99416	0.90124

, hybrid encryption has the highest mean entropy of 0.989152 and the lowest standard deviation of 0.001774, which constitutes a significantly high level of unpredictability for encryption quality. The AES approach also has a very high mean entropy of 0.987527, which is slightly less than hybrid encryption, with a relatively higher standard deviation of 0.002728, indicating higher variability in entropy levels. Similarly, DES follows very closely with a mean entropy of 0.988424, but also boasts the largest standard deviation of 0.003134 of the three, indicating more variation and a less uniform strength in encryption.

RSA, however, shows a considerably lower mean entropy of 0.895807 because it is plagued by message length problems, with 9 of the 31 messages being too long to encrypt. This highlights the practical limitations of its use on longer messages. In comparison, hybrid encryption is the top performer with the highest mean entropy and lowest variability, followed by AES with nearly equal mean entropy and fairly low variability. DES has high mean entropy but more variability, indicating a slightly lower consistency in encryption strength. RSA, as evidenced by its much lower mean entropy and utilitarian restriction on longer messages, is not as practical in cases of longer message/content form.

The byte occurrence pattern of the hybrid algorithm was compared with the byte occurrence pattern of each individual traditional algorithm, such as AES, DES, and RSA, as shown in Figure 25. The analysis of byte occurrence patterns for AES, DES, RSA, and hybrid encryption reveals significant differences in how each algorithm handles messages of varying lengths. RSA struggles with long messages as indicated by the missing nine messages due to encryption limitations. AES and DES perform well with short messages and with regular patterns, and are truncated after a specific byte range, indicating inefficiencies with larger data. Both the algorithms are able to handle short messages effectively but may require additional steps for encrypting longer ones.

In conclusion, hybrid encryption proves to be the most adaptable. It handles both short and long messages effectively, showing a variable byte distribution across different message lengths. By combining the strengths of GA-based encryption and asymmetric (RSA) encryption, it overcomes RSA’s limitations while maintaining efficiency for large datasets. This makes hybrid encryption the most versatile and capable solution among the four.

4.5. Impact of Genetic Operators on Cryptographic Performance

In genetic algorithm-based applications, genetic operators such as mutation rate and crossover significantly impact the convergence of a solution. The values of mutation rate and crossover are crucial for balancing exploration and exploitation [37]. High mutation rates may promote exploration but disperse the solution across generations. On the other hand, low mutation rates increase exploitation but risk premature convergence [73]. To evaluate the impact of these genetic operators, multiple single crossover points (1, 3, 5, 7 bits) and mutation rates were tested to observe the changes in decryption accuracy and encryption/decryption time. The results were recorded, as shown in

Table 10. Mutation rate and crossover bits’ impact on the cryptography in terms of decryption accuracy and encryption and decryption time.

Mutation Rate	Crossover Bits	Decryption Accuracy (%)	Total Time (s)
0.02	1-bit	90.00	0.454
0.02	3-bit	89.67	0.454
0.02	5-bit	87.42	0.440
0.02	7-bit	87.42	0.435
0.05	1-bit	95.00	0.654
0.05	3-bit	96.72	0.645
0.05	5-bit	98.82	0.670
0.05	7-bit	99.20	0.703
0.10	1-bit	99.99	0.740
0.10	3-bit	99.99	0.780
0.10	5-bit	100.00	0.853
0.10	7-bit	96.72	0.857
0.15	1-bit	0.00	1.220
0.15	3-bit	0.00	1.340
0.15	5-bit	0.00	1.520
0.15	7-bit	0.00	1.653
0.20	1-bit	0.00	1.682
0.20	3-bit	0.00	1.702
0.20	5-bit	0.00	1.732
0.20	7-bit	0.00	1.753

. The recorded results reveal that with low mutation rates, decryption performance slightly decreases at longer crossover bit lengths. The accuracy drops from 90.00% at 1-bit crossover to 87.42% at 7-bit crossover. Additionally, computation times for encryption and decryption generally increases with longer crossover lengths. The decrease in decryption accuracy can be attributed to premature convergence to the solution.

As the mutation rate increases to 0.05, the decryption accuracy also improves, reaching up to 99.20% with a 7-bit crossover, along with an increase in encryption and decryption time (represented as Total Time (s)). This is because a higher mutation rate allows for a more thorough exploration of the solution space, which leads to increased time but brings the algorithm closer to the optimal solution. Similarly, at a mutation rate of 0.10, the decryption accuracy peaks at above 99% for almost every crossover bit, making 0.10 the best mutation rate for encrypting alphanumeric data. However, for mutation rates higher than 0.10, decryption accuracy performs poorly, with a 0.0% accuracy score and a significant increase in time. This is due to the higher mutation rate promoting increased exploration, leading to greater randomness in the ciphertext but preventing convergence toward a balanced solution.

5. Conclusions and Future Works

This study proposes a novel hybrid encryption scheme that combines GA-based encryption with RSA and aims to enhance the security in an SDN environment. Using cryptanalysis and empirical tests, the encryption scheme was evaluated for its efficacy. An appreciable contribution of this work lies in the demonstration of a definitive edge over state-of-the-art techniques in SDN security with the provision of a much higher level of security for encryption than classical techniques. The findings of this research utilizing the GA-based hybrid encryption for maintaining confidentiality in SDN revealed significant insights into the practicality and scalability of such systems. The insights established through the rigorous study of encryption strength and network performance matrices revealed a possible trade-off between the network performance and encryption strength. While the strength of the encryption of the proposed method was significantly better and higher than standalone traditional algorithms, making it resistant to cryptanalytic attacks, the impact on network latency and jitter under unbalanced load distribution for packets undergoing encryption posed challenges for real-time applications, such as VOIP, real-time video streaming, and IOT control operations. However, the latency and jitter still remained viable for non-time-critical data transfer applications, where security is prioritized over speed.

The computational overhead in the mutation and crossover processes of the GA primarily caused the latency. This computational overhead is a major concern particularly for resource-constrained environments involving IOT devices. However, there are different ways to address this issue. To enhance performance and address scalability issues along with the security needs in the large network involving IOT devices, lightweight GA optimized for low-power devices can be explored [74,75]. Another way to enhance performance and scalability could involve the distribution of computationally intensive encryption tasks across multiple available resources and SDN controllers [76]. Furthermore, latency and jitter could be reduced to meet the needs of real-time applications through traffic-based load balancing mechanisms in a multi-controller SDN architecture [77] with adaptive encryption strategies, where traffic data classified as sensitive packets are encrypted and non-sensitive traffic data are allowed to pass without encryption. Furthermore, different from most of the existing research that utilizes GA-based encryption and measures [30,31,32,33,34,35], this study is unique in integrating GA with RSA within the SDN environment. It considers a broader set of criteria beyond just decryption accuracy, making it a more comprehensive approach for secure transmission. Although there are other data protection methods, such as obfuscated code quality measurement (OCQM) [78], the major difference between OCQM and encryption lies in the implementation. Encryption ensures safe communication through the mathematical transformation of plaintext into ciphertext. Obfuscation methods are intended to conceal data structures so that reverse engineering is difficult. Obfuscation techniques, such as instruction reordering, dead code injection, and variable renaming, have been traditionally implemented in software protection and prevention against malware.

The following limitations in this research provide areas for improvement for readers and future research:

• Network Topology: The network topology in the study is relatively small and, therefore, was easier to manage with the static method to derive a global network view in the controller. For the larger network dynamic, the OpenFlow discovery method would be more suitable, which may lead to added complexity.
• Limited Cryptanalysis: Although the cryptanalysis performed to evaluate the strength of the encryption algorithm was sufficient to draw a preliminary conclusion on strength, the complete conclusive statement can only be provided after considering the encryption algorithms against the APTs and quantum attacks.
• Optimization of Load and Network Parameters: The optimization of load distribution and network parameters, such as bandwidth, was not performed, and the observation of the performance of the SDN environment was performed on default settings.
• Application to Real-World SDN: This study is focused on analyzing the performance, strength, and efficiency of an encryption mechanism, using a controlled environment of Mininet for the emulation process to mimic the real-world SDN. The real-world application would introduce constraints including hardware limitations, heterogeneous device compatibility, network congestion, and unpredictable traffic patterns.
• More Detailed Comparative Study: Although the current study includes the encryption strength’s test against the standalone algorithm, there is still a need for a proper benchmark algorithm analysis of network performance under the same network conditions. Furthermore, the extension needs to include comparisons with other data protection mechanisms.
• Exploration with Approaches for Improvements in Network Parameters: The current study scope is focused on the deployment of encryption as a policy. Therefore, the issues with network parameters, such as delay, throughput, and jitter underperforming, which require different approaches to be combined with existing methods, have not been explored. This shall be a major concern of our future work, together with the security issues.

To address its current limitations, future work will focus on the following tasks:

• Implementation of the GA-based hybrid encryption in a large network in a real SDN environment involving a multi-controller setup.
• Exploration of a lightweight GA for encryption policy implementation in networks involving IOT devices.
• Adaptive encryption strategies for sensitive and non-sensitive packets utilizing a traffic-based load balancing mechanism in a multi-controller setup.
• Integrate post-quantum cryptographic techniques to improve resilience against quantum attacks.
• Exploration of utilizing GPU-accelerated parallel processing to reduce encryption and decryption latency.
• Extend security evaluation to include resistance against APTs and evolving attack models.
• Development of an intelligent SDN-based load balancing mechanism to optimize encryption workload distribution by distributing computationally intensive tasks across available resources.
• Conduct explorations to fine-tune SDN parameters such as bandwidth allocation and congestion control to enhance network efficiency.
• Performance evaluation against the traditional standalone and hybrid algorithms over the SDN network.
• Conduct a detail evaluation of the strength and performance of the proposed mechanism against other data protection methods.
• Integration of different strategies for enhancing network performance in high-load and high-data-transmitting network settings.

In conclusion, this research not only advanced the state-of-the-art in SDN security but also highlighted the trade-offs between security and performance that require optimization to address latency-related issues. The study opened up new avenues for further exploration with future research, aiming to enhance the scheme’s capabilities to real-time applications or latency-critical applications.