You are currently viewing a new version of our website. To view the old version click .
Electronics
  • Article
  • Open Access

15 June 2024

Open Source Software-Defined Networking Controllers—Operational and Security Issues

,
,
and
Institute of Telecommunications, Faculty of Computer Science, Electronics and Telecommunications, AGH University of Krakow, 30-059 Krakow, Poland
*
Author to whom correspondence should be addressed.
This article belongs to the Special Issue New Insight into Network Virtualization and Management

Abstract

The Software-Defined Networking concept plays an important role in network management. The central controller, which is the main element of SDN, allows the provision of traffic engineering and security solutions in single- and multiple-layer networks based on optical transmission. In this work, we compare selected open-source implementations of SDN controllers. Throughput and latency measurements were analyzed using the CBench program. The simulation of a link failure and a controller failure were conducted using the API provided by the Mininet network simulator. To detect security vulnerabilities, a dedicated program, called sdnpwn, was used. This work provides an overview of the selected controllers, indicating their strengths and weaknesses. Moreover, some implemention suggestions and recommendations are presented.

1. Introduction

Software-Defined Networking (SDN) is a popular approach for developing and managing telecommunication networks. It allows network administrators to manage traffic flows and policies of the entire network from a single centralized device. The concept is implemented by separating the control plane from the data plane. The control plane determines where to send traffic, while the data plane actually forwards the traffic in the hardware. This could be a cable switch or an optical device [1].
Using a centralized entity to control the entire network eliminates the need to manually configure devices and allows changes to be made more efficiently. It also provides independence from equipment manufacturers, allowing the usage of devices delivered by different vendors in the network.
The SDN controller is responsible for all the intelligence and decision making within the network. There is a wide range of solutions available, both proprietary and non-proprietary. As the main purpose of SDN is it being available to everyone, this article focuses on those provisioned as open source. Their source code is publicly available: anyone can use, modify and distribute it. The three open-source controllers that were selected for comparison are: OpenDaylight [2], Floodlight [3] and Open Network Operating System [4].
Considering wide adoption of SDN, there is a need for a comprehensive analysis of the practical capabilities and limitations of SDN controllers. In-depth evaluations of controller implementations are necessary to guide future research and development. This paper provides a detailed comparative analysis of performance, resilience to failures and the security features of selected controllers. By highlighting the advantages and limitations of each connection, the controller provides valuable guidance in selecting the most appropriate solution for specific applications. The findings may also pave the way for future research on SDN technologies.
The remaining part of this article is organized as follows. Section 2 discusses the current state-of-the-art related to SDN controllers and their possibilities. Section 3 describes our testing environment. Afterward, in Section 4, we show the results related to our experiments. We discuss throughput, latency, failures and security issues in various topologies. Finally, Section 5 concludes the paper by discussing the most important findings.

3. Testing Environment

The research was conducted within a controlled virtual environment utilizing VirtualBox as the hypervisor to manage four distinct groups of virtual machines: Controller VMs, Network Simulator VM, Performance Testing VM and Security Testing VM. Controller VMs were based on the Ubuntu 22.04 LTS (Jammy Jellyfish) operating system. Each VM hosted a different network controller: Floodlight, OpenDaylight and ONOS. The Network Simulator VM, also utilizing Ubuntu 22.04, was dedicated to running Mininet, a network simulator used to create a virtual network environment managed by remote controllers. The Performance Testing VM had a purpose of running CBench, a performance testing tool. Due to compatibility requirements with certain libraries unavailable in newer operating systems, this VM operated on an older version of Ubuntu, specifically, Ubuntu 14.04. For the Security Testing VM, Kali Linux was selected as the operating system. Due to its comprehensive suite of security testing tools, it was found suitable for conducting security assessments. All VMs were configured to use the same internal network within VirtualBox, facilitating inter-VM communication while isolating them from external networks. This configuration ensured a secure and controlled environment for the research, allowing the VMs to interact exclusively with each other without any external access.
To measure the controllers’ performance, Controller Benchmarker (CBench) [16] was used. It is a dedicated tool for rating the performance of controllers which use the OpenFlow protocol [17]. CBench works by emulating a large number of switches that connect to a controller, generating PacketIn messages for each new flow. Two metrics used by CBench are latency and throughput. Throughput is defined as the rate at which the controller processes flows. It is the number of received PacketOut messages, that can be compared with the number of PacketIn messages sent by switches in a unit of time [18]. Controller latency is the time the controller needs to process a single packet. CBench measures latency by sending a single PacketIn packet to the controller and waiting for a response before sending another packet. This operation is repeated several times over a set time interval, and the average time is calculated [19].

4. Results

In this section, we present the results of carefully selected simulation experiments.

4.1. Throughput in Relation to the Number of Switches

In the first experiment, CBench was run in the Throughput Mode. The number of switches in the topology was changed in each run to values: 1, 2, 4, 8, 16. To prevent anomalies that may occur in individual executions and provide statistically valid results, the experiment was repeated 20 times. A script was used to run the program multiple times and output the results to files, which were then used to calculate average values. Twenty executions were found to be optimal, as they allowed for the generation of results with a 95% confidence interval, which was considered sufficient to provide reliable results. The results are presented in Table 2 and Table 3. The error bars are calculated using the t-Student distribution (confidence interval equal to 0.95). The results are visualized in Figure 1.
Table 2. Number of flows processed by Floodlight controller in 1 ms.
Table 3. Number of flows processed by ONOS controller in 1 ms.
Figure 1. Throughput vs. number of switches.
The throughput provided by ONOS is twice as large as for Floodlight. It was observed that for both controllers, throughput decreases as the number of switches doubles. The relation between those values is proportional, resembling a linear function. It was expected and confirmed by the simulation results that the operational possibility observed for the controller decreases with the increasing number of devices it has to communicate with.
During the experiment’s execution, the OpenDaylight failed to provide any results. It returned zeros instead of meaningful values. However, the investigation indicated that this problem has already been observed by the authors of the article [20] and is known as the “all-zeros problem”. Since the CBench program is unable to measure the throughput of the OpenDaylight controller, it was not included in the calculations.

4.2. Throughput vs. Number of Unique MAC Addresses on a Switch

The second experiment involved measuring throughput in relation to the number of unique MAC addresses in the network. CBench was again run in the Throughput Mode, repeated 20 times, and the number of MAC addresses was altered within a range from 100 to 1,000,000. Table 4 and Table 5 and Figure 2 show the results.
Table 4. Number of flows processed by Floodlight controller in 1 ms.
Table 5. Number of flows processed by ONOS controller in 1 ms.
Figure 2. Throughput vs. number of unique MAC addresses on a switch.
Figure 2 shows the changes in throughput in relation to the number of unique MAC addresses on the switch. OpenDaylight again failed to return valid results. ONOS was observed to provide higher throughput than Floodlight. Throughput did not change significantly when the number of unique MAC addresses in the switch was between 100 and 100,000. A significant decline in throughput occurred when the number of unique MAC addresses on the switch exceeded 100,000. However, this may not be the result of the low performance of the controller but caused by the limitations of the simulation environment. The measurements associated with the ONOS controller have a larger standard deviation and, consequently, the confidence intervals are much larger than for Floodlight.

4.3. Latency vs. Number of Switches

In this section, the CBench program was run in the Latency Mode. The first experiment involved measurements of delay in relation to the varying number of switches. The results are shown in Table 6, Table 7 and Table 8 and in Figure 3.
Table 6. Latency in milliseconds for OpenDaylight controller.
Table 7. Latency in milliseconds for Floodlight controller.
Table 8. Latency in milliseconds for ONOS controller.
Figure 3. Latency vs. number of switches.
Figure 3 shows the relation between the latency and the number of switches in a range from 1 to 16. OpenDaylight provides the lowest latency. Its values are approximately constant, which leads to the conclusion that they do not depend on the number of switches. The shape of the graphs for the Floodlight and ONOS switches is close to a linear function when increasing the number of switches. Floodlight achieved the highest latency values of those three compared controllers.

4.4. Latency in Relation to the Number of Unique MAC Addresses on a Switch

The last measurement using CBench conveyed the measurement of latency against the number of unique MAC addresses in the network. The results are presented in Table 9, Table 10 and Table 11, and in Figure 4.
Table 9. Latency in milliseconds for OpenDaylight controller.
Table 10. Latency in milliseconds for OpenDaylight controller.
Table 11. Latency in milliseconds for OpenDaylight controller.
Figure 4. Latency in relation to the number of unique MAC addresses on a switch.
It can be observed in Figure 4 that there is no specific relation between latency and the number of unique MAC addresses in the switch. Considering the measurement uncertainties visible as error bars in the graph, the function describing this relation can be compared to a linear function with a constant value. The lowest latency is achieved by OpenDaylight, while the highest by Floodlight. It can be concluded that the latency does not depend on the number of unique MAC addresses on the switch.

4.5. Controller Failure

A simulation of a controller failure was conducted in Mininet. A minimal topology was built—two hosts connected to a switch, as shown in Figure 5.
Figure 5. Network topology used in the controller failure simulations.
The time of a controller failure was measured as the time observed since the loss of connectivity between h1 and h2 (caused by the controller being turned off) to the restoration of reachability. It was observed during the experiment that, when using the OpenDaylight controller, turning off the controller did not impact the network connectivity. The switches did not remove entries from their flow tables despite the loss of communication with the controller. Such behavior can be considered beneficial, as network operation is maintained despite the failure of the controller.
However, to include OpenDaylight in the comparison, flows were manually removed from the switch after each simulated failure. The results obtained are shown in Figure 6.
Figure 6. Network restoration time after controller failure.
The majority of restoration time is the time it takes the controller to reboot. It also includes a significantly smaller amount of time from the time instant when the controller switches back on to the time instant when connectivity between devices on the network is restored. The results shown in Figure 6 contain the sum of both time periods. ONOS needs a very long time to start up, with an average of one minute and a maximum of one minute and 16 s, which is due to the fact that it is the most advanced controller and needs the longest time to load its functionalities. OpenDaylight obtained better results than ONOS, achieving a minimum of 38 s and a maximum of 44 s. Floodlight performed best, having the restoration time of, on average, 5 s and a maximum of 10 s.

4.6. Link Failure

To measure restoration time after a link failure, a script written in Python using the Mininet API was used. A network consisting of four switches was constructed, with one host connected to each switch (Figure 7). Additionally, a backup link was added between the two unconnected switches.
Figure 7. Network topology used in link failure simulations.
The link failure restoration time was measured as the time from the loss of connectivity between h1 and h2 caused by disabling the direct link between them, to the restoration of connectivity after switching to the backup link, as shown in Figure 8.
Figure 8. Network restoration time after link failure.
According to the study, in the event of a link failure, the OpenDaylight and ONOS controllers take approximately 1 s to switch to an alternative route. Floodlight, on the other hand, requires an average of 8 s to do so. In the event of a controller failure, Floodlight achieved the best recovery time. It restarts much faster than the other controllers. an average of 5 s, while OpenDaylight took approximately 40 s and ONOS as long as a minute.

4.7. Security

For security testing, a dedicated tool was used—sdnpwn. It is a penetration testing program that allows known attacks to be performed on selected controllers to identify their vulnerabilities.

4.7.1. Detecting the Presence of a Controller in the Network

The first experiment was conducted to determine whether it is possible to detect and identify a controller in the network. This was verified using the sdnpwn controller-detect module. The OpenDaylight controller was detected, though its version was identified incorrectly. The Nitrogen version was recognized as Hydrogen—the first version of the controller. The ODL Nitrogen run was incorrectly recognized as Hydrogen. Other versions (Lithium, Boron) were also examined, providing the same result. This may indicate the inability to determine the version without having access to the controller. The port number on which the GUI is located has been correctly identified, which may potentially pose a risk. An unauthorized user, knowing on which port the GUI is available, may try to log into it. If the administrator leaves the default password or sets a non-complex password, an attacker will be able to perform network changes from the GUI. Both Floodlight and ONOS were recognized correctly. In both cases, the version was not specified, leading to the conclusion that sdnpwn does not recognize versions at all. To minimize probability of attack, we suggest changing the default port on which the GUI is located and, of course, the default login and password.

4.7.2. LLDP Replay

In SDN networks, the LLDP protocol is used to detect links between devices. The controller recognizes the topology of a given network based on the LLDP packets received from the switches. The LLDP Replay attack consists of intercepting an LLDP packet generated by the selected switch and sending it from another location in the network. This results in a message being sent to the controller containing information about a link that is not actually present in the network. The controller includes this link in the topology and attempts to send packets through it, which will then fail to reach the recipient.
Figure 9 shows the topology used when performing the LLDP Replay attack. During the experiment, it was observed that ONOS is the only one of the compared controllers with protection against the LLDP Replay attack. Floodlight and OpenDaylight do not verify LLDP packets and may accept incorrect topology information, which may result in packets being lost or sent to the wrong recipient. We suggest implementing the LLDP Replay attack protection system similar to that used in ONOS in other controllers. They should be able to verify LLDP packets and not accept incorrect topology information.
Figure 9. LLDP attack simulation.

4.7.3. ARP Poisoning

ARP Poisoning is a type of attack in which an attacker sends forged ARP messages to another device on the network. The device places an entry in the ARP cache that binds an IP address from an actual device on the network to the attacker’s MAC address. The attacker can then receive messages directed to the other device.
This is not a type of attack specific to SDN networks, but a centrally managed network should implement mechanisms to protect against such an attack. The controller should monitor all ARP traffic on the network and prevent it from being “poisoned”. It is suggested to implement a protection system from one of the following groups:
  • Flow Graph-based solutions;
  • Traffic Patterns-based solutions;
  • IP-MAC Address Bindings-based solutions.
Methods to prevent an ARP Poisoning attack are described in [21].
The attack was carried out using sdnpwn’s dp-arp-poison module. Topology is presented on Figure 10. It allowed the MAC address of host h2 in the ARP table h1 to be replaced with the MAC address of the attacker (h3), causing all traffic destined for h2 to be directed to h3 instead of h2. This procedure was carried out successively for networks with the OpenDaylight, Floodlight and ONOS controller. In each case, the attack was successful. The lack of protection against ARP Poisoning means that other network attacks such as eavesdropping, Man-In-The-Middle and MAC Flooding are possible. This poses a serious threat to network security.
Figure 10. Topology used in ARP Poisoning attack.

4.7.4. Results Summary

In this section, we analyzed carefully selected parameters of SND controllers. We observed:
  • Throughput in relation to the number of switches—it was shown that the best results were obtained for ONOS and that for ONOS and Floodlight controllers, throughput decreases as the number of switches doubles.
  • Throughput vs. the number of unique MAC addresses in a switch—it was shown that the best results were obtained for ONOS and that throughput did not change significantly when the number of unique MAC addresses in the switch was between 100 and 100,000, for both ONOS and Floodlight.
  • Latency vs. number of switches—it was shown that OpenDaylight provides the lowest latency and its values are approximately constant; the shape of the graphs for Floodlight and ONOS switches is close to a linear function when the number of switches increases.
  • Latency in relation to the number of unique MAC addresses in a switch—it was shown that there is no specific relation between latency and the number of unique MAC addresses in the switch; the best results were observed for OpenDaylight.
  • Reaction to a controller failure—the lowest restoration time was noticed for Floodlight, while the lowest restoration times after a link failure were observed for OpenDaylight and ONOS.
  • Security issues—Floodlight and ONOS were recognized correctly in a network, while OpenDaylight was detected, but its version was identified incorrectly; ONOS is the only one of the analyzed controllers with correct protection against the LLDP Replay attack; for each controller the ARP Poisoning attack was conducted successfully.

5. Discussion

The OpenDaylight controller development slowed down recently due to an insufficiently large community of its programmers. Its successive versions differ significantly in terms of supported modules. It performs well in latency measurements but fails to provide any throughput measurements through the CBench tool.
Floodlight offers acceptable performance and provides stable operation and short recovery times. It is the lightest of the compared controllers and also has the fewest known vulnerabilities described in CVE.
ONOS offers the best performance of those three controllers, providing throughput twice as big as Floodlight. However, it performs worst in terms of response time in the event of failure. It is the only controller that provides protection against LLDP Replay.
The obtained results lead to the conclusion that none of the compared controllers is objectively the best. Each one has its advantages and disadvantages. The decision on choosing the best implementation for a given network should be made individually by the network administrator, considering the observations described in this paper.
We suggest implementing performance and security solutions similar to those already implemented in ONOS. However, all open-source drivers require further optimization and implementation of new effective solutions. To our knowledge, this is consistent with current practices in SDN controllers. One of the key issues is limiting the possibility of attacks on SDN controllers. Our analysis shows that open-source controllers lack effective security systems. We suggest implementing, among others: verification of LLDP packets and not accepting incorrect topology information, as well as implementing one of the available ARP poisoning protection systems.

Author Contributions

Conceptualization, A.M. and J.D.; methodology, A.M. and J.D.; validation, J.D., E.B. and R.W.; formal analysis, A.M.; investigation, A.M., J.D., E.B. and R.W.; resources, A.M. and J.D.; data curation, A.M.; writing—original draft preparation, A.M. and J.D.; writing—review and editing, E.B. and R.W.; visualization, A.M.; supervision, J.D., E.B. and R.W.; project administration, J.D.; funding acquisition, J.D., E.B. and R.W. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the Polish Ministry of Science and Higher Education with the subvention funds of the Faculty of Computer Science, Electronics and Telecommunications of AGH University.

Data Availability Statement

Data supporting reported results can be found at: https://drive.google.com/drive/folders/1EoChOKS9FVVVy298_Rm3TJKCs-LjeYtT?usp=sharing (accessed on 28 May 2024).

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Kantor, M.; Biernacka, E.; Boryło, P.; Domżał, J.; Jurkiewicz, P.; Stypiński, M.; Wójcik, R. A survey on multi-layer IP and optical Software-Defined Networks. Comput. Netw. 2019, 162, 106844. [Google Scholar] [CrossRef]
  2. OpenDaylight Documentation. Available online: https://docs.opendaylight.org/en/stable-argon/ (accessed on 28 May 2024).
  3. Floodlight Documentation. Available online: https://floodlight.atlassian.net/wiki/spaces/floodlightcontroller/overview (accessed on 28 May 2024).
  4. Onos Documentation. Available online: https://opennetworking.org/onos/ (accessed on 28 May 2024).
  5. Mendoza, D.H.; Oquendo, L.T.; Marrone, L.A. A comparative evaluation of the performance of open-source SDN controllers. Lat.-Am. J. Comput. 2020, 7, 64–77. [Google Scholar] [CrossRef]
  6. Salman, O.; Elhajj, I.H.; Kayssi, A.; Chehab, A. SDN controllers: A comparative study. In Proceedings of the 2016 18th Mediterranean Electrotechnical Conference (MELECON), Lemesos, Cyprus, 18–20 April 2016; pp. 1–6. [Google Scholar] [CrossRef]
  7. Gupta, N.; Maashi, M.S.; Tanwar, S.; Badotra, S.; Aljebreen, M.; Bharany, S. A Comparative Study of Software Defined Networking Controllers Using Mininet. Electronics 2022, 11, 2715. [Google Scholar] [CrossRef]
  8. Stancu, A.L.; Halunga, S.; Vulpe, A.; Suciu, G.; Fratu, O.; Popovici, E.C. A comparison between several Software Defined Networking controllers. In Proceedings of the 2015 12th International Conference on Telecommunication in Modern Satellite, Cable and Broadcasting Services (TELSIKS), Nis, Serbia, 14–17 October 2015; pp. 223–226. [Google Scholar] [CrossRef]
  9. Kaur, K.; Kaur, S.; Gupta, V. Performance analysis of python based openflow controllers. In Proceedings of the 3rd International Conference on Electrical, Electronics, Engineering Trends, Communication, Optimization and Sciences (EEECOS 2016), Tadepalligudem, India, 1–2 June 2016; pp. 1–4. [Google Scholar] [CrossRef]
  10. Tok, M.S.; Demirci, M. Security analysis of SDN controller-based DHCP services and attack mitigation with DHCPguard. Comput. Secur. 2021, 109, 102394. [Google Scholar] [CrossRef]
  11. Gautam, Y.; Gautam, B.P.; Sato, K. Experimental Security Analysis of SDN Network by Using Packet Sniffing and Spoofing Technique on POX and Ryu Controller. In Proceedings of the 2020 International Conference on Networking and Network Applications (NaNA), Haikou City, China, 10–13 December 2020; pp. 394–399. [Google Scholar] [CrossRef]
  12. Yoo, Y.; Yang, G.; Shin, C.; Lee, J.; Yoo, C. Machine Learning-Based Prediction Models for Control Traffic in SDN Systems. IEEE Trans. Serv. Comput. 2023, 16, 4389–4403. [Google Scholar] [CrossRef]
  13. Yoo, Y.; Yang, G.; Shin, C.; Lee, J.; Yoo, C. Control Channel Isolation in SDN Virtualization: A Machine Learning Approach. In Proceedings of the 2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid), Bangalore, India, 1–4 May 2023; pp. 273–285. [Google Scholar] [CrossRef]
  14. Abdou, A.; van Oorschot, P.C.; Wan, T. Comparative Analysis of Control Plane Security of SDN and Conventional Networks. IEEE Commun. Surv. Tutor. 2018, 20, 3542–3559. [Google Scholar] [CrossRef]
  15. Zhu, L.; Karim, M.M.; Sharif, K.; Xu, C.; Li, F.; Du, X.; Guizani, M. SDN Controllers: A Comprehensive Analysis and Performance Evaluation Study. ACM Comput. Surv. 2020, 53, 1–40. [Google Scholar] [CrossRef]
  16. CBench Documentation. Available online: https://floodlight.atlassian.net/wiki/spaces/floodlightcontroller/pages/1343657/Cbench+New (accessed on 28 May 2024).
  17. Laissaoui, C.; Idboufker, N.; Elassali, R.; El Baamrani, K. A measurement of the response times of various OpenFlow/SDN controllers with CBench. In Proceedings of the 2015 IEEE/ACS 12th International Conference of Computer Systems and Applications (AICCSA), Marrakech, Morocco, 17–20 November 2015; pp. 1–2. [Google Scholar] [CrossRef]
  18. Haggag, A. Benchmarking and Performance Analysis of Software Defined Networking Controllers in Normal and Failsafe Operations using Multiple Redundant Controllers. Turk. J. Comput. Math. Educ. 2021, 12, 5192–5202. [Google Scholar]
  19. Khattak, Z.K.; Awais, M.; Iqbal, A. Performance evaluation of OpenDaylight SDN controller. In Proceedings of the 2014 20th IEEE International Conference on Parallel and Distributed Systems (ICPADS), Hsinchu, Taiwan, 16–19 December 2014; pp. 671–676. [Google Scholar] [CrossRef]
  20. Elmoslemany, M.; Tag Eldien, A.; Selim, M. Performance Analysis in Software Defined Network (SDN) Multi-Controllers. Delta Univ. Sci. J. 2023, 6, 181–192. [Google Scholar] [CrossRef]
  21. Shah, Z.; Cosgrove, S. Mitigating ARP Cache Poisoning Attack in Software-Defined Networking (SDN): A Survey. Electronics 2019, 8, 1095. [Google Scholar] [CrossRef]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.