1. Introduction
Networks are organized groups of devices or nodes with communication links among them. Traditional IP networks are termed legacy networks. Despite the world-wide use, the traditional network is complex and tedious to manage. Adding a new device or changing the network configuration is complex in traditional networking, that is, it should be implemented using low-level languages and rigid commands thus taking days and months for large networks. Thus, the expansion of the existing legacy network is fairly expensive. The demand for network based services is increasing rapidly. To meet those growing demands, the numbers of running hardware should be increased and changes should be made in software leading to a higher cost in system upgrades. Manual configuration, inconsistent policies and inability to scale are the major problems of the traditional IP networks. Vertical integration has made traditional IP networks more complex in their operation and management. Legacy networking devices have the control plane and data plane integrated into the same physical device. The data plane, also called the forwarding plane, is a hardware unit whose job is to collect the network packets and forward them to the destination based on the entries provided in the routing table. Network policies are enforced in the control plane. The control plane determines how the packets in the data plane should be handled, for example, drop, reject, forward and so forth.
In SDN, network devices are controlled by an SDN controller and the applications are installed on the top of the controller. When the control and data plane functionalities are segregated, the network switch becomes simply a data plane forwarding device. The control logic is implemented in a logically centralized controller, which runs a network operating system (NOS) that runs on commodity server hardware to provide necessary resources to facilitate the proper control and operation of data plane devices based on a logically centralized, abstract network view [
1]. The logical centralization of the control logic makes it flexible and provides error-free policy deployment through high-level programming languages as an application programming interface (API), compared to the low-level device-specific configurations frequently used in legacy networks. The controller provides the global view of the overall network, which simplifies the development of more robust network functions, services and applications. The resulting network is also programmable through software applications running on top of the controller OS, which interact with the underlying data plane devices. The APIs at the controller can automatically react to spurious changes of the network state and keep high-level policies intact [
1].
Though SDN implementation in datacenter networks is popular, its proper implementation with the migration of a legacy network to carrier grade ISP (CG-ISP) and telecommunication (Telcos) networks is becoming a central challenge for service providers due to the need for real time migration as well as the higher cost of investment [
2,
3]. Additionally, real time implementation of SDN in production networks is the subject of ongoing research. Hybrid SDN is the only solution for the smooth migration of legacy ISP/Telcos networks into SDN [
4,
5]. In this paper, an emulated network environment is created in Mininet for hybrid SDN implementation and routing performance evaluation by comparing legacy routing and SDN routing with their interoperability through an experimental analysis using an open network operating system (ONOS)/SDN-IP [
6]. The viability of the SDN implementation in ISP/Telcos networks is then evaluated. The major contributions of this paper are as follows.
A multi-domain hybrid SDN environment is created and implemented in the routing between SDN and legacy networking domains.
Routing performance is evaluated, comparing legacy routing and SDN routing using ONOS/SDN-IP. Routing using SDN-IP has a better performance than legacy routing.
This experimental analysis evaluates the viability of SDN implementation in ISPs and Telcos networks. This encourages the service providers to smoothly migrate their legacy networks into SDN.
The rest of this paper is organized as follows.
Section 2 presents the background and related work in the field of SDN and legacy network integration.
Section 3 describes the research problem and the proposed method. We present the details of the experimental setup, analysis, and evaluations in
Section 4.
Section 5 provides a discussion and suggests future work, and the paper is concluded in
Section 6.
2. Background and Related Work
The cost of new technology implementation is generally higher in terms of capital and operational expenditure (CapEX and OpEX) investment and the development of technical human resources (HR) to manage and operate those newer technologies for ISPs and Telcos service providers. There are also certain implementation challenges with respect to management, availability of technological standards and user interface provisioning when providing new technology-based services to customers during and after network migration [
7,
8].
A legacy network is less flexible to customized programming and is more vendor specific, leading to higher dependency on support, management and operation. A better solution would be the implementation of SDN. In SDN, networks are controlled by software applications running on the top of an SDN controller. The separation of control and data plane operations in SDN simply transforms the network switches into simple forwarding devices.
NOS provides the essential resources and abstractions to facilitate the programming of forwarding devices based on a logically centralized, abstract network view. The resulting network is also programmable through software applications running on top of the NOS, which interact with the underlying data plane devices. ONOS [
9] is the robust and distributed controller OS, which provides a better solution for building next-generation SDN/NFV solutions. The need for ISPs and Telcos service providers can simply be fulfilled by the introduction of ONOS to carrier-grade solutions that leverage the economics of white box merchant silicon hardware while offering the flexibility to create and deploy new dynamic network services with simplified programmatic interfaces [
9]. ONOS is developed with a set of several other applications that make it flexible, modular, and scalable in terms of both the architecture and the cluster, and the distributed SDN controller. SDN-IP is one of the ONOS applications developed to implement routing with external legacy networks using the standard border gateway protocol (BGP) to enable interoperability with legacy routing. In ONOS/SDN-IP, the SDN can be treated as a single autonomous system (AS) and communicates simply with external AS, as with traditional routing. SDN-IP integrated BGP and ONOS enable communication with external AS in the hybrid SDN so that SDN-IP behaves like a regular BGP speaker and uses its services to install and update the appropriate forwarding state in the SDN data plane devices.
Dawadi et al. [
3,
10,
11,
12] presented approaches for legacy network transformation to SDN and IPv6 networking so that service providers can smoothly plan for network transformation with optimised migration cost. The authors implemented IPv4/IPv6 routing in a multi-domain SoDIP6 network using ONOS/SDN-IP.
Friyanto [
13] presented the use of multiple ONOS controllers in a cluster for high available services using ONOS/SDN-IP reactive routing. Efficient routing implementations in the different aspects of wired and wireless SDN [
14,
15,
16,
17] are the primary concerns, but their implementations in production level networks have to be evaluated considering the smooth transition from legacy to hybrid SDN to pure SDN.
There are many studies on hybrid SDN implementations. For example, HARMLESS [
18], Panopticon [
19], RouteFlow [
20], Fibbing [
21] and OSHI [
22] are some of the attempts, but most of this research has not been implemented at the production level. OSHI considers the hybrid switch, while our implementation only considers the hybrid SDN/legacy network as a mix network and routs implementations on that mixed network. Tomovic et al. [
23] presented a modified version of three dynamic routing algorithms viz. MIRA, DORA and SWP, to be implemented with the OpenFlow controller. These routing modules are implementable only in an SDN environment.
Similarly, few studies have dealt with the implementation of SDN in CG-ISP networks using ONOS/SDN-IP [
6,
24,
25,
26], even though ONOS is the best controller in terms of its features and dedication towards transport SDN over a wide area network [
27]. Kong et al. (2013) [
28] evaluated the performance of an OpenFlow switch in SDN by comparing it with legacy networking and found that OpenFLow based communication did not perform well. During that period, SDN was just at the emerging stage but now it is well matured [
29,
30,
31] and robust. Research on implementing SDN in an ISP network is in progress [
24,
32]. SDN is popularly implemented in data center networks [
33,
34] and its implementation in ISP networks is part of ongoing research. Because we are at the early stage of SDN implementation in service provider networks, we found limited or no research related to routing performance comparison in terms of BGP implementation in legacy networks and SDN in the service provider networks. Our approach in this study is not scoped towards migration techniques, instead we focused on the production level implementation of routing in hybrid SDN and its performance in comparison with legacy network to measure the viability of legacy network transition towards SDN by implementing ONOS/SDN-IP.
3. Problem Description and Proposed Method
The overall structure of our experimental evaluation is based on the generic architecture of ONOS/SDN-IP implementation [
35] as shown in
Figure 1 and generated the experimental output of our research use case using Mininet as shown in
Figure 2. All the external networks, that is, external ASs, are supposed to be the legacy networks, directly connected with backbone transit AS, which is SDN based.
For routing purposes, the Quagga routing suite [
36] was used, in which BGP was implemented. For SDN integration with a legacy network, the ONOS controller was configured to implement BGP and SDN-IP with reactive routing enabled.
A complete legacy network was created as shown in
Figure 3, where BGP and OSPF were used for routing purposes. Each router is a virtual machine loaded with BGP and OSPF configurations. In
Figure 3, r5 is a route reflector router [
37] and r4, r3, r6, r7, and r8 are its clients, whereas r4-r2 and r3-r1 are BGP peers.
The SDN integrated legacy network and pure legacy network were chosen in the present study to analyze network performance. For comparison between them (hybrid SDN and legacy network), a distributed internet traffic generator (D-ITG) [
38], a Packet InterNet Groper (
PING), Iperf, and statistical rules were applied. A varying number and size of packets were created and transmitted from one host to another in each network, and the corresponding delay, latency and total packet transmission were observed. The
PING utility was used to check the network connectivity between hosts. In both networks, first,
PING tool was used to ensure total communication in the network; then it was used for finding out RTT. We used
IPERF tool to determine the maximum bandwidths offered by the network. The compatibility and performance of SDN integration with the legacy network for multi-domain routing purposes were clearly depicted by the statistical values and comparison techniques required for SDN and the legacy network. Mininet emulated the hosts as a Linux machine. ONOS, running on the same machine, was used as an SDN controller.
The experimental setup of this study is depicted in
Figure 4. The experiment was executed in a virtual environment for which we used Oracle VirtualBox as the hyper-visor. The guest operating system was allocated with 4 GB of base memory and two processor cores. The experiment was carried out on the Ubuntu 14.10 (64 bit) operating system. The network topology file used Mininet to create routers, hosts and OpenFlow switches. Quagga version 0.99.23 was configured in the routers for BGP routing in SDN, while OSPF and BGP were configured for the legacy network. Routers and hosts received data routed through the OpenFlow switches. The ONOS (version 1.2.1 user) interface provided the visualization of the network. DITG, PING, and IPERF tools were used for the data collection. The collected data were analyzed and visualized using the Python Matplotlib module. A brief descriptions of the APIs enabled over ONOS are follows.
OpenFlow: It is the southbound protocol in the SDN control plane. It enables communications between the data plane and the control plane devices so that the control plane deploys flow rules in the OpenFlow table of the data plane and provides a decision for every packet incoming to the data plane devices.
Config: This application is activated for loading the network configuration.
ProxyARP: It is the process of resolving the address of a host. It responds to address resolution protocol (ARP) or neighbor discovery protocol requests on behalf of a target host if the target host is known to ONOS (i.e., it is present and active in the host store). ProxyARP considers the network to be a single IP network, therefore, if it receives a request for which the target host is not known, it will flood the request to every edge port in the network (apart from the input port) in the hope that the host exists in the network and will reply.
SDN-IP: It is an ONOS application for routing implementation that allows an SDN to connect to external networks on the internet using the standard BGP.
After running those APIs, we can see the routes learned by ONOS, which can be seen using the ’routes’ command on the ONOS console. In the legacy ASs, an eBGP router is connected with a single host. The SDN network in our proposed use case consisted of a BGP border router, which acts as iBGP to SDN-IP and eBGP speaker to the external legacy networks. The BGP policies advertised by external network border routers will be received by the ONOS/BGP speakers in the transit SDN and re-advertised to other external legacy networks by SDN-IP as an iBGP peer. SDN-IP translates the best route policies into the ONOS application intent request (AIR) and deploys them into data plane devices as forwarding rules to route the transit traffic into appropriate external domains. Internal BGP speakers were configured to use the TCP port at 179 to communicate with other BGP speakers and the TCP port at 2000 to send the route information to ONOS/SDN-IP instances, and the external BGP speakers were connected to OpenFlow enabled switches.
The BGP routers were emulated using the Linux hosts in which the Quagga routing suite was running. The BGP daemon of the Quagga routing suite was used for the BGP routers.
Figure 2 shows SDN-IP implementation for an ISP network, where BGP speakers 10.0.1.1, 10.0.2.1, and 10.0.2.101 were connected to OpenFlow switches. BGP speaker 10.0.1.101 was connected to ONOS, that is, SDN-IP. It advertises route information to SDN-IP; using this, the SDN network acts as a transit AS. BGP speakers 10.0.1.1 and 10.0.2.1 were connected to the external AS. These speakers advertise routes to the AS and to other BGP speakers. iBGP speaker 10.0.1.101 listens to these routes and sends them to SDN-IP. SDN-IP changes the known BGP routes to intents and ONOS turns the intents to OpenFlow entries. The OpenFlow entries are then deployed into the OpenFlow switches using the OpenFlow protocol.
Using D-ITG, UDP flows were generated with varying packet size and packet rate for 15 s. Two log files were generated for both the sender and receiver side. The log file generated at the receiver side contained values for different performance indicators such as delay, jitter, bit-rate, bytes received, packets dropped and average loss-burst size. On decoding the log file, we received a stream of data containing these performance indicators. The values of these parameters were taken from the log file on receiver’s side. Multiple unidirectional flows were sent for fifteen seconds between pairs of hosts in turns by varying transmission rate and packet size to study the performance of the network under low and high load. The packet size set were 512 and 1024 bytes and the packet transmission rates were varied to 100, 1000, and 3000 packets per second. These flows were sent in the traffic via UDP where each host acted both as a client and a server. Ultimately, the log files of flows generated from ten hosts (ref.
Figure 2) in the SDN network were further processed to a CSV file using a Python script. The D-ITG data samples generation snapshot is shown in
Figure 5.
Moreover, 507 data samples were captured during the simulation.
Figure 6a–d shows the pattern and distribution interval of the data samples for delay, jitter, bit rate and the packet rate, respectively.
The bi-variate distribution of numerical attributes can be observed in the pair-plot shown in
Figure 7. A pair-plot visualizes given data to understand the best set of features to explain a relationship between two variables (attributes) or to form the most separated clusters. The plot is in matrix format where the
x-axis is represented by row name and the
y-axis is represented by column name. The main-diagonal subplots are the univariate histograms (distributions) for each attribute. For example, the plot in the fourth row and the fourth column shows the univariate histogram for average-jitter, where the minimum and maximum of the parameter can be observed. On the non-diagonal subplots, bi-variate distribution between attributes can be observed. Some plots, such as those in the first column and the first row, represent separate clusters, which show that no relationship exists between the attributes. Certain scatter plots that follow a clear linear pattern with an increasing or decreasing slope can be seen, which shows that some conclusions can be drawn from the distribution. Plots with a linear relationship between the two attributes make it possible to identify in which space the classes will be well separated from each other. For example, the plot in the second row and the fourth column shows a linear relationship between the two variables. Such linear patterns represent dependencies among the attributes, bringing us to the conclusion that average delay and average jitter are correlated.
The pair-plot is an efficient tool for exploratory data visualization and analysis but does not show the strength of correlation between the attributes with dependencies. This can be achieved through a correlation heat-map using the heatmap() function available in the Seaborn package. The correlation of all the numerical attributes as a heat-map using the Spearman correlation coefficient is shown in
Figure 8. The scale of color in the heat-map varies from blue to yellow, where blue signifies a correlation of +1 and yellow signifies a correlation of −1. The Spearman correlation coefficient measures the extent to which two variables tend to change together. The coefficient describes both the strength and the direction of the relationship. The Spearman’s rank-order correlation is the non-parametric version of the Pearson correlation. Spearman’s correlation coefficient (
) measures the strength and direction of association or the monotonic relationship between two ranked variables. It requires two variables that are either ordinal, interval, or ratio. For a sample size of n, the attributes
and
are converted to ranks
and
. Spearman correlation coefficient (
) is computed by Equation (
1),
where
is the co-variance of the rank variables.
and
are standard deviations of the rank variables.
It can be observed from
Figure 8 that average delay and average jitter are highly correlated with a Spearman coefficient of 0.92. Similarly, bytes received, average bit rate, and average packet rate are highly correlated as well. Parameters, for example, average delay and average packet rate, can be observed to be negatively correlated, with a Spearman coefficient of −0.75.
5. Discussion and Future Work
Virtualization capability of SDN enables powerful APIs, which can be used to control many network functionalities with intelligence. The lack of backward compatibility of SDN with the legacy network enforces research communities to develop a robust system for interoperability between SDN and legacy networks for smooth and low cost migration approaches. Hybrid SDN is the only solution for having a smooth transition to pure SDN for service providers so that customers can receive uninterruptible services during the migration period. SDN-IP is an ONOS application that is used to peer SDN networks with external networks on the Internet using the standard BGP so that service providers can run hybrid SDN in their existing network during the transition period of their network migration. The SDN-IP controlled network acts as a transit AS that interconnects different legacy IP networks, considering each external network as a different AS domain, and interfaces with the SDN network through its BGP-speaking border routers.
In this study, an SDN integrated network was created with Mininet and experimental tests were carried out to ensure successful communication between the hosts in different ASs. The output of the tests showed a smooth transmission of data between hosts, thus providing testament to the possibilities of interoperability between two different networking paradigms. A similar legacy network was also created and observed to note the difference in the characteristics of an integrated SDN and legacy network.
To give statistical significance to this experiment, correlation coefficients of the QoS parameters were determined. These values can be further utilized with artificial intelligence technologies or to develop machine learning models or rules about the network. These insights can be useful in wide areas of applications such as traffic classification, routing optimization, quality of service and quality of experience prediction, resource management, security enhancement and many other purposes.
The comparison between legacy and SDN integrated networks aimed at verifying the speculations about SDN performing better than legacy networks. By studying both the networks based on key performance indicators, such as bandwidth, packet transmission rate, and round trip time, the performance of the SDN network was observed to be much better than its legacy counterpart. Similarly, the exploratory data analysis of the values of the QoS parameters collected from the network provided us with insights about the network.
The scope of this experimental analysis was limited to an IPv4 addressing network only. A routing performance analysis with the networks enabled with IPv6 addresses could further help to establish the significance and prospects of SDN implementation with future networking technologies. Additionally, this study was limited to comparing the performance of hybrid SDN and a legacy network to determine the feasibility of the migration of the legacy networks into SDN. We read some of the literature related to SDN routing approaches [
22,
23], already mentioned in
Section 2, since we are particularly dealing with routing in hybrid networks. Reference [
23] deals with dynamic routing algorithm implementation in a pure SDN environment, while the article OSHI [
22] considers hybrid switch networks. However, our scenario of routing implementation does not particularly match with the concepts provided in [
22,
23]. OSHI [
22] considers hybrid or dual stack switch, that is, legacy routing and OpenFlow implementation, in a single switch. The dual stack switch consumes a higher load as compared with a single stack switch because it has to handle dual packets routing/forwarding at once as well as a separate access control list needing to be maintained. Hence, we will consider the comparison of performance evaluation of routing in an SDN only environment using ONOS/SDN-IP with other SDN routing approaches for future work.
The next generation wireless networking viz. 5G/6G network is considered to be fully SDN based [
39,
40]. The paradigm shifts in mobile communications by the conceptualization and implementation of 5G/6G added a strong requirement for SDN in the modern network environment. The wired/wireless network and server virtualization, high speed communications at highly dense smart devices with ultra low latency for real time mission critical applications, energy efficient smart network deployment with efficient radio access network (RAN) design, smart spectrum management and implementation are the part of 5G and beyond networks that directly concern people’s modern living standards. Hence, from core network to end-access network service provisioning, cloud computing to fog computing to edge computing—they are mostly related to the availability of 5G and SDN, and the 4G to 5G migration and the performance evaluation of SDN based 5G network as compared with 4G networks are considered to be future work. Though network congestion optimization is not within the scope of this study, queue length analysis of the network device provides the trade-off between cost and performance of the network to understand the impact of congestion in SDN. Queue length analysis is also considered as future work.
Routing implementations over hybrid SDN with multiple controllers and their placement, a large and varying number of data plane devices, and so forth, could be the subject of future work to reflect the situations of CG-ISP networks when implemented into reality. A single instance of an ONOS controller is used in the present study. Multiple instances can be run at the same time for high available services. Multiple instances help in the load balancing of the whole network and add reliability to the network. Additionally, other available network controllers can be used to replicate the results for comparison in addition to ONOS.