Practical Implementation of a Blockchain-Enabled SDN for Large-Scale Infrastructure Networks

Abstract

The network function virtualization (NFV) feature lies at the core of modern networking, and it allows on-demand real-time integration of new network functions, which is a great benefit for large-scale infrastructure networks. In contrast to the functional benefits, NFV introduces software complexity and computational overhead through additional abstraction layers. The current article addresses the function validation problem in large-scale infrastructure networks of Internet Service Providers (ISPs) and proposes the utilization of blockchain as a validation technology, as opposed to implementing a custom validation solution. The current work showcases a practical architecture implementation to address the service validation in service provider large-scale networks. The POX-based solution to control software-defined networks (SDN) for NFV is extended to offer additional blockchain capabilities. Thus, a blockchain node is integrated and executed in the POX SDN controller. Transaction experiments are performed between two endpoints located in remote locations on the Internet, and the detailed results are presented to validate the utilization of the blockchain technology used on SDNs’ control plane.

1. Introduction

In recent years, modern networking concepts have evolved far beyond the traditional network systems where the network devices were not flexible with their services and they only contained a predefined set of supported protocols and instructions. Such obsolete hardware devices are tedious to update with firmware upgrades, and sometimes an easier solution is to simply replace the devices with newer products. All of this manual interaction with the hardware has led researchers and major network operators to design new ways of designing networks using the principle of network function virtualization. NFV assumes that any hardware device can run hypervisors that enable and disable services depending on the current business demands. This allows for seamless real-time software updates of the middleware network devices that create the Internet and gives the opportunity to enhance the network devices with multiple paradigms (e.g., SDN) and with multiple technology stacks (e.g., blockchain, AI, etc.). Therefore, heterogeneity in regard to both hardware and used technologies is a main characteristic of modern networking.

The current paper explores complex practical use cases where Internet Service Providers (ISPs) can improve their services by integrating the SDN paradigm together with a blockchain system. The solution proposes the use of SDN controllers as blockchain nodes and presents an architectural solution for managing large-scale infrastructure networks, specifically a federation of heterogeneous ISPs, that integrate the proposed solution. The SDN architecture adds flexibility to network management through NFV, improving network efficiency using new high-level abstractions. The blockchain system adds transparency, trust, security, ledger traceability and improved energy efficiency through optimized resource allocation in the business network. The practical use cases refer to both business design ideas, but also to the implementation details of the proof of concept that was created, which is using Mininet as the SDN platform in a Linux environment and the POX SDN controller, whose source code was enhanced with blockchain functionality using the go-ethereum implementation of the Ethereum protocol. One of the objectives of the current research is to provide a framework for developing ideas and other practical use cases with the aforementioned practical implementation details, which, to the knowledge of the authors, is one of the first attempts to integrate these technologies in a concrete proof of concept.

One use case that is supported by the current article is highlighted in Figure 1, namely, it shows multiple virtual ISPs (vISPs) who individually manage their own internal networks using the SDN architecture, having an SDN controller that manages the networking infrastructure. The ISPs are part of a federation, and each ISP can offer services to any other ISP in the federation, e.g., the domain registration service, which may only be available from certain ISPs, not all of them. A federation refers to a group of entities that adhere to, and undertake to conform to, integrate a certain negotiated standard and function according to the adopted standard. The blockchain system is used to validate the service provisioning of an ISP for the other ISPs. The blockchain system is implemented on the SDN controllers, i.e., the SDN controllers communicate between themselves over the blockchain network, validating and confirming that the services are indeed correctly and legitimately being provided.

The proposed solution is relevant in the context of the evolving technology landscape and the demands of modern network environments for several reasons:

• It is using well-established technologies for large-scale systems, namely Software-Defined Networking and blockchain;
• It encourages the collaboration between ISPs and offers a technical solution for the formation of heterogeneous ISP federations;
• ISP federations bring benefits to customers, improving their internet experience and access to services;
• It provides an architectural framework for improved network scalability, security and reliability;
• It can reduce energy consumption and optimize resource allocation by integrating blockchain nodes directly on SDN controllers.

Novel concepts are introduced in the current research to integrate blockchain and SDN technologies on large-scale infrastructure networks. The main goal of the article is the integration of blockchain technology into SDN infrastructures, mainly using SDN controllers as blockchain nodes across a federation of ISPs. Existing solutions in the state of the art do not offer a complete way of integrating the two technologies and are focused mainly on specific architectural or functional aspects (for example, using blockchain as a distributed encrypted archive). By using existing SDN controllers as blockchain nodes, organizations can optimize resource utilization, reducing the need for additional hardware. SDN controllers typically have sufficient computational resources and storage capacity to accommodate blockchain node functionalities. The proposed use case—using SDN controllers as blockchain nodes across a federation of ISPs—also highlights the business aspects by providing operational efficiency and scalability. The main originality aspects of the current article and the complexity of the proposed solution are highlighted below:

• This is one of the first integrations, to the best of the authors’ knowledge, between SDN and blockchain, where SDN controllers are used as blockchain nodes at the ISPs’ level, using emulators and physical devices;
• An architectural solution for interconnecting heterogeneous ISPs in an ISP federation;
• This work contains, to the best of the authors’ knowledge, the first Pox SDN controller extensions with blockchain features;
• In order to manage the subject of integrating the two technologies from a complete architectural view, without focusing only on specific aspects, a novel concept was introduced: vISP (virtual ISP) that allows the creation of a federation of ISPs and abstracts the complexity of interconnecting heterogeneous ISPs;
• With the introduction of the concept of virtual ISP over the ISP federation, the multitude of ISPs can be treated uniformly, and the problem of ISP heterogeneity can be addressed and solved. In this regard, a virtual ISP will consist of ISPs that choose to participate in the blockchain integration over their SDN infrastructures and will have the required high-level architecture role needed to treat uniformly heterogenous ISPs;
• The complexity of current work derives also from the integration of multiple tools, technologies and programming languages to create the research and test environment. The Mininet network emulator [1] was used to create the needed network topologies in the testing setup. Geth was used to create the blockchain network. The source code of the POX SDN controller was modified to integrate the Geth functionality and thus create the SDN–blockchain integration at the technical level. Python programming language and Linux bash scripting language were utilized to achieve this integration. Wireshark was used as a network monitoring tool to validate the implementation and test blockchain traffic being transmitted from the POX SDN controller customization.

The remainder of the article is structured as follows: Section 2 provides work related to the current research. Section 3 describes the system design and architecture. The implementation details are provided in Section 4. Section 5 presents the obtained experimental results. Section 6 discusses the current research in the context of other tools and technology currently used in the industry. Section 7 concludes the paper.

2. Related Work

This section addresses related research to blockchain, ISP networks, SDNs used in the WAN with blockchain enhancements and simulator utilization in the aforementioned domains.

In recent years, blockchain technology has garnered substantial attention. In the following section, we highlight articles that discuss applications of blockchain, focusing on scenarios that are of particular relevance to the current research. The study presented in [2] proposes a system for cross-blockchain smart contract synchronization. The system creates a client contract on another blockchain that mirrors the logic and state of the original contract. The client contract is synchronized in a verifiable manner using Merkle proofs without the need for trusted intermediaries. The system also supports lightweight synchronization due to transition confirmations without the need for transaction re-execution. The system requires that the targeted blockchains support the same execution environment (i.e., Ethereum Virtual Machine-compatible blockchains). The study showcased in [3] proposes a solution that combines federated learning (FL), blockchain technology and multi-agent systems in order to obtain an architecture that enables the training of a neural network to solve a specific problem. The proposed solution is inspired by the smart contract concept and uses smart agents that have both the role of a peer in the blockchain network and the role of being a participant in the FL task. The study obtains a distributed solution that avoids being vulnerable to the information leakage attacks of the central server. The study in [4] researches the means of obtaining a solution that provides secure storage and access to the task-scheduling scheme on a consortium blockchain and interplanetary file system (IPFS). The study compares different versions of Hyperledger Fabric in order to evaluate their functionality and features. Also, they select a blockchain as a service provider that supports Hyperledger Fabric in order to use and validate the proposed solution. The study presented in [5] proposes a new consensus algorithm for a lightweight blockchain. This type of implementation targets distributed systems with resource constraints such as IoT architectures. This solution aims to introduce decentralization in these systems by using blockchain technology with minimal impact on the associated devices. Also, with the proposed solution, the blockchain becomes application-specific.

Next, articles are presented that address ISPs’ integration or ISP activity over network traffic. Paper [6] presents the ISPs’ interactions and the types of ISPs. Inter-ISP communication is presented in regard to traffic control, and the need for new dynamic control protocols related to this is discussed. Inter-ISP cooperation techniques, such as caching and content-peering, are presented. In this context, the authors propose a cooperation scheme for inter-ISP traffic control that uses game theory (cooperative game). The interactions between ISPs are formulated as a two-step bargaining game model in which the steps contain the main issues: the service price and the limited bandwidth. The proposed cooperation scheme has the goal of reaching a consensus between ISP interactions while maintaining fairness for every participant. The study presented in [7] has a similar focus. The interactions between ISPs of identical and diverse tiers are explored, along with the interaction involving content providers. The study advocates for a solution that implies cache sharing between the different entities and also between different tier ISPs. Using aspects derived from game theory, a solution for optimizing pricing negotiations in collaborative settings, which takes into consideration QoS (Quality of Service) and OPEX (operational costs), is investigated and proposed. In [8], the effect of the zero-rating method is analyzed as a differential pricing scheme. Models with both monopolistic ISPs and multiple CPs (Content Providers) and oligopolistic (multiple) ISPs with multiple CPs are considered and studied, in order to predict how this would affect end -users, CPs and ISPs. The complex interaction system between the mentioned players was modeled as a non-cooperative game. Based on the analysis, the differential pricing scheme can be unfair to the CPs with better QoS and can promote the situation in which the CPs with poor QoS receive more traffic, which will downgrade the experience for the end user. The study concludes that CPs with poor QoS should not be able to offer higher sponsorship to ISPs than CPs with better QoS, for the end user to have a similar or better experience. The study in [9] proposes a solution for automating NOC (Network Operations Center) tasks and helping with taking the recommended actions for maintaining the network’s health and assuring SLAs (Service Level Agreements). The proposed solution is said to help ISPs balance between offering high QoS to customers in order to meet SLAs and keeping the operational costs at a minimum level. The study used supervised machine learning to train the model used in the solution and obtain automation for selecting the best action, the results in the controlled environment being similar to the ones of expert technicians.

Next, articles are presented that use SDN technology in the WAN together with blockchain. The study in [10] proposes a solution to improve SDN security for IoT deployments, using blockchain. The solution proposes three key elements for every IoT deployment: IoT Application Server, SDN network and the secure gateway node introduced by this solution. This node is connected to the sensors and gathers the communication from sensors and redirects it to the SDN network nodes. The secure gateway validates the external traffic. The authenticity of the external traffic is validated using blockchain as the primary distributed encrypted archive which has the up-to-date information. The information here cannot be modified or corrupted, so the authorized nodes can keep track and verify the data generated by the IoT devices by checking the most recent blocks. In [11], a secure distributed SDN architecture is proposed for IoT using blockchain. This architecture is called DistBlockNet and adopts a secure distributed control of the network, using blockchain to improve security, scalability and flexibility without using a centralized controller. In the proposed architecture, the controllers in the IoT network are interconnected in a blockchain network-like manner to facilitate the communication between the IoT forwarding devices. There are three modules used: OrchApp, Controller and Shelter. The network traffic is monitored and filtered by the proposed modules. The architecture assures operational flexibility and incident prevention based on recurrent threats. The study claims that the architecture is agile, modular and safe, adapting dynamically to threats without any intervention from administrators. The distributed blockchain network contains verification nodes and forwarding nodes. The verification nodes ensure that the flow rules in the database are up to date. The forwarding nodes update the flow rules table from the blockchain network. The rule set obtained from a controller is verified by checking if there is synchronization with the other nodes and the rules are not altered and up to date. The work in [12] provides a comprehensive survey of SDN integration with blockchain technologies for IoT networks. It explores the potential benefits and challenges of this integration, named BC-SDIoT, and offers insights into current research and developments in this promising area. Integrating SDN with blockchain offers a feasible solution to cybersecurity challenges in IoT. Paper [13] proposes a system that uses distributed SDN controllers over different network domains for managing the inter-domain SD-WAN (Software-Defined Wide-Area Networks) networks. The integration of blockchain technology addresses the heightened security vulnerability in the control plane posed by SDN controllers being located in different domains. The blockchain is installed to run on every controller. By integrating blockchain, the system can use the smart contracts feature that the blockchain offers. The proposed system uses an architecture with two smart contracts defined in the same chain of the blockchain network. One of the smart contracts has the role of helping with the authorization, enabling controllers to establish trust between themselves and with the management of the controllers. The other one has the role of helping with coordination, validating if the participating controller has been authorized. Ref. [14] proposes a framework that provides authentication and auditing capabilities to tenants in multi-tenant, multi-vendor network environments. The framework uses blockchain as an addition to SDN (SD-WAN) technology in order to solve the issues of a single point of failure—central points of operation related to the centralized aspect of SDN, by a decentralized orchestration. This orchestration can be acquired by using tenant and vendor nodes from a peer-to-peer network. These nodes use the decentralized applications and smart contracts characteristics of blockchain technology. In this way, the framework manages to allow the SDN clients and vendors in the presented environment to handle services in an auditable and zero-trust-based way. The study in [15] proposes an energy trading scheme based on SDN and blockchain in order to obtain a distributed energy internet. The proposed scheme leverages SDN to obtain a separation of the operation, control and management of the energy internet that will add flexibility to it. Besides the advantages, some disadvantages of using SDN are presented, and the use of blockchain technology is suggested in order to address the issues regarding the privacy and the security of the energy transactions. The study concludes that further research is needed in order to obtain a practical solution. The work of Oktian et al. [16] proposes a solution to manage network bandwidth using SDN and blockchain at an ISP level, identifying three major use cases. The SDN will automatically manage QoS configurations, and the blockchain will provide a decentralized payment system. The paper provides a theoretical approach, deferring the technical details regarding the proposed use case protocols for further investigation. The research in [17] continues this work by implementing the solution using Ethereum and POX SDN controllers. The results proved that the framework is capable of trading bandwidth resources. The SmartContractChain (SC2) [18] traffic management framework integrates flow-based control from SDN with the decentralized technology of blockchain and embedded smart contracts. Several simulations have been conducted to demonstrate the feasibility of the SC2 framework.

Table 1. Related work and proposed approach summary.

Reference	Blockchain	SDN	ISP
[2]	cross-blockchain smart contract synchronization	-	-
[3]	blockchain and federated learning	-	-
[4]	consortium blockchain and IPFS	-	-
[5]	consensus algorithm for a lightweight blockchain	-	-
[6]	-	-	inter-ISP traffic control
[7,8]	-	-	game theory, ISP interaction and QoS
[9]	-	-	supervised machine learning, ISP and QoS
[10]	blockchain and IoT	SDN network	-
[11]	blockchain for flow rule table	SDN and IoT network	-
[13]	blockchain-enabled secure coordination among controllers	SDN and SD-WAN	-
[14]	blockchain-agnostic	SDN network	multi-stakeholder SDN
[15]	energy trading schemesupported by blockchain technology	SDN network	energy internet
[18]	blockchain integrated into the SDN controller nodes	SDN network	multiple ISP
[proposed solution]	blockchain integrated into the SDN controller nodes	SDN network	heterogeneous ISP federation

summarizes the related works and compares them with the proposed approach.

The aforementioned related research addresses the general concepts encountered throughout the current paper. Subsequently, this section provides pertinent research findings with a focus on the technical details involved in deploying the proposed solution. The study in [19] explores the various SDN controllers and other tools that are useful in SDN experiments. The experimental work includes three network topologies with different numbers of switches and with different SDN controllers being used in order to simulate and measure throughput, jitter, latency and latency for both TCP and UDP traffic. With their work, the study provides insight related to the use of each controller. The use of Mininet as an environment for network simulation is also validated as being appropriate. In [20], different network topologies are used in order to simulate and compare the POX and Ryu SDN controllers provided in Mininet. Different metrics were taken into consideration, and from the simulation results, the utilization of Ryu was considered to be the one that is more favorable. The study in [21] uses the Mininet-Wifi network emulator with the provided Ryu controller. The objective was to simulate an SDN environment to assess the efficacy of multipath routing in enhancing QoS.

3. System Proposal

In the preceding sections of the article, the utilization of ISP federation and virtual ISP were theoretically formulated and conceptualized. This chapter elaborates on their potential interactions and communication methods, presenting a proposed model for the integration of blockchain and SDN technologies.

When observed externally, the ISP federation collectively engaging in the integration of blockchain across their SDN infrastructures can be conceptualized as a virtual ISP. Thus, for the external ISPs, the federation is seen as a single, larger ISP. The communication between external ISPs not affiliated with the federation and the ISP federation mirrors that of standard communication between two conventional ISPs (external to the federation). In this way, homogeneity and uniformity can be obtained on both levels of the heterogeneous environment between the service providers. A homogeneous environment can be found within the federation, as all the service providers take part in integrating the blockchain technology over their networks. Also, a homogeneous environment can be found in the exterior too, as the federation is considered a virtual ISP in the way that it behaves as any other conventional ISP toward the rest of the participants.

In the proposed model, to preserve privacy, an ISP should not be able to access/read data or information belonging to the other ISPs. Furthermore, the interaction of a service provider with the federation blockchain should not affect, by no means, the interaction of other service providers with the blockchain. Thus, the proposed system leverages blockchain technology to guarantee data integrity while simultaneously preserving data confidentiality.

Considering the above aspects, the subsequent section will describe the architectural framework of the ISP federation platform. The proposed solution assumes that each ISP will establish a connection and participate in the creation of a permissioned blockchain within the federation. Any new ISP joining the federation will also connect to this blockchain and expand it further. The permissioned blockchain will be used as the repository for data selected by the service provider for storage within the context of integrating the blockchain into its SDN network. Thus, when the ISP joins the federation, it integrates at least one blockchain node that will provide access to the blockchain in order to store and read the desired information. Figure 2 illustrates how the ISPs interact with the blockchain, showcasing various methods of interaction. The blockchain integration within service providers’ SDN networks remains consistent; however, the communication with the blockchain varies depending on the specific integration approach employed by the service provider. Since the service providers implement and run a full node, the interaction with the blockchain is direct. It also depends on where the service provider implements the blockchain node. Usually, the blockchain node and the integration with the blockchain are implemented at the SDN controller level. In this manner, the controller communicates directly with the blockchain, both for storing data on the blockchain and for reading information from the blockchain.

Every SDN network within the federation belongs to an organization (ISP). The control authority of the SDN network is administered internally from within the organization; thus, every SDN network is a part of the consortium that has permission to access the permissioned (consortium) blockchain. All the data that correspond to the criteria established by every ISP in part are stored in the blockchain. The SDN network controller manages the communication with the other (external) blockchain nodes and the operation of the local blockchain node. Every ISP is able to store the information marked as being of local importance—information that the ISP does not want to share or that is strictly regarding the details of the internal network—in a manner that ensures that other participants will not be able to read it. The data marked as public are stored in the blockchain without any other additional intervention. The blockchain can function as a secure platform for sharing data marked with this scope, besides maintaining all the information from all the providers safe, both public and private. The blockchain operation is maintained through the active involvement of every ISP in the federation, by operating a node (at least).

In Figure 3, the interaction between the service providers and the integrated permissioned blockchain can be observed. The service providers have several options available for integrating the blockchain. These alternatives are related to the communication methodologies established between the service provider and the blockchain. The blockchain integration will manage the data processing tasks essential for storing information on the blockchain platform.

The ISPs maintain permissioned blockchain nodes in their SDN networks using SDN controllers. The permissioned blockchain stores common information (public data) of interest for all the service providers in the federation, but it also stores private information—data that are of interest and that should be available only for the ISP that committed the data to the blockchain, as already mentioned. Using private key sets, every ISP is able to store private data—data strictly related to its own network—on the permissioned blockchain. In this way, the data are encrypted when stored on the blockchain and could be decrypted only by having the keys. Thus, every ISP can have a private section of data on the federation’s permissioned blockchain to use as they prefer. Leveraging the federation blockchain by integrating it over the SDN network not only enhances security but also enhances the other properties of the blockchain due to the extended surface of the blockchain. As an increasing number of ISPs adhere to the federation and integrate the blockchain, the area of the blockchain expands, resulting in enhanced security of other intrinsic attributes of the blockchain.

Blockchain technology can be extended within the platform over the service providers’ SDN networks that adhere to the federation. This process can be carried out by extending the SDN network devices’ functionality with a part that handles blockchain functionality. As it was previously mentioned, the blockchain nodes (full/complete) offer a detailed perspective of the information stored in the blockchain. The SDN network devices, according to their capacity, can take over blockchain node functionality. The main coordinator for an SDN network is the SDN controller. This network device controls and ensures the good functioning of the whole SDN network having control over all the network devices. Thus, such a device would be appropriate to take over the functionality of a full blockchain node within the integrated blockchain extended over all the SDN networks from the federation. In this manner, the SDN controller also has control over the data that will represent the transactions stored in the distributed ledger and, implicitly, control over the new integrated part of functionality.

Due to redundancy considerations and enhanced security requirements, the centralized control component of the SDN network can be improved through the adoption of distributed controllers, as indicated by existing research in this domain [11,13] and as survey articles on SDN technology also demonstrate interest over time, as indicated by the research findings [22,23,24]. Taking inspiration from the distributed manner of the blockchain, such an implementation would increase the covering area of the SDN network and would also benefit the proposed model. By taking the capability of a blockchain full node also, the controllers could increase the covering area of the blockchain integrated within the federation. Thus, by replicating the mentioned model within the whole federation of service providers, the implemented integrated blockchain would have an increase in benefits from the perspective of the features and properties that come with the blockchain. The importance of extending the blockchain over the SDN networks is given by the constraint that the blockchain has—the necessity for a sufficient extension to render data modification difficult or virtually impossible. This goal can be achieved by including as many service providers as possible in the federation of SDN networks. Benefiting from such an organization, the abstraction of the federation to what is referred to as a virtual ISP is easier to comprehend and implement. This is also the focus presented in [25], where a collaborative game theory approach is proposed for validating the feasibility of integrating a blockchain solution in a federation of service providers. A set of incentives organized as a mechanism is proposed in order to stimulate cooperation between multiple service providers and to demonstrate that it is more beneficial for service providers to adhere to such a federation.

Drawing from the outlined proposed model, several use cases warrant exploration. A potential application scenario involves leveraging the permissioned blockchain within the ISP federation to promote collaboration among ISPs. Service providers would have the opportunity to disseminate public information using the blockchain, offering benefits and assistance to fellow participants within the federation. This approach ensures the accessibility of information to all involved parties. Communicating and crowd-sharing information regarding problems on different segments of the networks can be of benefit to other ISPs. This approach facilitates the implementation of effective measures for resolving and mitigating the identified challenges. Solutions to different issues of common interest can also be stored in the blockchain, including information pertaining to security issues, threats, vulnerabilities and recommended solutions. Thus, the distributed ledger will also act as a history logger.

Regarding ISPs’ interaction, several advantageous use cases could be derived. As presented in [6,7], it may be beneficial to use shared caching mechanisms for collaborative information retention. The proposed solution would allow for incorporating negotiated prices and contracts into the integrated blockchain, for enhanced operational efficiency. This approach ensures the immutability of information, allowing for historical verification and analysis. In the context outlined by [8], the ISPs could leverage the integrated blockchain within their SDN infrastructure to record agreements with CPs. This approach ensures the availability and transparency of information while preventing data alteration.

4. Implementation and Validation Steps

In the previous chapter, the conceptual architecture solution for the system was described. Since the presented architecture requires a blockchain network with multiple nodes available and running, and also several entities/organizations that will run at least a blockchain node, the actual practical implementation is not straightforward, but rather difficult to achieve. To show the validity of the solution, the chosen approach involves implementing a Proof of Concept through the simulation of participants responsible for operating the blockchain nodes.

The initial phase of the experiment involves the validation of the operability of blockchain nodes within a computing system. To demonstrate this step, a virtual machine running a Linux distro OS was used. On the virtual machine, a blockchain solution was configured and set up. Ethereum was selected as the blockchain solution owing to its versatility, incorporation of smart contract capabilities, and widespread popularity. In order to integrate Ethereum as the blockchain solution, Geth [26]—which is an implementation of Ethereum and has been a core part of Ethereum since its beginnings—was installed. After the environment set-up phase, a private blockchain network was configured consisting of several nodes that run on the Virtual Machine (VM) and that can interact with each other within the private blockchain network.

Having accomplished the initial phase, the subsequent task involves establishing an additional system dedicated to running blockchain nodes. The objective is to confirm the seamless integration of nodes from both systems into a private network, enabling effective interaction between them. Hence, in order to validate the approach, a new virtual machine was employed having identical configuration parameters as the one previously presented. Upon confirming the operational status of both virtual machines (VMs) within a shared network, it was verified that the private blockchain network could successfully expand across the secondary system. The blockchain nodes residing on the two VMs were observed to be integrated within the same private network, demonstrating their capability to engage in seamless interaction. With this step, it was validated that the private blockchain network can be extended over multiple systems.

Given that the proposed solution comprises a federation of multiple organizations/ISPs functioning as autonomous systems with individual networks, additional validation was requisite to illustrate the efficacy of the private blockchain network across multiple interconnected networks or the Internet. In order to validate this requirement, an additional VM located in a remote location was employed. Upon configuring the requisite parameters pertaining to the connectivity of virtual machines (VMs) with their respective networks and establishing the necessary port settings for Internet accessibility, the validation phase for the remote communication among blockchain nodes was prepared for execution in the experiment. The blockchain nodes hosted on the remote VMs demonstrated the capability to establish connectivity and interact. This step validated that remotely located blockchain nodes, from different systems, can successfully form and join the same network and conduct the necessary interactions.

The ability of multiple systems (autonomous systems) to extend and to form, or to be part of, the same blockchain network validates the first phase of the proposed solution. Validating that blockchain can be integrated over the SDN networks of participating service providers within the federation is the next required phase.

The next step aims to demonstrate that SDN devices are able to implement a blockchain node and integrate the blockchain over the SDN network. To achieve this objective, Mininet was configured on the VM. Mininet was selected for its capability as a network emulator, providing the option to create a network of virtual hosts and network devices such as controllers and switches. Mininet allows the simulation of an SDN network deployed by a service provider, bringing us closer to the requisite validation criteria. Mininet can also integrate and use SDN controller solutions, such as the POX controller. Having the POX controller connected to the Mininet network, a blockchain node is run via Geth on the controller. This was needed to confirm the possibility of running a blockchain node on the SDN network controller and that a node is able to join the other nodes within the deployed blockchain private network. Having the blockchain node running on the POX controller, it is possible to confirm that the node is able to join the private network and interact with the other blockchain nodes that are running on the other systems (virtual machines). Thus, it was validated with this step that it is possible to have multiple systems/autonomous systems that are able to integrate and use blockchain nodes to join and form a blockchain nodes network. Furthermore, it was validated that blockchain nodes can be integrated over SDN networks using the SDN controller, which brings us to the proposed solution.

In order to implement the details mentioned in the previous section, the functionality of the POX SDN controller was expanded to support the execution of the Ethereum implementation, namely Geth. The host_tracker module of POX was enhanced with additional source code designed to start the Geth process in the underlying operating system which is running the Mininet. It is important to note here that the POX SDN controller running in the Linux OS is actually starting an executable that is available to any other process in the operating system, POX being responsible only for the lifetime of this process and not for the implementation details of this process that was being run. Thus, the operating system is responsible for ensuring the blockchain communication between the Geth nodes running on different computers. Since this experiment was run on different computers located at different locations in the WAN behind different routers, the port forwarding option needed to be configured on the routers to allow connection redirection to the respective POX instances. The POX instances are bound to a specific IP address and a specific port number. The IP address is configured statically on the Linux OS, and the port number is manually chosen during the implementation and testing phases. Given that each blockchain node will use designated ports during operation and utilize both TCP and UDP protocols for communication, it may be imperative to configure permissive rules on the operating system’s firewall to facilitate external communication with other nodes.

The blockchain nodes need to be created locally before being used (have their folder where information like address and access password is kept). The configuration file (genesis.json) pertinent to the blockchain network, within which the nodes are operational, must be made available to these nodes. The nodes’ hexadecimal addresses need to be added to the configuration file along with their balance. The bootnodes should also be specified. As a note, the balance of a node should be high enough in order to be able to generate transactions. If this condition is not met, the behavior may be unexpected and the provided information unclear and misleading.

The following set of instructions comprises the main operations essential for the initiation of Mininet, POX, and Geth instances (these procedures represent a subset rather than the exhaustive compilation of commands, encompassing only the most relevant commands):

• Start POX
• Start Mininet
• Setup Geth:

• The establishment of a Geth blockchain node can be achieved through the following procedure:

>geth init –datadir node1 genesis.json, where node1 is the specified node, and genesis.json is the configuration file.

• A node can be started using the following command:

>geth --datadir node1 --port 30306 --bootnodes enode://2ef258931a0bb2eb4ae009cc70ad2f32442936a36f83aef5750d64f69a7da4bfa5c1d7bab9dd9e7004b697c702d5f830d6df10fb3644de6392e74ad610e4332@192.168.132.129:0?discport=30305 --networkid 9357737739201 --unlock 0x802bcEA8E0f9Bc7331d74196981193Aeb4eEF891 --password node1/password --authrpc.addr 192.168.132.129 --authrpc.port 8551 --mine, where the values for --port, --bootnodes, --networkid, --unlock, --password, --authrpc.addr, and --authrpc.port parameters are decided and given by us based on the used configuration; Note 1: --bootnodes can contain more than one such value; Note 2: There are other parameters that can be used such as (only some parameters that were used in experiments are mentioned in the command): --nat, that can specify the IP address on which the node can be publicly accessed from the Internet; --netrestrict, that can specify a set of IP addresses to which the network access will be restricted; --mine, that will set the node as a miner node.

• Locally, on the host machine where the Geth blockchain node runs, the node can be accessed to query network properties information using:

>geth attach node1/geth.ipc; Note: then commands such as: net.peerCount can be used for retrieving the number of nodes connected to the node; admin.peers can be used for obtaining information about the currently connected nodes.

As presented in the preceding paragraphs, a proof of concept was created to demonstrate the viability of the proposed solution. Employing Mininet to emulate SDN networks, utilizing the POX controller as the SDN controller and incorporating blockchain nodes with the help of Geth, validated the feasibility of attaining the proposed platform.

5. Experiments and Results

As presented in the above section, the proposed solution was validated in a series of steps. This section aims to offer a pragmatic perspective on the validation process of the proposed solution, presenting accompanying results.

It was mentioned that the last step of validation was using remote virtual machines that had available blockchain nodes, and the experiment was to confirm that they are able to connect and communicate. During this experiment, the traffic was monitored using the Wireshark tool in order to capture the communication between the nodes. The experimental duration spanned slightly under one hour, and during this period of time the ongoing traffic was captured. The nodes kept the connection and communication alive even if there was no specific information to be sent from one to another.

In the course of the experiment, a total of 1173 packets were exchanged, as illustrated in Figure 4. There were multiple conversations during the exchange, shown by the multiple TCP three-way handshakes present in the capture log. The connection was established, several packets were exchanged, and then the connection was closed. The TCP three-way handshake duration was between 4.193681 and 16.477670 ms (with the SYN -> SYN ACK duration between 0.02004 and 0.11177 milliseconds, and the SYN ACK -> ACK duration between 4.147911 and 16.45667 ms). A range of 18 to 21 TCP segments (with the connection management packets included) were transmitted between nodes until the connection was closed. The connection duration was between 17.372734 and 28.115327 ms. Among the TCP traffic, UDP packets were also captured. The UDP packets were usually exchanged after the TCP connection was closed; if this was not the case, they were always exchanged after the connection was set up. The UDP streams were of different lengths and were not necessarily between two TCP connections.

Long UDP streams were interchanged between TCP streams, as presented in Figure 5. Due to the considerable variability in datagrams within the UDP streams, it is unnecessary to report the number of packets and the duration of communication.

Figure 6 shows the distribution of the packets exchanged over the two protocols.

During the experiment involving the specified blockchain nodes, additional connections and interactions with other blockchain nodes were observed. The communication between the specified nodes did not exhibit any discernible impact from these concurrent connections. Thus, it is not possible to attribute the fluctuation in packet communication duration to the presence of other interconnected nodes.

Analyzing the capture, it can be observed that the background interaction between the nodes was consistent in regards to the timing. The interaction was always kept alive by establishing a TCP connection, and this happened at regular times, as can be observed in Figure 7. In between the TCP connections, the UDP datagrams were sent, facilitating UDP communication.

A communication delay was noted upon analyzing the recorded network traffic between the two hosts. There was a deviation in the duration of the three-way handshake time. It can be observed that, on one host, the duration between the SYN and SYN ACK packets was longer than on the other host (107.944382 milliseconds compared to 0.02314 milliseconds), and that, on the other host, the duration between the SYN ACK and ACK was longer than on the first host (105.105241 milliseconds compared to 0.061759 milliseconds). This difference can be attributed to the route the packet was sent on with the delay for SYN -> SYN ACK on one host causing a delay for SYN ACK -> ACK on the other host.

Analyzing the captured packets, a quantitative measure can be carried out regarding the size of the packets exchanged during communication. In the case of TCP packets, the mean size was 117 bytes per packet. In the case of UDP packets, the mean size was 573 bytes per packet. Figure 2 provides informative insights regarding how many packets were captured from different size intervals, how much they represent from the total number of packets and which is the mean value for the size of a packet from the mentioned categories. As can be seen in Figure 8, the packet sizes are within the middle sections of the distribution curve, indicating that the network is efficiently handling traffic.

In the experimental part, a total of 1173 packets were exchanged, with multiple TCP three-way handshakes indicating various conversations. The TCP three-way handshake duration ranged from 4.19 to 16.48 milliseconds, with 18 to 21 TCP segments transmitted between nodes until connection closure. UDP packets were also captured, usually exchanged after TCP connections were established or closed, with varying lengths and timing. Additional connections and interactions with other blockchain nodes were observed, but they did not significantly impact specified node communication. Background interactions between nodes remained consistent in timing, with regular TCP connections and intermittent UDP datagram exchanges. A communication delay was noted, attributed to route differences causing varying durations in the three-way handshake process between hosts. From the conducted experimental work, it can be noticed that both TCP and UDP communication between the Geth-enabled nodes is encrypted (Figure 9).

To conclude the information provided regarding the experimental work in this section, it can be noted that the communication among the nodes appears to be robust, without deviations or problems that would disrupt the information exchange, with default encryption handled by the Geth protocol which manages the underlying communication endpoints. Also, the packet sizes usually fall in the average section, which means that the data exchange is expected to be balanced.

Taking into consideration the points that were just mentioned, the integration of the blockchain node inside the SDN controller is anticipated to impose a manageable computational overhead on the hosting SDN controller.

6. Discussion

As the current work is focusing on the practical details concerning the implementation and development of a blockchain-enabled SDN in the context of ISP services validations, the current research is mainly focused on the implementation of a proof of concept for future research endeavors. Future development of this idea may involve exploring custom WAN topologies for controller placement, the utilization of SDN hypervisors to better accommodate dynamic network needs and the integration of already-existing widely used applications for network management.

In order to accommodate custom WAN topologies, ISPs would need to agree on specific decisions, and certain network developments and configurations might need to be created at the higher infrastructure interconnecting layer. Artificial Intelligence (AI) can be used to help in the deployment and coordination of multiple parties, especially with the federated learning paradigm gaining popularity in recent years. Organizing the ISP network as a decentralized federated learning federation and infusing the blockchain capabilities with AI features would drastically improve the entire system’s security and authenticity verifications. The classic federated learning paradigm uses a hierarchy, where an entity has higher privileges, but in the currently proposed ISPs’ scenario, a better approach is to organize the WAN SDN controllers in a flat architecture, where each of them has the same privilege.

Modern networking brings dynamic network function programmability to an existing network. An approach to further enhance network functionality is to enhance the NFV by using network hypervisors, more precisely deploying hypervisor solutions on generic hardware devices, and allowing the hypervisors to activate and deactivate different virtual machines that enable different services that are useful in the entire federation. The CoVisor hypervisor is presented in [27], and it allows multiple SDN controllers to be run concurrently on the same computer, sharing their best features to optimize network functions. In the proposed ISP use case, the main ISP controller may employ various solutions for the different services that it agrees to participate in, and in this case, CoVisor would be an ideal enabler tool. Ref. [28] presents the Libera network hypervisor, which can create a dedicated network infrastructure for handling specific network traffic and could be used in conjunction with CoVisor by an ISP to accommodate different network policies and changing internal networks. Such hypervisor solutions are deployed in the middle layers of the network, which in this case would be an advantage for dynamic real-time network functions programmability and configurations.

Major ISPs already use and reconfigure a large segment of their own networks using tools such as Network Cloud by DriveNets and Apstra by Juniper Networks, to name only a few. These applications, in summary, make the network behave like a cloud where various services can be enabled on various middle devices to avoid congestion and improve network functions. A new paradigm that is an evolution of modern networking is intent-based networking, which is integrated into the aforementioned technologies. These solutions are created to operate only at the internal ISP’s network; however, similar solutions can be developed to help organize the upper infrastructure layer between the ISPs, where, with the help of an AI-based federated learning approach, certain features could be requested (or favored) from certain ISPs.

7. Conclusions

This work focused on and validated the possibility of integrating and using blockchain nodes in SDN controllers. The connection and communication between the blockchain nodes outside of their organization’s networks were also addressed and validated. The means of using Mininet as an SDN platform and the provided POX controller together with the Ethereum blockchain implementation in Geth were presented. The potential for augmenting the POX SDN controller to facilitate the integration of a blockchain node and the execution of said node on the controller has been demonstrated. The paper also proposes an architecture model for integrating and using blockchain technology in the SDN networks, it presents the benefits and provides use cases and ideas that can be applied (independently or combined) in such a combination of integrated technologies.

The research focused on the collaboration between ISPs aimed at minimizing the impact of running and maintaining blockchain nodes over the resources of a single ISP. For this purpose, an original concept of organizing individual ISPs in a federation—virtual ISPs (vISPs)—was provided to allow the interconnection of heterogeneous ISPs.

Using the proposed approach, resources can be shared for the benefit of the entire federation, and regulations and standardization can be implemented in order to obtain an improved functionality and experience of using the integrated technologies. By integrating the aforementioned components, a framework in the form of proof of concept was created, to facilitate further developments and improvements in the area of SDNs and blockchain integration and for large-scale network infrastructures.

As future work, we plan to investigate how the blockchain nodes integrated into the federation’s SDN controllers can be properly isolated from other blockchain nodes that may be trying to connect and communicate over the federation’s blockchain network. In this way, the federation’s blockchain traffic will be fully isolated, and the nodes will exclusively be tasked with managing the internal intended traffic and data. Another key point for our future work will be the development of smart contracts to provide extended functionality. Deriving from this, we plan to research how smart contracts will affect the SDN controller and how we can correlate the smart contract complexity with increased demand in terms of resources regarding the controller. In this way, we would have a better perspective on how much the load on the SDN controller would increase by adding additional functionality, and we would be in a better position to investigate the right balance for a controller between handling blockchain tasks and the usual SDN tasks. Furthermore, the proposed solution will investigate how the proposed integration scales with increasing numbers of nodes and network traffic to ensure efficient communication and transaction processing in larger networks. To enhance overall network performance and efficiency, optimization techniques will be studied to reduce communication delays and packet exchange overhead. Furthermore, the experimental part will be extended to trials in real-world large-scale networking environments to validate the effectiveness and practicality of blockchain integration in SDN controllers. Although not an objective of our current paper, we also consider the need to evaluate the energy efficiency problem of blockchain-integrated SDN controllers and to provide energy-efficient mechanisms to minimize resource consumption while maintaining network performance at the ISP federation level.

As a further step in the integration of these technologies, AI holds immense potential for addressing various challenges. Using Predictive Network Analytics to analyze historical data and real-time network traffic patterns, SDNs could dynamically allocate bandwidth based on changing network conditions and QoS requirements. Furthermore, AI-powered agents can automate the negotiation and enforcement of SLAs between service providers and customers. The convergence of AI, SDN and blockchain could optimize network resources and adapt to changing traffic patterns in SDN environments or enable autonomous decision-making processes.