5G is expected to be the ubiquitous fabric to provide reliable, low-latency and high-speed connectivity for a wide variety of services (Internet of Things (IoT), mobile, Industry 4.0, automotive, etc.). However current network architectures are not yet able to provide these characteristics, mainly due to their monolithic nature which limits their flexibility, scalability, and programmability. To facilitate the adoption of 5G, the consensus in the industry is to utilise technologies and workflows from the field of Information Technology (IT), such as the cloudification and virtualisation of services [1
]. This way, network services that previously comprised dedicated and monolithic hardware appliances (e.g., Firewalls) will now be virtualised instances in commodity servers in the cloud. This shift is expected to offer a much higher degree of flexibility, scalability, and automation to service providers, as they will be able to modify the virtual setups dynamically and programmatically.
To fully reap the benefits provided by virtualisation and cloudification, the deployment and management of services must be decoupled from the underlying computing and networking infrastructure. A paradigm that can facilitate this requirement is the Platform as a Service (PaaS), as it provides a centralised and reliable environment, with high degrees of agility and automation, on which services can be deployed and managed. The benefits provided by the PaaS model can be further enhanced by incorporating the paradigms of Software Defined Networking (SDN) and Network Function Virtualisation (NFV). The scope of SDN would be to facilitate the control and programmability of the network infrastructure while NFV would allow for the efficient virtualisation and management of services on commodity servers.
In contrast with IT services, 5G services are more diverse and stringent in terms of their network and computing requirements. These requirements imply the need for a more flexible and customisable PaaS model [2
] than the currently available non 5G oriented solutions. One such model is the Next Generation Platform as a Service (NGPaaS) [3
], which is built around the principles of micro-services, modularity, and build-to-order. In short, NGPaaS allows for the deployment of custom-built platforms and services, on a diverse set of infrastructure technologies, using automated and technology agnostic workflows. Through the same workflow, any Vertical Service Provider (VSP) (e.g., a Telco provider or IoT provider) can deploy either a custom platform on the available infrastructure or services on top of an existing platform.
The content of this paper is structured as follows. At first, a study of related work is presented in Section 2
. Section 3
presents the implementation of the requirements and architecture of NGPaaS, as conceptualised in references [3
]—specifically, it provides the NGPaaS architectural design, the concept of Reusable Functional Blocks (RFBs) and introduces the RFB Description and Composition Languages Design, Deploy and Direct (RDCL 3D) tool [5
]. Later in Section 4
, the NGPaaS workflow is practically validated through a use case. This use case (Telco PaaS) is based on the Central Office Re-architected as a Datacentre (CORD) [6
] platform and provides Virtual Network Function (VNF) as a Service (VNFaaS) capabilities to a VSP. In addition to describing the use case and the components that comprise it, this paper also details the process of how these components were made NGPaaS-ready in Section 5
. Later, in Section 6
a brief description of another use case targeted by NGPaaS (5G Public Safety) is provided. Finally, Section 7
provides a critical conclusion of the presented work.
2. Related Work
As mentioned in the introduction, most telco-grade platform environments have originated from existing IT-based cloud technologies (e.g., OpenStack, Kubernetes), but have been extended with new features to be suitable for telco environments, such as edge, access, and core networks. The scope of this section is to contextualise telco-grade platform solutions with recent PaaS and NFV related work.
The most prominent standardisation effort in the NFV platform area is the ETSI NFV Management and Orchestration (MANO) specification [7
], which details an NFV-based platform architecture. There, a fixed set of PaaS related features is coupled tightly to the underlying infrastructure managers. However, unlike NGPaaS, ETSI NFV MANO lacks the option to customise which specific PaaS components are included in a deployment, consequently, the supported virtualisation technologies and infrastructure types cannot be modified easily or swapped. Instead, the MANO functionality focuses on the lifecycle management of specific services, controlled by pre-integrated infrastructure managers and PaaS functions. Different flavours of MANO platforms are available (e.g., OSM [8
] or 5GTANGO [9
]), and each one may have proprietary interfaces and service descriptor formats, which can act as a barrier to the integration of other PaaS solutions. The authors in reference [10
] and reference [11
] show that the characteristics might differ significantly when comparing existing MANO solutions. It may be necessary, depending on the targeted use-case, to augment an existing MANO platform and integrate new functionalities ([10
In addition to MANO platforms, there are many advances are being made in the NFV technology space. One such example is that in addition to common x86 server hardware, the use of FPGAs in data centre infrastructure is being presented as virtualised resources, as demonstrated in reference [13
]. In reference [14
], a management framework is presented for using FPGA resources in data centres, also enabling FPGAs to run Virtual Network Functions (VNFs). The authors of reference [15
] have implemented a framework to design network functions running on split CPU and FPGA architectures—this provides a method to offload computationally–intensive processing of the network functions to the FPGA.
The development of improved algorithms to assist in the deployment and operation of virtualised network functions is a highly active research area. A method for customised VNF placement is proposed in 5G networks [16
], showing that due to the diverse performance requirements among different 5G scenarios, an adaptive VNF placement approach is needed to accommodate service-specific requirements automatically. In addition to placement, scaling algorithms can also enhance the operation of VNFs. In reference [17
], a formal method is described to predict VNF traffic and optimally scale the VNF resources accordingly in an elastic way. Similarly, in reference [18
], a novel analytical model based on Stochastic Network Calculus is presented to investigate the end-to-end performance bound of chained VNFs quantitatively. By modelling (non-)bursty network traffic, the proposed analytical model provides an efficient method for service providers to determine which service chaining and placement configuration are more beneficial to meet the SLA requirements in terms of violation error and latency performance. Another operational enhancement is an optimised load balancing system for VNFs, presented in reference [19
] and the placement of multiple VNFs in geo-distributed infrastructures is investigated in reference [20
]. The latter is particularly applicable in a telecom scenario, where an optimal choice must be made in which central offices the VNFs should run.
In additional to VNF placement and scaling, automated healing actions also need to be implemented to quickly analyse and solve any issues with the running network service before performance is degraded. Autonomous healing actions in reference [21
] are based on pre-defined workflows developed by experts. The primary idea behind the adopted approach is to transfer the current expert domain knowledge and manual action triggers into a rule-based self-healing solution. Moreover, in reference [22
], it is shown that deep learning can be used to identify anomaly events from NFV system logs reliably. A technique using a supervised machine learning method for online classification of anomaly states based on similarities between anomaly type-specific density grid patterns is presented in reference [23
]. The detection is done by analysing CPU, memory and network usage; the procedure used, the density grid mapping, is inherited from the theory of grid-based clustering. Operational procedures such as healing combine techniques from various research areas and gather information from a wide range of data, implying the integration of a diverse set of tools and libraries. The above-described references show that the PaaS should cater for quick and easy upgradeability, in order to incorporate the rapid evolutions happening in the NFV area and to allow easy customisation for a Telco or other use-case. The platform should follow a modular approach, and a relevant PaaS architecture for this method is advised in reference [3
]. Later, in Section 4
, we exemplify this modular setup by showing how we implemented in NGPaaS the functional blocks of the orchestration mechanism, network control, monitoring framework and the analysis of VNF anomalies to trigger healing actions.
According to reference [24
], a unified PaaS interface can be derived from existing PaaS Application Programming Interfaces (APIs), or put another way, a common set of functionalities regarding service instantiation and configuration can be grouped under a standard API. This allows for a unified interface for application deployment and management, among different cloud platforms, avoiding technology lock-in effects. Unfortunately, considerable effort is needed to maintain and translate the unified API to the proprietary PaaS APIs; a common trade-off between the overhead and benefits of using an API abstraction layer. In contrast to reference [24
], NGPaaS adopts a workflow-based orchestration mechanism [3
], hence the API or descriptor format of the targeted PaaS or infrastructure manager is addressed natively in the workflow and thus minimises the effort required to integrate support for new APIs.
The NGPaaS architecture is innovative in the sense that both PaaS and service components are considered as the same module type, using the same generic orchestration mechanism based on the RFB model. As argued in reference [25
], if software suppliers adopt the generic RFB descriptor model, DevOps-based integration strategies will be greatly facilitated. The NGPaaS framework, therefore, facilitates customisation and the creation of a custom PaaS solution, with functionality targeted at specific use cases. Given the vast range of possible services, it can be challenging to integrate all existing options into a one-size-fits-all platform solution. NGPaaS implements the complete service lifecycle using build-ship-run actions: (1) What is needed in terms of service offering (build), (2) How the service can be deployed (ship) and (3) Where the service should be deployed (run). These fundamental design questions facilitate the mapping of the right PaaS technologies to the available IaaS and the requirements of the service that should be executed. Each PaaS can be enhanced with unique features related to network state, Quality of Service, high availability, auto-scaling or SDK toolsets [25
]. The deployed set of PaaS features is then customisable and use-case dependent. Before exemplifying this through the Telco PaaS use case, we first elaborate on the different processes and workflows which enable this modular NGPaaS framework.
4. Virtual Network Function as A Service, Overview
The scope of this section is to illustrate the operation of the NGPaaS workflow, via a demonstrable use case. It outlines a service scenario called “VNF-as-a-Service” (VNFaaS), which is based on the CORD platform. More specifically, this section highlights the composition of both the platform and services as RFB graphs and their subsequent deployment in their corresponding execution environment (IaaS for CORD, PaaS for the services).
4.1. The Telco PaaS Use Case: Virtual Network Function as A Service (VNFaaS)
In the Telco PaaS use case, the NGPaaS operator (e.g., a Tier 1 Telco operator) hosts VNFs on behalf of a Service Provider (the VSP). These VNFs could be virtual Routers and Firewalls, which the Service Provider can configure accordingly depending on the desired service (business VPNs, Internet access, etc.). In a “pre-NGPaaS” NFV architecture, the on-boarding of VNFs, to be made available in the service catalogue, is done by the Telco operator (based on the preferences of the Service Provider).
Using the NGPaaS workflow, the Telco operator instead utilises a PaaS-oriented approach instead. While the Telco provider still deploys the platforms, the Service Providers can onboard and administer specific VNFs on their own (e.g., vRouter, vFirewall) and also enable Value-added Service (VAS) capabilities. These VASs could be Telco-grade enhancements to the basic service (e.g., monitoring, healing, policy-based network control, etc.). This way, a pre-existing “VNF App Store” can be supplemented by allowing the Service Provider to onboard new VNFs via direct interactions with preferred vendors, resulting in a more diverse set of capabilities than could be obtainable with the pre-NGPaaS model. There many benefits associated with the adoption of the VNFaaS use case for both the VSP and its end customers. Most of these benefits are due to the nature of the VNFaaS use case, for example, the Service Provider does not need to ship hardware to the premises of its end customer, allowing for a more flexible and scalable business and service model.
illustrates the VNFaaS use case in more detail. There, a VSP has deployed two service graphs through the RDCL 3D tool, each comprising of two VNFs. Deploying these two graphs has resulted in the deployment of two service chains onto the CORD platform. Both service chains comprise an interconnected virtual router and firewall and are associated with a unique end user. The figure also illustrates the possibility to onboard and deploy VNFs of different vendors, thus providing flexibility to the end user.
4.2. The VNFaaS Proof of Concept
To validate the NGPaaS workflow through the VNFaaS use case, a Proof of Concept (PoC) scenario was designed and implemented. To be considered successful, this PoC must be able to showcase both the provisioning of a Telco-grade platform and the provisioning of services related to the VNFaaS use case.
illustrates an overview of the desired final status of the PoC. There, a Service Provider has been granted access to the RDCL 3D tool by the NGPaaS operator. Using this tool, the Service Provider is then able to compose and deploy service graphs of virtual routers and firewalls, which will provide the desired service to the provider’s end customers. Through the NGPaaS workflow, these service graphs are then translated to platform-specific APIs and passed down to the platform orchestrator. Since a service graph request will usually comprise both compute and network resources, the platform orchestrator will translate the request to API calls targeting both the underlying VIM and SDNC components of the platform. The VIM will then provision the required virtual computing resources and the SDNC will provide the required connectivity. It should also be possible for the service provider to monitor its deployment for specific KPIs. At the same time, an alerting function should continuously check the collected data and trigger automated healing workflows, when the state of a deployed service is undesirable.
The next couple of sections will provide more details on how this PoC was developed. More specifically, the technologies that comprise it will be detailed (e.g., Platform of choice), together with the steps required to integrate them into the NGPaaS workflow.
4.3. The Telco PaaS Platform: Central Office Re-Architected as A Datacentre (CORD)
The selected platform for the Telco PaaS use case is CORD and the current prototype uses version 6.0 [6
]. The primary reason for selecting CORD was that it was designed to provide edge network access, by acting as a virtualised Central Office. This implies that the core functionalities of the CORD platform could be effectively reused to fit the needs of the VNFaaS use case. For example, other platforms (e.g., OSM), have a more generic implementation which would make their integration into the VNFaaS use case a more complex task.
CORD’s architecture comprises three main functional elements, implemented as open source projects: (1) The XOS orchestrator, responsible for the joined control of ONOS and OpenStack. This joined control is facilitated by well-defined service models, which describe the supported services and service synchronisers. The synchroniser’s role is to align the operational state of CORD with the desired state, as defined by the administrator, in a programmatic way; (2) The ONOS SDN Controller (SDNC) is responsible for managing the physical and virtual networks; (3) The OpenStack platform is responsible for managing the lifecycle and resources of the deployed VNFs. In the used version of CORD, all the individual CORD components are deployed as sets of Docker containers, managed by Kubernetes and Helm. Figure 8
illustrates a high-level overview of the CORD architecture.
4.4. Supported Virtual Network Functions (VNFs) and Value Added Services (VAS)
4.4.1. Core Services: Virtual Firewalls and Virtual Routers
This section gives a brief overview of the VNFs and VASs supported by the Telco PaaS prototype. As part of the experimental set-up, we use a commercially available router and firewall VNFs. The Router VNF is an enterprise-class router designed to run on x86 standard servers but comprising all the same features as equivalent hardware routers [31
]. Similarly, the Firewall VNF is a virtual appliance aimed at enterprise-level security deployments [32
]. The implementation of both VNFs is based on virtual machines and in the presented prototype their running instances are hosted by OpenStack.
4.4.2. Value Added Services: Monitoring, Alerting, and Healing
In the Telco PaaS, monitoring utilises the ElasticSearch-Logstash-Kibana (ELK) stack [33
]. This stack consists of three main components: (1) Elastic Search: A distributed search and analytics engine based on REST APIs; (2) Logstash: A server-side data processing tool that can ingest data from various probes and manipulate it before sending it to ElasticSearch; (3) Kibana: A visualisation tool provided with ElasticSearch. As noted, in the Telco PaaS prototype the selected VNFs are based on proprietary VMs, thus deploying monitoring probes internal to the VNFs is not feasible. To overcome this limitation the probes in the Telco PaaS are deployed in a side-car fashion, externally to the target service; but can poll the target service for the required using the available interfaces. For this prototype, we rely on the SNMP protocol and REST interface to monitor the router VNF and ONOS SDNC respectively. More specifically, SNMP is used to poll the virtual router for its CPU utilisation, while REST is used to poll ONOS for network traffic information regarding the virtual interfaces of the deployed VNFs.
As an enhancement to baseline monitoring, this prototype also provides Alerting and Healing capabilities. The ELK stack can map trends in the CPU utilisation of monitored VNFs to predefined anomalous behaviours. Once such an anomaly is detected, a Healing function is called, which redeploys the anomalous VNF with more resources (e.g., more virtual CPUs or RAM). All healing actions are performed, by the Healing rule via the RDCL 3D tool, thus ensuring alignment between the state of the deployment and the view of the tool. Figure 9
shows the operation of the monitoring VAS in detail. As mentioned before, monitoring, alerting, and healing are complementary services provided selectively to VSPs. As a result, upon deployment of a VNF, the VSP can set a flag in the RDCL 3D metadata to trigger or not trigger the monitoring capabilities for this VNF.
illustrates the capability for live network traffic information for each interface of the provisioned VNFs. As mentioned, this information is collected by the monitoring probe through the ONOS REST API. On the other hand, Figure 11
shows how live information is reported about the CPU utilisation of all monitored VNFs. In addition, this segment is configured with three predefined thresholds (50%, 80%, and 90%)—when any of these thresholds is reached a corresponding alert is shown to the administrator. In the provided example, the critical threshold of 90% has been exceeded (94%); exceeding this threshold triggers an automated healing workflow that will re-provision the VNF with more virtual resources.
4.4.3. Value Added Services: Policy-Based Network Management
In addition to monitoring/alerting/healing, the Telco PaaS supports one more VAS, namely policy-based network management. This is achieved with the integration of an SDN-based network policy framework [34
] to the ONOS SDNC. By acting as a layer of abstraction between the network infrastructure and the VSP, the network policy framework allows for simplified control over the network infrastructure. Also, the network policy framework follows an entirely modular architecture, consisting of a Policy Manager and multiple separate Policy Type applications, thus allowing the VSP to add support for new policy types on demand, without affecting the runtime operation of the policy framework. Figure 12
illustrates a (simplified) example operation of the policy framework. There an administrator has deployed the policy manager with two policy type implementations (Firewall and NAT). Subsequently, the administrator issues a request to create a Firewall policy instance that will drop traffic between hosts H1 and H2. Upon receiving the request, the policy manager will validate it for correctness, and if successful, it will then request the Firewall Policy Type implementation to enforce the policy in the network. This will result in the Firewall Policy Type to install flow rules in the switches adjacent to H1 and H2, which will block traffic between the two endpoints.
6. 5G Public Safety Use-Case
As discussed, because the NGPaaS workflow is based on the concepts of build-ship-run and build-to-order, it allows for the deployment of a wide-range of platforms, services and technologies. So far, this paper focused on the Telco PaaS use case and the VNFaaS PoC. However, to demonstrate the flexibility of the NGPaaS workflow another use case has also been considered, namely the 5G Public Safety use case. This use case is briefly demonstrated herein, through the Mission Critical Push To Talk (MCPTT) PoC.
The idea of the 5G Public Safety use case is that an end-to-end cloud native mobile network supporting the MCPTT service is provisioned on-demand, through NGPaaS. Then it can be used by firemen during an intervention at an emergency incident (e.g., a building fire). Depending on the location of the Radio Access Network (RAN), this use case could involve different scenarios. In the prototype presented herein, the fire truck is assumed to have its own, mini, data centre with the capability to host VNFs. In this way the U-Plane as well as the Cloud RAN and MCPTT related VNFs can be hosted in the fire truck, while the MCPTT control plane VNFs can be hosted in a centralised data centre. In this PoC instead of CORD, two separate Kubernetes platforms are deployed: A Core Kubernetes PaaS and an Edge Kubernetes PaaS, tailored respectively for the Core and RAN service requirements. Both Kubernetes platforms are fully deployable and configurable through the NGPaaS workflow, the same workflow used for the Telco PaaS use case. Finally, the VNFs comprising the MCPTT PoC are deployed on their corresponding platforms, similarly to that of the VNFaaS PoC. Figure 16
illustrates the architecture and functional elements of this PoC. More details are available in the technical documentation of the NGPaaS [37
7. Discussion and Conclusions
This paper highlights some of the inherent complexities and challenges associated with service and platform deployment in cloud-based environments, as required for the successful adoption of 5G technologies. This complexity can be mainly attributed to the broad spectrum of available infrastructure and platform technologies, each of which brings their own interfaces and communication protocols. As a possible solution to this problem, this paper introduced a novel workflow for the composition and deployment of platforms and services in multi-cloud environments. The proposed workflow follows the NGPaaS concepts of build-to-order and Build-Ship-Run, by utilising the RFB model and the RDCL 3D tool. Build-to-order implies the possibility to define custom platforms and services agnostically to the underlying technologies, based on customer demand. Build-Ship-Run implies the possibility to deploy this platform or service compositions on a wide range of execution environments, through a centralised environment (RDCL 3D tool). By following the NGPaaS workflow, platform and service deployment is greatly simplified and facilitated, since most of the underlying technologies are abstracted through generic blueprints and APIs. Finally, this paper successfully validated the proposed workflow by presenting a proof of concept scenario (Telco PaaS) that includes the composition and deployment of both a platform and a set of diverse services. The Telco PaaS use case is based on the CORD platform and represents a VNFaaS scenario, in which a VSP offers VNF services to its end customers, together with many value-added services (monitoring, healing, policy-based network management). Finally, to showcase the flexibility of the NGPaaS workflow, a second use case is also presented herein.