C-RAN Traffic Aggregation on Latency-Controlled Ethernet Links

Featured Application

The mechanism described in this paper is thought to be implemented as enhancement of Ethernet switches and widely deployed in C-RAN for efficient and scalable transport of fronthaul and backhaul traffic in view of flexible 5G networking.

Abstract

Centralized/Cloud Radio Access Networks (C-RAN) are deployed in converged fixed-mobile networks to exploit the flexibility coming from joint application of Network Function Virtualization (NFV) and Software Defined Networking (SDN). In this context, optical links connecting C-RAN nodes, possibly based on the Ethernet standards, may carry traffic with different requirements in terms of latency and throughput. This paper considers the problem of traffic aggregation on C-RAN optical Ethernet links with latency control for fronthaul traffic and throughput capability for backhaul traffic. Integrated hybrid network technique is applied to show how time transparency can be enforced for Ethernet encapsulated Common Public Radio Interface (CPRI) traffic while allowing statistical multiplexing of backhaul traffic. Simulation results show the effectiveness of segmentation of backhaul traffic to allow exploitation of the available bandwidth even with high capacity CPRI options.

1. Introduction

Centralized/Cloud Radio Access Network (C-RAN) is a promising technique enhancing mobile network performance and flexibility [1]. Improving survivability and power saving are the two main key designs in this architecture [2]. C-RAN is based on the splitting of functionalities which are possibly remotized for the purpose of centralization of the higher layer baseband processing related to multiple Remote Radio Heads (RRHs), by the introduction of the fronthaul network segment [3]. In addition, this approach takes significant advantage of Network Function Virtualization (NFV) and Software Defined Networking (SDN) control and management to add flexibility and adaptability to 5G transport networks [4] and to allow migration of functionalities over the nodes of C-RAN for optimization purposes.

In recent proposals [1], the highly demanding traffic, generated by the above functional splits, is carried together with packet-based backhaul (BH) traffic over a common optical infrastructure on different wavelength channels of the same fiber links [5]. This solution can be designed to meet the latency requirements of fronthaul (FH) traffic but may limit the system scalability and lead to low resource utilization with consequently high deployment costs. Hence, techniques for improving the optical channel utilization while meeting the strict performance requirements of FH traffic are needed and investigated in this paper. Ethernet-based links can be adopted to implement FH and standardization bodies are recently very active on the definition of the requirements to support FH traffic on this widely deployed interfaces. Delay requirements are identified as quite challenging, the Ethernet not being originally designed for delay-sensitive applications, such as fronthauling [6].

In this work, an integrated hybrid architecture, previously presented in [7], is reviewed and extended with pre-emption to be applied to the C-RAN optical transport network. An integrated Ethernet-based interface, where FH and BH traffic are multiplexed to achieve high utilization of the wavelength resources, is proposed and evaluated to meet the delay requirement of FH traffic, while offering throughput capability to BH traffic. A converged fronthaul/backhaul scenario, as depicted in Figure 1, is considered for evaluating the additional BH throughput that can be obtained as a consequence of the application of the mechanism to wavelength channels. In Figure 1, a sample C-RAN topology is shown where Integrated Hybrid Nodes (IHNs) are interconnected by optical links, possibly forming a mesh or other topologies. IHNs are assumed to be equipped with Ethernet interfaces, which ensure high backward compatibility and low cost. RRH are connected to IHNs which host baseband unit (BBU) functionalities. The set of BBUs available in an IHN form the BBU Hotel. These functionalities can be virtualized in a C-RAN and moved throughout the network to optimize the access service. The management of migration of virtual BBU functionalities is performed by the SDN control/management plane, not shown in the figure. As a consequence of different possible locations for BBU Hotels in IHNs, FH and BH traffic can be present on each link of the C-RAN. FH traffic is assumed according to the Common Public Radio Interface (CPRI) standard as encapsulated in Ethernet frames. The traffic characteristics of the CPRI traffic as generated by the different standardization options are taken into account and managed by a suitably extended integrated hybrid mechanism. As a consequence of the deterministic behavior of the encapsulated CPRI traffic, the benefits of introducing a segmentation policy on the BH traffic is also considered. Segmentation is the act of carving up BH traffic into smaller pieces so they can be inserted into the possible FH traffic gaps.

Figure 1. Converged fronthaul/backhaul scenario.

The paper is organized as follows: in Section 2, research in this context is described. In Section 3.1, the application of the integrated hybrid concept in C-RAN is described. Section 3.2 describes how to insert BH traffic in the FH stream while preserving time transparency. In Section 4, evaluation of the proposed architecture is presented. Section 4.1 highlights the achievements while conclusions and open aspects are drawn in Section 5.

2. Related Works

Any transport network has among its main targets to serve different kinds of applications while being able to handle Quality of Service (QoS) requirements. These aspects are widely investigated in literature to design networks with enhanced flexibility and scalability, which have turned out recently to be the key enablers for cost-efficient 5G deployments [8,9]. The C-RAN concept jointly takes advantage of NFV/SDN [10] and different splitting options for 5G network design optimization [11]. As a consequence, the traffic on transport network links can have different QoS requirements, it being related to different sections of splitting [12]. Ethernet based networks play an important role in a wide variety of fields. For example, industrial communications are one of the technologies which benefit a lot from the Ethernet concept. As explained in detail in [13], the core of such systems is essential, being reliable for the exchange of information. The latest trend in technologies which influences industrial technologies is IoT (Internet of Things), which from the communication point of view relies largely on Internet-based communications. Since the real-time capabilities are one of the major characteristics of such systems, the IEEE TSN (Time Sensitive Networking) working group currently aims at improving the real-time capabilities of the standard of Ethernet. They are mainly targeting issues like reducing latency, independence from physical transmission rates, fault tolerance without additional hardware and support for higher security and safety. From another point of view [14], IoT can transform Electric Power and Energy Systems (EPESs) by providing a solution in order to meet the growing demands of access to affordable, clean and sustainable energy. The adaptation of IoT in EPES is also favored by the significant reduction of costs associated with sensors, bandwidth processing, and memory storage. Realizing the full potential of IoT is critical to enhancing the flexibility, asset management, operations and reliability of intelligent electric power networks. To ensure continued growth of IoT for EPESs, it is essential to develop viable solutions to handle its growing complexity.

The work referenced in [15] is focused on the evaluation of the best configuration for C-RAN in terms of cost of network resources such as wavelength. The authors present an implementable solution that can be useful in 5G studies. They specifically introduce an analytical model which proves that using multiple rings can eventually be more cost-effective in comparison with a single ring in terms of wavelengths. This is due to the fact that, although a single ring can provide the lowest cost solution because of high statistical multiplexing, it also needs a switching apparatus with a high number of ports. They proposed a transport solution for the fronthaul segment based on optical technology that allows for dynamic bandwidth allocation needed to carry Ethernet and CPRI flows with the consequent saving of bandwidth. They successfully show the 90% savings on wavelengths.

Ethernet interfaces are of increasing interest to implement packet-based multiplexing of FH and BH traffic for the wide adoption of this standard in transport networks [16]. Ethernet as a packet based technology suffers from statistical multiplexing delay, which is typically not bounded and variable. This is a key issue to be solved to design integrated interfaces for different levels of service [17].

The packet-based integrated hybrid optical network with ultra-low Packet Delay Variation (PDV) services and statistically multiplexed traffic was firstly proposed in [7]. A hybrid packet/circuit node was defined in order to add the ability to the network to manage priority and non-priority traffic. As introduced in the paper, the traffic in the network can be divided into two categories. The traffic with fixed delay and zero packet loss will be routed through the optical network. These types of traffic are given absolute priority and, since it is QoS sensitive, their delivery must be guaranteed. The other traffic category is less delay sensitive and is in the packet format that will be transported through packet switches. This separation of different priorities of traffic in the network is accomplished by employing Statistical Multiplexing (SM). This type of hybrid network is demonstrated to offer improved performance at a lower cost, compared to a pure circuit or pure packet switched networks.

Integrated Hybrid Optical Networks (IHONs) is a further evolution of the primary concept of hybrid node which was introduced in [18]. This new node architecture merges the circuit and packet network in the same wavelength to enable the circuit quality transport of demanding services with higher granularity than wavelengths and enabling statistical multiplexing for the throughput efficiency of packet networks. They used the same principle of dividing the traffic between priority and non-priority as mentioned before and demonstrate experimentally that priority traffic will have absolute transfer guarantees with no packet loss, low packet delay, and ultra-low PDV. Any capacity not utilized by the priority traffic is identified as idle time gaps which can be used by lower priority traffic. Implementation based on Ethernet has been demonstrated in C-RAN [19], assuming CPRI traffic transparently transported by a wavelength and using the leftover capacity for BH traffic. However, to prevent the FH traffic from PDV caused by BH traffic insertion, IHON adds a fixed delay, corresponding to the maximum duration of a BH frame, to all FH packets.

An extension of the IHON concept was proposed and evaluated in [20] for possible application in data center by considering three different QoS levels and the possibility of pre-emption for priority traffic. The priority scheduling of three different service profiles was investigated to maximize data center traffic throughput while guaranteeing time transparency for delay-sensitive services and zero loss/fixed delay for guaranteed connections.

The idea to use the integrated hybrid concept in C-RAN requires consideration of C-RAN traffic characteristics. In C-RAN, the interconnection between RRH and BBU Hotel is based on CPRI [21].

CPRI is nowadays the most common protocol to transport data in the FH segment of C-RAN between radio sites and baseband hotels/mobile edge computing. Being a Time Division Multiplexing (TDM)-based protocol, it poses strict capacity, latency and latency variation requirements on the transport network. Recently, packet-based functional splitting protocols, such as eCPRI [21], have been released, while splits proposed by 3GPP [22] are on their way to being standardized. However, so far, the majority of the proposed splits concern the radio physical layer and do not relax the strict delay requirements on the FH transport.

The utilization of Ethernet in this architecture is considered as a way for improving reconfigurability and efficiency in terms of both capital and operational expenditures. Since C-RAN is mainly dimensioned based on the peak traffic, it requires large capital expenditures. Moreover, implementing custom-made CPRI switches which enable reconfigurability to C-RAN adds extra costs to the network. In the work referenced in [23], authors focus on an FH solution encapsulating CPRI frames into Ethernet frames and implementing dynamic CPRI line bit rate reconfiguration. In their study, CPRI over Ethernet encapsulation/decapsulation is assumed to be performed at both the RRH and BBU Hotel. The encapsulation is performed as a size-based burst assembly and the minimum granularity of CPRI data to be encapsulated into an Ethernet frame is the CPRI basic frame. Performance evaluation results show that dynamic CPRI link bit rate reconfiguration is achieved within about one millisecond after rate reconfiguration triggering. However, if size-based encapsulation is utilized, the time to perform encapsulation varies as a function of the CPRI link bit rate, thus causing encapsulation delay and jitter. In [24], the authors provide an analysis of the impact of different parameters (e.g., BH packet size) on latency and latency variation of a converged FH and BH Ethernet-based network. This analysis is conducted on a real field trial and a discussion of the challenges when dealing with design of ultra low latency traffic aggregators is also reported. The capability of the integrated hybrid approach to support the high data rate required by CPRI have been demonstrated in recent experiments [25]. Mapping and decision-making on the placement of Ethernet encapsulated CPRI traffic in integrated hybrid networks is the step beyond the state of the art that this paper addresses. Some evaluations of the additional BH throughput achieved by the integrated hybrid approach aggregating FH and BH according to extension of the IHON principle were presented in [26]. Limitations of throughput for high capacity CPRI option were outlined. In this paper, the term aggregation is used to express the combination of BH and FH traffic into the same optical link. This paper intends to introduce the new multiplexing concept taking place on the Ethernet interface and discusses the benefits that can be achieved in the presence of synchronous traffic that needs to be transferred in a time transparent way. Further extension of the mechanism based on BH traffic segmentation is here proposed to achieve better throughput for BH traffic.

3. Traffic Aggregation

The exponential increase in mobile users and enormous bandwidth requirement by mobile applications emphasize the need for introducing a solution to increase the throughput. Traffic aggregation is one of the well-established concepts in the networking context and, in this section, the aggregation technique and how C-RAN can benefit from it will be explained in detail.

3.1. Integrated Hybrid Optical Network in C-RAN

Figure 1 shows a sample network topology implementing the C-RAN. The BBU serving a RRH can be activated in different hotels for resource optimization, service continuity or energy efficiency, thus possibly requiring dynamic association between RRHs and BBUs. This dynamic association is thought to be managed by a suitable SDN control/management plane. Moreover, traditional base stations (e.g., LTE eNB) may also be present in the same area, requiring connection to the core network. As a consequence, both FH and BH traffic need to be transferred on each optical network segment. A solution to deploy such a scenario can be to assign dedicated wavelength channels to each kind of traffic, either FH or BH, so that FH links can be designed to meet strict delay requirements and BH traffic is statistically multiplexed on separated channels. FH delay requirement comes from the 3 milliseconds budget for round trip time in Hybrid Automatic Repeat Request (HARQ). This budget includes the RRH and BBU processing, which leaves only 100–200 microseconds for fronthaul one-way propagation delay. In addition, CPRI has one-way jitter requirement of few nanoseconds.

The integrated hybrid multiplexing scheme IHON was firstly proposed to implement statistical multiplexing of the GST (Guaranteed Service Traffic) and SM (Statistical Multiplexed) in Ethernet packet-based nodes [7]. In IHON, a small fixed delay ($\Delta$) is added to guaranteed traffic (GST) so that statistically multiplexed (SM) traffic can be inserted in GST gaps, with minimum delay and zero PDV (Figure 2), as it was experimentally proved in [18]. With the aim to minimize the delay of GST traffic, IHON can be extended to allow GST traffic pre-emption on SM traffic and the effectiveness of this mechanism was analyzed in [20].

Figure 2. Integrated Hybrid Node (IHN) multiplexing scheme.

Here, the integrated hybrid concept with pre-emption is applied to a network segment of a C-RAN where FH traffic, i.e., CPRI flow encapsulated in Ethernet frames [27], is identified as GST, with zero PDV, while BH traffic deals with pre-emption as SM traffic. During the transmission of a FH frame, incoming BH packets are stored in a buffer until an output channel is free. A scheduler (represented by the block S in Figure 2) senses the input channels to detect FH frames and is in charge of deciding when to start and interrupt the transmission of BH packets on the output channels. IHON eliminates PDV of the FH traffic because the fixed delay $\Delta$ enables a time-window which gives sufficient time for processing and decision of BH packet preemption. This goes beyond, e.g., the IEEE 802.1Qbu pre-emption [28] recommended in the IEEE 802.1CM standard [29] for FH, where FH packets may experience PDV corresponding to the service time of 155 Bytes anyway.

3.2. Mapping of CPRI Traffic in IHON

CPRIoE (Common Public Radio Interface over Ethernet) traffic characterization has been analyzed in [27,30]. A list of parameters used in this study is reported in Table 1, while an example of the IHN output line is provided in Figure 3. RRHs generate CPRI flows at different rates ($R_{CPRI}$) set by the standard [21]. Each flow is composed of CPRI basic frames with fixed duration $T_{CPRI}=260$ ns, equal for all CPRI options. A certain number of CPRI basic frames ($N_F$) are encapsulated in an Ethernet frame forming the CPRIoE payload of length:

$$L_F=N_F*R_{CPRI}*T_{CPRI}.$$

(1)

Table 1. List of parameters used to describe CPRI over Ethernet (CPRIoE) and hybrid nodes.

Parameter	Description
$N_F$	Number of CPRI basic frames forming a CPRIoE payload.
$L_F$	Payload length for CPRIoE frame.
$R_W$	Output channel rate.
$T_G$	CPRIoE duration.
$T_{GAP}$	Gap duration.
$\Delta$	Fixed delay to avoid collision.
${\rho }_B$	Offered backhaul (BH) load per channel.
$L_B$	Average length of BH frames.
$T_{guard}$	Guard time.
$T_{CPRI}$	CPRI basic frame duration.
$R_{CPRI}$	CPRI flow generation rate.
$L_H$	Length of CPRIoE header.
m	Number of channels in the switch output interface.

Figure 3. CPRI over Ethernet (CPRIoE) parameters applied to hybrid node.

CPRIoE frames are then sent by RRHs towards IHON switches, where they are delayed by $\Delta$. In addition, conventional BH traffic reaches the switches, loading the output channels with parameter ${\rho }_B$. In order to avoid collision between different frames on the output line, a guard time $T_{guard}$ is applied during which the transmission of any data is not permitted. IHON switches have m output channels, each characterized by a rate $R_W$, and accommodates CPRIoE frames of duration:

$$T_G=\frac{L_H+L_F}{R_W},$$

(2)

where $L_H$ is the header of CPRIoE frames assumed to be 44 Bytes [27].

Depending on $N_F$ and $R_{CPRI}$, the gap duration $T_{GAP}$ is selected according to:

$$T_{GAP}=\frac{L_F}{R_{CPRI}}-T_G.$$

(3)

By looking at Formula (3), it is possible to notice that, depending on $R_{CPRI}$, different values for $T_{GAP}$ can be obtained for the same length of CPRIoE packets $L_F$. An example of $T_{GAP}$ using CPRI opt. 1 and 6 for a line rate of $R_W=10$ Gbps is depicted in Figure 4.

Figure 4. $T_{GAP}$ as a function of different values of payload length $L_F$ for CPRI opt. 1 and 6 on a 10 Gbps line.

This time gap is used in the hybrid multiplexing scheme in order to aggregate BH traffic on the same transport channel. To this end, two different policies are considered here:

• A BH packet is transmitted when a gap is available and it is possibly pre-empted upon arrival of a new GST burst, in case BH packet duration is longer than the gap itself. In case of pre-emption, the BH packet is lost. This policy is indicated as P policy, with insertion of an entire packet into the by-pass GST flow.
• A BH packet waits for a gap and, in case the BH packet is longer than the gap, it splits into segments that are transmitted by as many gaps as needed. This avoids the need of pre-emption but introduces some overhead to manage segmentation and additional functionalities. This policy is indicated as S policy, where packets are divided into $N_S$ segments of suitable size for their insertion into the GST flow.

4. Numerical Results

To evaluate the benefits introduced by the proposed mechanism, an event-driven simulator in C++ language has been developed. One RRH generates a CPRI flow according to two different options with rates $R_{CPRI}=614.4$ Mbps (option 1) and 6.144 Gbps (option 6). The IHON fixed delay $\Delta$ = 99.2 ns is assumed, which corresponds to the smallest segment (124 Byte) that can be preempted [29]. A time guard of 10 ns between frames is applied. A single output channel ($m=1$) with rate $R_W$ = 10 Gbps is considered. The number of CPRI basic frames in a guaranteed burst $N_F$ is varied over the intervals $[1,70]$ and $[1,7]$, for CPRI option 1 and 6, respectively [27], so that the payload length $L_F$ varies accordingly. A set of simulations varying the average BH packet length $L_B$ is obtained with a load ${\rho }_B$ such that a BH packet is always ready for transmission on the output channel. The length of BH frames is considered to be exponentially distributed with parameter $L_B$.

Figure 5 shows the success probability of the BH traffic, defined as the ratio between the packets not interrupted and the total packets in service, as a function of $L_F$, for both CPRI options, varying $L_B$. In both cases, the success probability increases with $L_F$, due to the resulting larger $T_{gap}$. Option 6 shows lower performance than option 1 due to the smaller size of the gap, especially when $L_F$ is low, thus suggesting to use larger $N_F$ in this case. However, increasing $N_F$ increases the encapsulation delay, which may impact the maximum reach of the FH connection.

Figure 5. Backhaul (BH) success probability as a function of payload length $L_F$ for different BH packet length $L_B$ using CPRI opt. 1 and 6 without segmentation.

Figure 6 reports the BH throughput, normalized to the output line rate (10 Gbps), as a function of $L_F$ for option 1 varying $L_B$. The figure also reports the maximum normalized capacity left by FH traffic. The value of throughput in the case of the P policy reaches 8.9 Gbps only for high values of $L_F$ with quite a limited influence of $L_B$. The S policy, instead, is able to better exploit the available capacity for any value of $L_F$, except for the influence of the transmission guard times inserted. The same evaluation obtained for option 6 in Figure 7 shows a remarkable effect of the shorter gaps in the FH flow, which also prevents the F policy to fully exploit the available capacity for low values of $L_F$, due to the high numbers of segments needed and related inserted transmission guard time.

Figure 6. Backhaul (BH) throughput, normalized to the output link capacity, as a function of payload length $L_F$ for different BH packet length $L_B$ using CPRI opt. 1. Solid lines for the case with segmentation (S), dashed lines for the no-segmentation case (P). Maximum normalized capacity by fronthaul (FH) is shown by a solid line.

Figure 7. Backhaul (BH) throughput, normalized to the output link capacity, as a function of payload length $L_F$ for different BH packet length $L_B$ using CPRI opt. 6. Solid lines for the case with segmentation (S), dashed lines for the no-segmentation case (P). Maximum normalized capacity by fronthaul (FH) is shown by a solid line.

Figure 8 reports the overhead introduced by the P and S policies calculated as the ratio of the number of bytes for Ethernet headers and the the total number of bytes transmitted as BH traffic for option 1. The same evaluation is presented in Figure 9 for option 6. The effect of the S policy is more evident with option 6 where, due to the smaller gaps in the FH flow, multiple segments are typically required to transmit each BH packet. In any case, the additional overhead is quite limited when increasing $L_F$. It is interesting to analyze the average number of segments to transmit a BH packet in option 1 and option 6, as shown in Figure 10 for the S policy. Option 1 allows transmission of a packet as a single segment in most cases for any $L_F$. In option 6, instead, reasonable values of $L_F$ seem to be not less than 1000 bytes which give an average number of segments less than 3 for any $L_B$, with a resulting overhead around 10%, which is reasonable as well. However, working with high $L_F$ increases the encapsulation delay, which in the worst case is 18.3 $\mathsf{\mu }$s for CPRI option 1 and 1.83 $\mathsf{\mu }$s for CPRI option 6.

Figure 8. Overhead for backhaul (BH) packets as a function of payload length $L_F$ for different BH packet length $L_B$ using CPRI opt. 1. Solid lines for the case with segmentation (S), dashed lines for the no-segmentation case (P).

Figure 9. Overhead for backhaul (BH) packets as a function of payload length $L_F$ for different BH packet length $L_B$ using CPRI opt. 6. Solid lines for the case with segmentation (S), dashed lines for the no-segmentation case (P).

Figure 10. Average number of segments ($N_S$) required to send a backhaul (BH) packet as a function of payload length $L_F$ for different BH packet length $L_B$ using CPRI opt. 1 and 6.

4.1. Evaluation Achievements

The results obtained have shown that the IHON mechanism can support FH and BH traffic multiplexing on the same wavelength channel while guaranteeing time transaparency for FH. BH packet transmission probability increases with the payload length $L_F$ of the CPRI frame but results are quite limited with option 6 if no segmentation is applied. Further comparisons show, instead, that the throughput, normalized to line rate, can reach almost the full residual capacity utilization with segmentation, at the cost of some additional overhead, especially for option 6, which requires more segments to fill the smaller gaps.

5. Conclusions

In this paper, the feasibility of the integrated hybrid network concept with pre-emption has been applied to C-RAN. FH traffic with time transparency, generated according to a CPRI standard, has been aggregated with pre-emptable BH traffic within the same optical Ethernet channel. Performance evaluations have been presented for different CPRI options, related to the amount of BH traffic which takes advantage of unused FH capacity. Scheduling BH packets transmission is first performed when gaps in the FH traffic are of suitable size as provided by the IHON mechanism. With this assumption, remarkable BH throughput is shown especially for CPRI option 1. CPRI option 6, instead, limits BH throughput because of the smaller packet gaps in the FH traffic that makes the fitting of the BH traffic sometimes even not possible. For this reason, segmentation of backhaul packets has been introduced and analyzed, showing a remarkable improvement in BH throughput performance. The mechanism has been shown to enhance the basic performance of an Ethernet switch by guaranteed time-transparent aggregation of fronthaul traffic with efficient transfer of backhaul traffic. The introduction of such equipment in the 5G transport network will allow compatibility with the widely deployed Ethernet standard while enhancing significantly the scalability and the flexibility of the C-RAN optical infrastructure. This research can be further developed to include the introduction of delay bounded mechanisms to better adapt BH traffic to the unused bandwidth in the FH flow.