Experimental Investigation of 400 Gb/s Data Center Interconnect Using Unamplified High-Baud-Rate and High-Order QAM Single-Carrier Signal

Abstract

In this article, we review the latest progress on data center interconnect (DCI). We then discuss different perspectives on the 400G pluggable module, including form factor, architecture, digital signal processing (DSP), and module power consumption, following 400G pluggable optics in DCI applications. Next, we experimentally investigate the capacity-reach matrix for high-baud-rate and high-order quadrature amplitude modulation (QAM) single-carrier signals in the unamplified single-mode optical fiber (SMF) link. We show that the 64 GBd 16-QAM, and 64-QAM signals can potentially enable 400 Gb/s and 600 Gb/s DCI application for 40 km and beyond of unamplified fiber link.

1. Introduction

To meet the ever-growing demands on the internet bandwidth, the Ethernet speed has evolved at an astonishing pace, from 10 Megabit Ethernet to 100 Gigabit Ethernet (100 GbE) within the past 30 years [1]. In 2017, 400GbE standard was ratified by the IEEE P802.3bs Task Force [2]. This built the foundation for industrial deployment of 400GbE in the global network. It is anticipated that 400GbE will be rapidly adopted over the next few years [3]. Recently, 400GbE pluggable optics is under development to satisfy the demand on bandwidth-consuming applications, including data-center-based cloud services [4,5,6].

For intra data center interconnect (DCI) application, optical direct detection with four-level pulse amplitude modulation (PAM-4) has been widely adopted to enable the low cost 400GbE optical transceivers, which can carry up to 400 Gb/s Ethernet data over either parallel fibers or multiple wavelengths. Direct detection PAM-4 optics has also been standardized by Multi-Source Agreement (MSA) groups to achieve link distance from 500 m to 10 km (e.g., 400G-FR8, 400G-DR4, 400G-LR4, etc.) [7]. For inter DCI application, pluggable 400G digital coherent optics (DCO) with single-carrier dual-polarization quadrature amplitude modulation (DP-QAM) signal is being proposed to achieve 40 km and beyond fiber link [8,9]. PAM-4 signal with direct detection only uses the intensity of the light to carry information. Instead, DP-QAM signal with coherent detection technology can utilize a few degrees of freedom for photons, including polarization, intensity, and phase. Consequently, coherent technology can transmit a single-carrier 400 Gb/s signal with significantly improved spectral efficiency. Furthermore, its receiver sensitivity can be significantly improved over the one of intensity modulation with direct detection (IM/DD) systems.

In this article, we first introduce the application scenarios of pluggable optics in DCI applications. Next, we discuss different perspectives on the 400G pluggable module, including form factor, architecture, digital signal processing (DSP), and module power consumption. Then, we experimentally investigate the fiber transmission performance of 16-QAM and 64-QAM single-carrier signals at various baud rates, aiming at 400 Gb/s DCI applications with unamplified link. Here, we consider 25% overhead (OH) soft-decision forward error correction (SD-FEC), which gives a 4.2 × 10⁻² bit error ratio (BER) threshold [10]. For 64 GBd 16-QAM signals with 12 dB and 18 dB link margins, it is possible to transmit 400 Gb/s net data rate over 40 km and 80 km unamplified fiber link, respectively. With 18 dB link margin, 64 GBd 64-QAM signal can produce greater than 600 Gb/s net data rate for an unamplified link as long as 40 km. We further study the impact of transmitter (Tx) side power on the transmission performance and obtain a high-baud-rate QAM signal transmission matrix in unamplified link. For various signals, extra 6-dB more power can extend the fiber transmission reach to approximately 30 km. It is thus critically important to increase the Tx output power or improve the Rx sensitivity, under the available power and space constrains of the DCO pluggable module.

2. 400G Data Center Interconnect Use Cases

Data centers have been proliferated over the past few years, which are driven by more and more bandwidth-hungry applications, including enterprise digital transformation, cloud services, and the coming 5G applications. In such an era, enterprises, service providers, and cloud providers need a DCI network that can quickly adapt to the changes in business and operational demands. To meet the above requirements, data center networks are expanding in number and capacity, driving the need for higher-speed DCI solutions. Top cloud service providers, such as Amazon, Facebook, Google, and Microsoft, have already adopted 100 Gb/s and will quickly transfer to 400 Gb/s and beyond.

Figure 1 illustrates typical application scenarios of 400 Gb/s pluggable optics for DCI. As shown in Figure 1a, the reach of an unamplified link is power limited, and it mainly depends on the optical transmitter output power, the fiber link loss, and the optical receiver sensitivity. For the amplified dense wavelength division multiplexing (DWDM) link displayed in Figure 1b, it is noise limited, and its reach depends on the available optical signal to noise ratio (OSNR) at the optical receiver.

3. 400G Pluggable Module Form Factors and Power Requirement

Due to the emerging bandwidth-hungry applications, the photonics communications industry is unceasingly driven towards higher-capacity, smaller-size, and lower-power solutions. Standardization organizations, including the Institute of Electrical and Electronics Engineers (IEEE) and the Optical Internetworking Forum (OIF), are leading the industrial transformation of photonics communications society from using discrete optical components to utilizing photonics integration enabled packet optics [11,12]. Lately, line and client-side industrial trials on interoperability have been demonstrated successfully using 100 Gb/s CFP2 and 400 Gb/s CFP8 pluggable optical modules, respectively [13,14,15,16,17]. Nowadays, the industry is marching towards 400 Gb/s pluggable optical modules with even more compact form factors. The first debuts of 400 Gb/s client and line-side pluggable QSFP modules are planned for year 2019 and 2020.

Based on the DSP’s physical location, there are two types of pluggable coherent modules: analog coherent optics (ACO) and DCO. For 400G pluggable coherent transceivers, the industry has reached an agreement on implementing DCO, which has the DSP inside the module. A few form factors have emerged recently as the candidates for 400G DCO pluggable modules, such as CFP2, CFP8, OSFP, and QSFP-DD [18,19,20,21,22], whose mechanical dimensions and power consumptions are shown in Figure 2. These transceivers are equipped with eight pairs of differential signal pins, which can support 400G transmission speed using 50G PAM-4 electrical interface [23]. One can see that CFP2 and CFP8 have larger physical dimensions than those of OSFP and QSFP-DD, thus, they have more room for additional components and allow a larger power budget. Consequently, OSFP and QSFP-DD are targeted for the DCI application, while CFP2 and CFP8 are planned for the long-haul applications. The line-card fitting into one rack unit (RU) space can accommodate 36 QSFP-DD pluggable modules. The total capacity of the line-card is 14.4 Tb/s. Currently, the industry is converging behind QSFP-DD due to its high port density in the front panel, its backward compatibility with QSFP28/QSFP+, and thus a larger ecosystem.

Figure 3 illustrates the typical architecture of a 400G pluggable coherent transceiver. There are three functional blocks inside the module, one DSP chip, one transmitter and receiver optical subassembly (TROSA), and one power/control unit. Besides the power and control signal, the host side also provides up to 400GbE client traffic to the pluggable coherent module through eight pairs of differential signal pins. 400 Gb/s single-carrier signal (>60 GBd 16-QAM) can thus be generated out of the Tx output. The other port of receiver (Rx) input captures the received signal, and coherent detection is performed by the TROSA and DSP.

The pluggable coherent optics starts from 100 Gb/s CFP. During the past few years, its evolution of density has been quite amazing. Meanwhile, a lot of challenges to the whole industry have been triggered. Power consumption is always one of the key constrains for industrial applications. As shown in Figure 2, the requirement on power consumption for 400 Gb/s QSFP-DD DCO is less than 15 W. Over the years, the power consumption of the coherent DSP has reduced significantly [24,25,26].

Table 1. Coherent digital signal processing (DSP) power consumption evolution. CMOS: complementary metal-oxide-semiconductor.

DSP	2012	2015	2017	2019
CMOS node	40 nm	28 nm	16 nm	7 nm
Power/100G	<50 W	<20 W	<7 W	<2 W

illustrates the evolution of the power consumption of coherent DSP. One can see that, with 40 nm complementary metal-oxide-semiconductor (CMOS) process, 100G coherent traffic consumes approximately 50 W power. Nowadays, with the latest 7 nm CMOS technique, the industry can bring the power consumption of DSP down to less than 2 W, which is larger than a 20 times reduction. This tremendous improvement can potentially fit the 400G-ZR QSFP-DD DCO under a 15 W power envelop. As shown in

Table 2. 400G-ZR QSFP-DD power consumption. TROSA: transmitter and receiver optical subassembly, QSFP-DD: quad small form factor double density.

Module Power	DSP	TROSA	Other
QSFP-DD	7 W	6–7 W	1–2 W

, DSP, TROSA, and the other components are allocated only 7 W, 6–7 W, and 1–2 W power budget, respectively. In such tight power and size constrains, the industry has adopted 10–12 dB link budget in the power limited unamplified case for up to 40 km reach.

4. Experimental Setup

Figure 4 shows our experimental setup to investigate single-carrier high-baud-rate and high-order QAM signal transmission in unamplified fiber link. A 92 GS/s digital-to-analog converter (DAC) with 32 GHz analog bandwidth is used to generate electrical signals for the four tributaries of dual-polarization (DP) in-phase and quadrature (IQ) modulator. The trace loss between the DAC and the DP-IQ modulator is compensated in the digital domain by a finite impulse response (FIR) filter [27]. The root-raise cosine filter with a roll-off factor of 0.2 was applied in the following experiment. We used two tunable narrow-linewidth (<100 kHz) lasers with 16 dBm maximal output power, and set them to 1550.12 nm (i.e., 193.4 THz). One is with maximal power for the optical input to the DP-IQ modulator, and the other is set to 12 dBm to act as the local oscillator (LO) for the coherent receiver. The laser to the modulator is separated into two orthogonal polarizations, and each polarization is modulated by the corresponding IQ signals. An automatic bias control (ABC) is used to control the bias voltage applied to the DP-IQ modulator. At the Rx side, the optical signal is first de-modulated by using a 90° heterodyne coherent detector. Four linear trans-impedance amplifiers (TIAs) with differential outputs are set to automatic gain control (AGC) mode. This minimizes the BER performance dependence on the LO power, and four tributaries of analog electrical signals are generated. Then, an 8-bit analog-to-digital converter (ADC) with 63 GHz analog bandwidth and 4.5 effective number of bits (ENOB) samples the analog signals into discrete digital samples at 160 GS/s for post signal processing. In the DSP, chromatic dispersion (CD) is first compensated. Then polarization mode dispersion (PMD) is compensated and polarization tributaries are de-multiplexed by a blind equalization using the radius directed equalization (RDE). A 21-tap equalizer is implemented in the time domain with a Ts/2 spacing [28]. Furthermore, the blind estimation algorithm is used to estimate and compensate the carrier frequency offset between the Tx signal and the LO laser, followed by the carrier phase noise [29].

In our experiment, we tried to emulate the power limited unamplified case, which is being adopted by the industry with 10–12 dB link margin. According to the receiver sensitivity of 64 GBd 16-QAM signal, we adjusted the variable optical attenuator (VOA) to set the transmitter optical output power to a reference point P_{Tx Ref}, which is 12 dB greater than the receiver sensitivity at a BER of 4.2 × 10⁻². In our experiment, we generated 16-QAM and 64-QAM with symbol rates from 45 to 86 GBd. By adjusting the VOA, we also increased the Tx optical power into the single-mode optical fiber (SMF) by 3 dB and 6 dB. Optical 16-QAM and 64-QAM signals at various baud rates were then transmitted over an SMF link with distance from 10 to 110 km. Different combinations of a few fiber sections were used to achieve specific fiber length, and their corresponding losses are illustrated in

Table 3. Fiber loss characterization.

Length (km)	20	40	60	80	90	100	110
Loss (dB)	4.7	7.8	12.6	15.5	17.6	20.0	22.8

.

5. Fiber Transmission Performance Results and Discussion

In this section, we first study the fiber transmission performance with high-baud-rate 16-QAM signals in the unamplified link. Then, we extend our experiment to the high-baud-rate 64-QAM signals. Finally, we summarize a transmission matrix showing capacity and distance for 16-QAM and 64-QAM signals at various baud rates.

5.1. High-Baud-Rate 16-QAM

For 400 Gb/s DCI applications, the industry is adopting DP-16QAM signals. With additional FEC overhead, the DP-16QAM signal can also fuel the 400 Gb/s metro, regional, and even long-haul DWDM applications. Figure 5a displays the fiber transmission performance of the 56 GBd 16-QAM signal with different Tx side optical power. As the optical link is power-limited, the transmission performance can be improved for longer transmission by increasing the output power of Tx. With 6-dB more optical power than P_{Tx Ref}, the 56 GBd 16-QAM signal can transmit over 100 km with a pre-FEC BER of 2.5 × 10⁻². From the figure, one can see that, with 3-dB more optical power than P_{Tx Ref}, it is sufficient to achieve 80 km error-free transmission after FEC. With different optical power of the Tx side, the fiber transmission performance of the 16-QAM signals at 64 GBd is shown in Figure 5b. Using 25% OH SD-FEC, the 64 GBd 16-QAM signal can still carry over 400 Gb/s net data rate. Experimental result shows that a larger than 50 km unamplified optical communication link can be achieved using P_{Tx Ref}. With 6-dB more optical power than P_{Tx Ref}, the system can achieve larger than 90 km transmission in the unamplified link. Consequently, enhanced output power of Tx or improved sensitivity of Rx would be beneficial for 80 km extended long reach (ZR) application [9].

Furthermore, we compare 16-QAM signal transmission performance at various baud rates with P_{Tx Ref} and 6-dB more Tx side optical power, respectively. As shown in Figure 6a,b, we transmitted the 16-QAM signal with up to 86 GBd symbol rate. For the 86 GBd 16-QAM signals with P_{Tx Ref} and 6-dB more Tx side optical power, their reaches are around 40 km and 75 km, which are mainly limited by the increased receiver sensitivity (i.e., more receiving optical power is required) for higher baud rate system.

5.2. High-Baud-Rate 64-QAM

Compared with 16-QAM, the 64-QAM signal has 1.5× more spectral efficiency, and thus can carry more information bits. Figure 7a,b illustrates the fiber transmission performance of the 45 GBd and 64 GBd 64-QAM signals over different link distances. Compared with the 16-QAM, the 64-QAM signal has a smaller Euclidean distance between neighboring constellation points. It is thus much more susceptible to noise sources, such as laser phase noise and Rx electrical noise. In addition, compared with 16-QAM, the nonlinearity introduced by the transfer function of the IQ modulator has more impact on the 64-QAM signals. In our experiment, the modulation nonlinearity is mitigated by adjusting the electrical driving signal. As shown in Figure 7b, with 6-dB more optical power of the Tx side optical power than P_{Tx Ref}, the 64 GBd 64-QAM signal can transmit up to 40 km in the unamplified link, which gives greater than 600 Gb/s net data rate.

5.3. High-Baud-Rate QAM Transmission Matrix

After transmitting 16-QAM and 64-QAM signals with various baud rates and different Tx side powers, we summarized the single-carrier signal transmission matrix, using the net data rate and achievable unamplified link distance. As shown in Figure 8, the link distance data points are interpolated at 4.2 × 10⁻² SD-FEC threshold. The net data rate calculates only the information bits, by excluding the 25% FEC OH. For each combination of baud rate and QAM order, we chose two points of optical power at the Tx side, P_{Tx Ref} and 6-dB more optical power than P_{Tx Ref}. For the 64 GBd 16-QAM signals, these conditions provide 12 dB and 18 dB link margin, respectively. From Figure 8 one can see that, for the same modulation format, lowering the baud rate gives an extended reach at the sacrifice of the net data rate. Similarly, for the same baud rate, increasing the QAM order gives a higher net data rate at the expense of the system’s reach. For P_{Tx Ref} Tx side optical power, the 45 GBd 16-QAM signal can still achieve 80 km reach in the unamplified link, which provides 288 Gb/s net data rate. For the Tx side with 6-dB more optical power than P_{Tx Ref}, the 64 GBd 16-QAM signal can achieve both 80 km transmission and larger than 400 Gb/s net data rate. In such a condition, the 64 GBd 64-QAM signal can produce greater than 600 Gb/s net data rate for an unamplified link as long as 40 km. By comparing the two conditions of optical power at the Tx side, one can see that 6-dB more power can extend the fiber transmission reach to approximately 30 km. Consequently, it is critically important to increase the output power at the Tx side or improve the sensitivity at the Rx side. Increasing the laser’s power, reducing the modulation loss, using two separate lasers for Tx and LO are a few potential options.

6. Conclusions

In this article, we introduce 400GbE DCI using pluggable optics. Different aspects of the 400G pluggable module, including form factor, architecture, and power consumption, are discussed. Experimental investigation shows that the 64 GBd 16-QAM and 64-QAM signals can potentially enable 400 Gb/s and 600 Gb/s DCI application for 40 km and beyond unamplified fiber link. Capacity-reach matrix is finally generated with different baud rates, QAM order, and Tx side power in the unamplified fiber link. We find that increasing the Tx output power or improving the Rx sensitivity is the key for better system performance in the power-limited coherent fiber link. A 6-dB more transmitted power can extend the fiber transmission reach to approximately 30 km for both modulation formats investigated. It is thus important for the industry to develop accordingly, under the power and space constrains of the 400G DCO pluggable module.