Geometrically Shaped Odd-Bit QAM Constellations Suitable for Principal Component-Based Phase Estimation

Abstract

For high-speed optical communication systems, laser phase noise (LPN) stands as a pivotal factor influencing the quality of the received signal. Therefore, the employment of a highly accurate carrier phase recovery (CPR) algorithm at the receiving end is indispensable to ensure the reliability of transmission. While a CPR algorithm called principal component-based phase estimation (PCPE) has been proven to be capable of achieving low-complexity and high-performance phase recovery for even-bit quadrature amplitude modulation (QAM) (i.e., square QAM) signals, it is not compatible with traditional cross-shaped odd-bit QAM signals. To circumvent this problem, a signal constellation design scheme based on geometric shaping (GS) is proposed. The pair-wise optimization (PO) algorithm is used to optimize the constellation structure of 32QAM and 128QAM signals in order to obtain results that are compatible with the PCPE algorithm. Monte Carlo simulation results reveal that for odd-bit QAM signals utilizing PCPE for phase recovery, the proposed GS constellations enhance the mutual information (MI) performance across the entire measured signal-to-noise (SNR) range. Moreover, compared with regular 32QAM and 128QAM constellations using the well-known blind phase search (BPS) algorithm, the proposed GS and PCPE scheme can achieve SNR gains of 1.10 dB and 2.59 dB, respectively, when considering the 20% soft-decision forward error correction (SD-FEC) overhead. Verification through commercial simulation software corroborates these findings, demonstrating that the proposed GS constellations are particularly suitable for the PCPE algorithm, especially under conditions of high optical signal-to-noise ratio (OSNR). To the best of our knowledge, this is the first time that the incompatibility between the PCPE algorithm and odd-bit QAM signals has been investigated, and the proposed GS scheme has broadened the application scope of the low-complexity CPR algorithm.

1. Introduction

With the continuous growth of global data communication traffic, optical communication systems, as the bearers of global interconnection, are evolving in the direction of large capacity, low complexity, high flexibility, high reliability, and low cost [1,2,3,4,5,6]. To meet the application needs of optical communication systems in various scenarios, such as ultra-long-distance transmission [7], optical wireless communication [8,9], and data center interconnection (DCI) [10], a large number of cutting-edge technologies have emerged in recent years to ensure that optical transmission systems can provide high-speed, low-latency, large-capacity, and low-noise transmission services for global users.

For data center interconnection optical networks, the most urgent evolution requirements are to increase capacity and reduce power consumption to achieve the goals of green, low-carbon, energy-saving, and emission reduction while coping with the rapid surge of computing power traffic in the artificial intelligence (AI) era [11]. On the one hand, the most direct and effective solution to increase the capacity of the DCI system is to use a high-order modulation format, which can improve the single-wavelength rate without changing the baud rate and channel interval settings by improving the spectral efficiency [12]. On the other hand, the higher-order modulation format has a lower tolerance for damage such as phase noise and non-linearity in the transmission system, and often needs to use complex digital signal processing (DSP) algorithms to achieve equalization and compensation, which will pose new challenges to the power consumption of the system [13,14,15,16]. Therefore, how to design and use low-complexity digital signal processing algorithms to reduce the resource occupation required for hardware implementation as much as possible and reduce system power consumption on the premise of ensuring reliable system communication is a very noteworthy problem in DCI networks.

As an emerging low-complexity carrier phase recovery (CPR) algorithm, principal component-based phase estimation (PCPE) is very promising in the field of high-speed optical communications, due to its low cycle-slip rate (CSR), and better mutual information (MI) performance under low signal-to-noise ratio (SNR) compared with the wildly used blind phase search (BPS) algorithm [17]. However, the existing research is only limited to the application of PCPE in the square-quadrature amplitude modulation (QAM) constellations [17,18,19,20]; few studies in the literature deal with the compatibility of PCPE with odd-bit QAM constellations. As described in detail later, for the regular cross-shaped 32QAM and 128QAM constellations, due to the lack of signal points at the four corners, the PCPE algorithm will not be able to accurately estimate the phase rotation, resulting in severe achievable information rate (AIR) loss, especially when the SNR is high.

In this investigation, we conducted qualitative and quantitative simulations to explore the previously unaddressed issue of compatibility between the PCPE algorithm and the conventional cross-shaped constellation structure. Our findings reveal a noteworthy performance discrepancy, particularly concerning a 128QAM signal subjected to a slight phase shift of π/6, resulting in a substantial MI performance degradation of 0.99 bit/symbol attributed to the convergence error of the PCPE algorithm. Consequently, we introduced a tailored signal constellation design strategy specifically designed for odd-bit QAM constellations, leveraging geometric shaping (GS). Through iterative optimization utilizing the pair-wise optimization (PO) algorithm and setting the objective function as the MI performance of the PCPE algorithm under specific SNR and line width conditions, we effectively addressed the inherent compatibility challenges between the PCPE algorithm and odd-bit QAM formats.

To the best of our knowledge, this is the first time that the GS scheme has been studied to adapt the odd-bit QAM signals to the PCPE algorithm. Both numerical simulation and VPItransmissionMaker results consistently demonstrate that the proposed GS constellations are notably more suited for the PCPE algorithm, yielding superior mutual information performance across the entire measured signal-to-noise ratio (SNR) range. The simulation results show that the GS-32QAM and GS-128QAM constellation structures proposed in this paper can achieve MI performance gains of up to 0.50 bit/symbol and 1.62 bit/symbol, respectively, compared with the traditional cross-shaped constellation structure. Furthermore, the utilization of the proposed GS constellations and the PCPE algorithm substantially reduces the required SNR to achieve the 20% soft-decision forward error correction (SD-FEC) threshold, when compared to conventional cross-shaped constellations and either BPS or PCPE algorithms. Specifically, the proposed GS-32QAM scheme can achieve SNR gains of 1.10 dB and 1.60 dB compared to the schemes using the BPS algorithm and the PCPE algorithm for regular 32QAM, respectively.

The remainder of this paper is organized as follows. Section 2 provides the principles of the PCPE algorithm, qualitatively and quantitatively analyzes the interplay of it and odd-bit QAM signals, and introduces a pairwise optimization (PO) algorithm-based GS QAM generation procedure, which can design odd-bit QAM constellation structures suitable for the PCPE algorithm. In Section 3, the MI performance of the proposed scheme is verified based on both Monte Carlo numerical simulation and commercial simulation software. Section 4 is a detailed discussion of the innovations and results of this paper, including the design rationale of the proposed scheme, sensitivity analysis in different channel conditions, and compatibility with a cascading algorithm named PCPE-BPS. Section 5 summarizes our work and provides an outlook for future research details.

2. Principles

2.1. Principal Component-Based Phase Estimation Algorithm

The flowchart of the PCPE algorithm is shown in Figure 1. The main idea of this algorithm is to square the received constellation, and then extract the principal component (PC) vector corresponding to the major axis of the squared constellation through the principal component analysis (PCA) algorithm. Since there is a linear relationship between the angle of the PC vector and the phase noise suffered by the signal, the phase recovery process can be performed based on the results of PCA analysis [17]. The specific implementation process of the PCPE algorithm can be expressed by the following mathematical formulas.

Considering the situation of implementing the PCPE algorithm in a block-wise way, for the k-th data block $x_k={\left[x_k\left[1\right], x_k\left[2\right], \cdots ,x_k\left[N\right]\right]}^T$ with a length of N, the first thing that needs to be performed is to obtain $x_k^2$ by the square operation, and then separate the real and imaginary parts are separated to construct a $2\times N$ matrix $A_k$ [17]:

$$A_k=\left[\begin{array}{c}Re\left(x_k^2\right)\\ Im\left(x_k^2\right)\end{array}\right]=\left[\begin{array}{cccc}{Re(x}_k^2\left[1\right]),& {Re(x}_k^2\left[2\right]),& \cdots ,& {Re(x}_k^2\left[N\right])\\ {Im(x}_k^2\left[1\right]),& {Im(x}_k^2\left[2\right]),& \cdots ,& {Im(x}_k^2\left[N\right])\end{array}\right]$$

(1)

where $Re(.)$ and $Im(.)$ denote the real and imaginary parts of a complex number, respectively. Next, the covariance matrix $C_k$ corresponding to the current block of data can be calculated as:

$$C_k=A_k{A_k}^T$$

(2)

After obtaining the covariance matrix, the power iteration method (PIM) commonly used in PCA is performed to update and normalize the principal component $v_k={\left[\begin{array}{cc}{v}_k\left[1\right]& v_k\left[2\right]\end{array}\right]}^T$:

$$v_k=C_k{v}_{k-1}$$

(3)

$$v_k=\frac{v_k}{\sqrt{v_k^2\left[1\right]{+v}_k^2\left[2\right]}}$$

(4)

It should be noted that the initial PC vector is set as $v_o={\left[\begin{array}{cc}1& 0\end{array}\right]}^T$, in order to avoid the performance degradation caused by the mismatch between the initial value and the actual phase rotation corresponding to the first data block, the operations in (3) and (4) are usually repeated for three times using $C_1$ when extracting the PC of the first data block.

Finally, for each data block, the estimation of the phase noise can be achieved by utilizing the linear relationship between the angle corresponding to the PC and the phase rotation of the signal block ${\widehat{⌀}}_k$, which can be expressed as:

$${\widehat{⌀}}_k=\frac{1}{2}arctan\left\{\frac{v_k\left[2\right]}{v_k\left[1\right]}\right\}-\frac{\pi }{4}$$

(5)

In addition, in order to circumvent the phase ambiguity problem caused by the four-fold symmetry of the QAM signal [21], it is necessary to perform the phase unwrapping operation, so as to enhance the reliability of phase estimation. The process of updating the ${\widehat{⌀}}_k$ of the current signal block according to the ${\widehat{⌀}}_{k-1}$ corresponding to the previous block can be expressed as [17]:

$${\widehat{⌀}}_k\leftarrow {\widehat{⌀}}_k+⌊\frac{1}{2}+\frac{{\widehat{⌀}}_{k-1}-{\widehat{⌀}}_k}{\pi /2}⌋\frac{\pi }{2}$$

(6)

2.2. Interplay of PCPE and Odd-Bit QAM Signals

Although PCPE is very effective for square QAM constellations [17], as illustrated in Figure 2a~c, when it is applied to cross-shaped odd-bit QAM constellations, it will cause serious incompatibility problems. As shown in Figure 2d,e, the absence of signal points at the four corners of the odd-bit QAM constellation corresponds to the absence of high-power points on the main axis of the squared constellation (as shown by the green circles), which will seriously reduce the accuracy of the PCPE algorithm. Taking 128QAM and 256QAM as an example for a qualitative comparison, after applying the PCPE algorithm to perform phase recovery on the constellation diagrams in Figure 2a,d, the results obtained are as shown in Figure 2c,f. It can be clearly seen that the residual phase error in the 128QAM constellation diagram is more serious than 256QAM.

A quantitative analysis of the interplay between PCPE and odd-bit QAM signals was performed by Monte Carlo simulations. Under different SNR conditions, the PCPE algorithm is applied for phase recovery of the 128QAM and 256QAM signals subject to a fixed π/6 phase rotation, and the MI performance curves obtained are as shown in Figure 3. For 256QAM, the PCPE algorithm can achieve relatively accurate recovery performance in the entire SNR interval being tested, and the AIR loss compared to the MI performance in the ideal state is no more than 0.41 bit/symbol. In contrast, for 128QAM, which is an odd-bit QAM, the PCPE algorithm will have obvious performance defects in the high SNR range, resulting in an AIR loss of up to 0.99 bit/symbol. These results show that the PCPE algorithm is not compatible with the regular cross-shaped odd-bit QAM signal, which will lead to a serious decrease in the estimation accuracy, and the deterioration of the reliability of the entire communication system.

2.3. Geometric Shaping Scheme Based on Pair-Wise Optimization Algorithm

As one of the typical research hot spots in the field of modulation format optimization technology, geometric shaping is widely used in optical communication systems. By changing the geometric distribution of the signal constellation, GS technology can effectively improve the overall quality of the received signal [22,23,24,25,26,27]. The pair-wise optimization (PO) algorithm is a commonly used geometric shaping method to adjust and optimize the signal constellation structure iteratively [25,26,27]. The main idea of this scheme is to randomly select two constellation points as a pair during each iteration and optimize their coordinates according to a specific objective function (usually the bit error rate or symbol error rate under a specific channel condition), subject to [22]:

$$\sum _{k=1}^M{s}_k=0 and \sum _{k=1}^M{‖s_k‖}^2=M$$

(7)

where $s_k$ represents any constellation point in the $M$-order constellation diagram, and $‖.‖$ represents the norm operation of a vector. For any pair of $s_i$ and $s_j$, where $1\le i\le M, 1\le j\le M,i\ne j$, both the zero mean and average power constraints in (6) can be rewritten as [26]

$$s_i=-b-s_j$$

(8)

$${‖s_j+\frac{b}{2}‖}^2=\frac{M-d}{2}-\frac{{‖b‖}^2}{4}$$

(9)

where $b={\sum }_{k=1,k\ne i,k\ne j}^M{s}_k$ and $d={{\sum }_{k=1,k\ne i,k\ne j}^M‖s_k‖}^2$. In other words, the optimization process can be simplified as finding the position of $s_j$ on a circular trajectory centered at $-\frac{b}{2}$ with a radius of $\sqrt{\frac{(M-d)}{2}-\frac{{‖b‖}^2}{4}}$.

2.4. Geometrically Shaping of Odd-Bit QAM Constellations for Compatibility with the PCPE Algorithm

In terms of improving the adaptability of the signal structure to the receiving-end CPR algorithm through the GS technology, optimization schemes for the BPS algorithm [22], the differential BPS algorithm [23], and the pilot-based phase recovery algorithm [24] have been reported in the current literature. However, the GS scheme costumed for the PCPE algorithm has not yet been reported. To solve the incompatibility between the PCPE algorithm and odd-bit QAM signals, we adopted the PO algorithm, and set the objective function in the PO process to the MI performance of odd-bit QAM signals after PCPE recovery under specified SNR and line width. Moreover, to further simplify the geometrical designing process, we adopted the same approach as in [27], that is, only the constellation points in the first quadrant were subjected to the PO process, and then flipped along the in-phase axis and quadrature axis, respectively, to obtain the overall constellation.

During the PO process, the MI performance corresponding to each constellation is the average after 100 Monte Carlo simulations with a symbol rate of 32 GBaud, a combined linewidth of 200 kHz, a total symbol sequence length of 16,384, and a block length of PCPE phase recovery of 64. As for the specified SNR, it is 13 dB for 32QAM and 18 dB for 128QAM, these parameter settings refer to the SNR required for both signals to reach the 20% SD-FEC threshold when only affected by AWGN. In the GS process, the structure of the constellations is adjusted to maximize the MI values that can be achieved by PCPE under these conditions, which is conducive to reducing the SNR cost required for reliable communication as much as possible in practical applications.

The MI trace curves corresponding to GS-32QAM and GS-128QAM signals are shown in Figure 4a,b. Notably, post the PO process, effective resolution of the incompatibility issue between PCPE and odd-bit QAM signals is observed. Substantial enhancements in MI performance are noted under the evaluated SNRs for both 32QAM and 128QAM, approximately 0.33 bits/symbol and 0.72 bits/symbol, respectively. The optimal constellation diagrams for these two modulation formats are depicted in Figure 5a,b, while their corresponding squared constellation diagrams are presented in Figure 5c,d. A comparative analysis of the constellation structures in Figure 2 and Figure 5 reveals that the GS process facilitates the transformation of the odd-bit QAM constellation diagram from a regular cross shape to a square shape. This transformation imparts a constellation structure akin to that of even-bit QAM, playing a pivotal role in elevating the estimation accuracy of the PCPE algorithm.

3. Results

3.1. Monte Carlo Simulation Results

To further evaluate the performance of the proposed geometric shaping constellation designs under different SNRs, and compare it with the performance of regular constellations, a numerical simulation was carried out. In the simulation system, the symbol rate of the odd-bit QAM signal was 32 GBaud, while the total linewidth of the lasers on the transmitting end and receiving end combined was 200 kHz. Cycle slips were ideally compensated to simplify the MI computation, as [17]. Each MI value was an average obtained through 32 realizations with 1024 blocks of 64 symbols.

Figure 6 shows the MI performance using the GS constellations and PCPE algorithm, as well as the MI performance using the regular constellations and two different phase recovery algorithms, namely the BPS algorithm and the PCPE algorithm. For the BPS algorithm, a two-stage implementation is adopted, and the number of test phases in each stage is set to 11, which realizes the effect of searching among a total of 121 test phases. This implementation and parameter setting are consistent with [17].

As shown in Figure 6a, for a regular cross-shaped 32QAM constellation, applying the PCPE algorithm can achieve better MI performance than the BPS algorithm in the low SNR value range. However, when the SNR is high, due to the incompatibility between the cross-shaped constellation and the PCPE algorithm, the severe residual phase error will result in a maximum AIR loss of about 0.38 bit/symbol compared with the BPS algorithm. By using the proposed GS-32QAM constellation, the MI performance advantage of the PCPE algorithm relative to BPS in the low SNR stage can be further expanded, and the AIR loss in the high SNR stage compared to BPS can be effectively reduced to below 0.06 bit/symbol. Throughout the entire evaluated SNR range, by replacing the regular 32QAM (R-32QAM) with the proposed GS-32QAM, the MI performance improvement of the PCPE algorithm varies from 3.48% and 15.73%. When considering the threshold corresponding to 20% SD-FEC overhead, the proposed scheme can achieve SNR gains of 1.10 dB and 1.60 dB compared to the schemes using the BPS algorithm and the PCPE algorithm for regular 32QAM, respectively.

Similarly, the MI curves using 128QAM format are shown in Figure 6b. For the cross-shaped constellation, the AIR loss of using the PCPE algorithm under high SNR compared to using the BPS algorithm is up to 1.30 bit/symbol. After employing the proposed GS-128QAM constellation, the AIR loss can be greatly reduced to 0.05 bit/symbol. Throughout the entire evaluated SNR range, by replacing the regular 128QAM (R-128QAM) with the proposed GS-128QAM, the MI performance of the PCPE algorithm is improved by a percentage above 7.34% and below 22.17%. When considering the threshold corresponding to 20% SD-FEC overhead, the proposed scheme can achieve an SNR gain of 2.59 dB compared to the traditional scheme using the BPS algorithm and a regular 32QAM constellation.

3.2. VPItransmissionMaker Simulation Results

Moreover, the proposed GS scheme is rigorously validated through a VPItransmis-sionMaker simulation, conducted within a 32 GBaud single polarization coherent detection system, as illustrated in Figure 7. In the transmission process, a co-simulation program is utilized at the transmitter end to generate odd-bit QAM signals corresponding to both cross-shaped constellations and GS constellations, and the generated digital sequences are converted into radio frequency (RF) signals at a rate of eight samples per symbol. These signals, after being amplified by identical electrical drivers, are then subjected to electrical-optical conversion using an IQ modulator with a half-wave voltage of 5 V, an insertion loss of 6 dB, and an extinction ratio of 35 dB. For the continuous wave (CW) laser that provides the optical carrier signal, its center wavelength is 193.1 THz, the linewidth is 100 kHz, and the average power is 5 dBm. The modulated signal’s power is precisely adjusted to 0 dBm via an Erbium-doped fiber amplifier (EDFA). Control over the optical signal-to-noise ratio (OSNR) is achieved through the Set OSNR component for MI measurement. On the receiving end, the de-sired signal is combined with a local oscillator (LO) signal within the coherent receiver, which also has a center frequency of 193.1 THz and an output power of 0 dBm. Subsequently, the electrical signal is captured at a sampling rate twice the baud rate and with a resolution of 8 bits, and the stored data undergo a comprehensive digital signal processing program for signal recovery and demodulation. This process encompasses normalization, the Gram–Schmidt orthogonalization process (GSOP), frequency offset estimation (FOE), PCPE, and MI calculation.

Figure 8a gives the measured MI versus OSNR curve for 32QAM obtained by VPItransmissionMaker simulation. For regular 32QAM signals using PCPE for phase recovery, even when the OSNR is as high as 30 dB, there is still an AIR loss of about 0.07 bit/symbol. In contrast, the AIR loss corresponding to the proposed GS-32QAM signal after PCPE recovery is almost negligible when the OSNR is about 26 dB. It can be seen from the insets of constellation diagrams in Figure 8a that, under the same OSNR, the residual phase noise of the GS-32QAM after PCPE phase recovery is significantly smaller than R-32QAM. In addition, the mean absolute error (MAE) values of the residual phase error between the recovered R-32QAM and GS-32QAM signals and the original signals are also calculated under different OSNRs, as shown in Figure 8b. The measured results show that the MAE values corresponding to the GS-32QAM are significantly lower than those of the R-32QAM throughout the entire tested OSNR range, which indicates that the proposed constellation structure can indeed improve the accuracy of the PCPE algorithm. Moreover, as the value of OSNR increases, the MAE reduction percentage that can be achieved by replacing R-32QAM with GS-32QAM gradually increases from 15.48% to 39.05%, which once again proves that the proposed scheme can effectively solve the incompatibility problem of the traditional constellation structure and the PCPE algorithm especially in the high SNR range.

The performance comparison between the proposed GS constellation and the regular constellation in 128QAM modulation format is shown in Figure 9a. For the regular 128QAM constellation, the MI performance cannot correctly converge to 7 bit/symbol. Even when the OSNR is high enough, there is still an AIR loss of about 1.06 bit/symbol. On the other hand, the GS-128QAM constellation can achieve a much better MI performance, especially when the OSNR is high. Throughout the entire tested OSNR range, the MI performance can be improved by at least 0.85 bit/symbol, which indicates that the proposed constellation diagram structure significantly improves the estimation accuracy of the PCPE algorithm. As shown in the insets of constellation diagrams and the MAE curves in Figure 9b, the residual phase noise of R-128QAM after PCPE recovery is much greater than GS-128QAM. In the OSNR interval from 25 dB to 40 dB, the MAE reduction percentage that can be achieved using the proposed GS constellation surges from 35.55% to 52.42%, which proves that the proposed scheme can achieve the most significant improvement in phase estimation accuracy and reliability under the channel condition of high OSNR.

4. Discussion

4.1. The Design Rationale of the GS Constellation Structure

In this subsection, the design rationale behind the selected geometric shapes for odd-bit QAM constellations is discussed. Based on the analysis in Section 2.2, it can be seen that the core reason why the traditional cross-shaped odd-bit QAM signals are not compatible with the PCPE algorithm, is the lack of constellation points at the four corners of the constellation diagram, which leads to the decrease in the obviousness of the PC vector in the squared constellation diagram. Since the rotation angle of the PC corresponding to each block is extracted by only one PIM iteration in the PCPE algorithm, the estimation accuracy of the PCPE algorithm will be significantly deteriorated when the obviousness of the principal component decreases.

To address the compatibility challenges between the PCPE algorithm and odd-bit QAM signals, we employ the geometric shaping technique, specifically the PO algorithm, to reconfigure the constellation structure of odd-bit QAM. The previous literature primarily utilized the PO algorithm to optimize constellation designs for achieving maximal MI or BER performance in channels affected solely by AWGN. Consequently, the typical constellation structures designed tend to approximate a circular shape, ensuring that the distribution of the entire constellation closely resembles the Maxwell–Boltzmann distribution when the occurrence probability of each constellation point is uniform [28,29,30]. However, such a constellation design scheme cannot effectively solve the incompatibility problem between the PCPE algorithm and the odd-bit QAM signal. In order to achieve the adaptation between the two, it is necessary to try to approximate the constellation structure to a square, so as to ensure that the PIM can quickly and accurately capture the PC vector. Therefore, in this paper, we select the MI value after PCPE recovery of odd-bit QAM signals as the objective function of the PO iteration, and obtain the constellation structures of GS-32QAM and GS-128QAM that tend to be square.

To elucidate the impact of the proposed constellation structure on the recovery performance of the PCPE algorithm, we conducted a comparative analysis of the iteration counts required for PIM convergence using ideal GS-32QAM and R-32QAM signals subject to a π/6 phase shift, as depicted in Figure 10. Our findings demonstrate that the employment of the GS-32QAM constellation structure, as devised in this study, effectively expedites the convergence process of PIM, achieving highly accurate phase estimation after only four iterations. Conversely, for the R-32QAM, approximately sixteen times more iterations are necessary to attain comparable recovery accuracy. This analysis underscores that the GS scheme devised herein markedly enhances the clarity of the PC vector in the constellation diagram following squaring of the odd-bit QAM signal. It notably accelerates the convergence speed of PIM and diminishes the residual phase error magnitude, thereby enhancing the recovery efficacy of the PCPE algorithm.

Furthermore, it is imperative to consider the comprehensive impact of both LPN and AWGN on signal performance when determining the SNR for executing the pair-wise optimization (PO) iteration. This consideration involves balancing the adaptability of the designed constellation structure to both the principal PCPE algorithm and the goal of approximating a square shape. Simultaneously, it is crucial to ensure that the designed constellation structure maintains reasonable MI performance even under the sole influence of AWGN, emphasizing the need for a sufficient minimum Euclidean distance in the constellation diagram. This intricate trade-off is achieved through the iterative process of the PO algorithm, performed with the appropriate SNR value. As shown in Figure 6a, 13 dB is selected as the SNR for PO iteration of 32QAM, and the designed constellation structure not only achieves the approach from the cross type to the square, but also ensures the Euclidean distance between adjacent constellation points, so the MI performance can be significantly improved in the entire SNR range.

In contrast, if the SNR chosen for the PO iteration is too high (e.g., 25 dB), the GS process will make the constellation structure of the design approximate a square by sacrificing the minimum Euclidean distance excessively, as shown in Figure 11a. As can be seen from the performance comparison in Figure 11b, the constellation structure in Figure 11a incurs an SNR penalty of approximately 0.35 dB at the 20% SD-FEC threshold, compared to the constellation structure in Figure 5a.

Overall, according to the results shown in Figure 6 and Figure 11, the constellation structures given in Figure 5 exhibit good universality, and the MI performance close to the theoretical value can be obtained under different SNRs. Therefore, it is recommended to select the SNR value corresponding to the FEC threshold for constellation design when performing the PO iteration, to ensure that the designed constellation structure can achieve reliable transmission performance under different channel conditions.

4.2. Sensitivity Analysis

In this section, we investigate the sensitivity of the proposed GS constellation structures when confronted with varying parameters between the training and testing channels. Specifically, we simulate and test the MI performance of the GS-32QAM and GS-128QAM constellations, which were trained and derived under a 200 kHz system line width condition, across different channel conditions where the combined line widths are 100 kHz and 500 kHz, respectively. The resulting simulation outcomes are depicted in Figure 12. First, the results in Figure 12a,b show that GS-32QAM and GS-128QAM can improve the MI performance of the PCPE algorithm by up to 0.24 bit/symbol and 1.04 bit/symbol, respectively, when the linewidth value of the testing channel is 100 kHz. Although the MI improvement that can be achieved by the GS scheme is slightly reduced compared with the scenario with a testing linewidth of 200 kHz, the fact that the GS constellation structures can effectively adapt to the PCPE algorithm does not change. Then, as shown in Figure 12c,d, the two GS constellations deliver up to 0.50 bit/symbol and 1.62 bit/symbol MI performance improvements, respectively, at a linewidth of 500 kHz for the testing system.

Drawing upon the aforementioned findings, it is evident that despite variations in parameter settings between the testing and training channels, the proposed GS constellation structures consistently enhance the MI performance of the PCPE algorithm across all assessed SNR ranges. Furthermore, as the line width of the testing systems increases, the MI performance gains are observed through the adoption of the proposed GS constellation structures in lieu of traditional cross-shaped constellation configurations.

In addition, the tolerance and sensitivity analysis of the proposed scheme for chromatic dispersion (CD) effect, amplified spontaneous emission (ASE) noise, and Kerr nonlinear effect are also studied after transmission through the fiber channel, to verify the feasibility of the proposed scheme in actual communication scenarios. The transmission system evaluation is still based on the VPI simulation software. The difference compared to the system setup in Figure 7 is that the Set OSNR component is removed and replaced with a fiber link consisting of a single-mode fiber (SMF) spanning 80 km, an EDFA with a gain of 16 dB, and a noise figure of 4 dB, and a recirculation fiber loop. The attenuation value of the fiber is set to 0.2 dB/km, the dispersion coefficient is set to 16 ps·nm⁻¹km⁻¹, and the nonlinearity coefficient is 2.6e⁻²⁰ m²/W. Moreover, in the DSP chain at the receiving end, the CD compensation algorithm based on frequency domain filtering is supplemented after the GSOP algorithm.

Firstly, the transmission performance of GS-32QAM signal is analyzed under the condition that the total length of the fiber link is 400 km (that is, the number of recirculating loops is 5). For the launched optical power in the range of −5 dBm to 5 dBm, the MI performance corresponding to the two modulation formats is tested by adjusting the output power of the EDFA connected to the output of the IQ modulator, and the results are shown in Figure 13. The MI performance of the GS-32QAM is better than that of the conventional R-32QAM after the recovery of the PCPE algorithm over the entire measured launched optical power range. Specifically, in the process of increasing the launched power from −5 dBm to 1 dBm, that is, in the range where the system performance is mainly dominated by the ASE noise, the MI values of the two modulation formats increase with the increase of optical power. Then, when the launched power exceeds 1 dBm, the distortion caused by the fiber nonlinear effect begins to become the main factor that dominates the system performance. As a result, the MI performance of the two modulation formats start to gradually decrease after reaching the maximum values of 4.99 bit/symbol and 4.76 bit/symbol, respectively, and will drop to 4.67 bit/symbol and 4.45 bit/symbol when the launched power is 5 dBm. Overall, for the constellation structure of GS-32QAM designed in this paper, its sensitivity to nonlinear noise does not exhibit obvious differences from that of the traditional R-32QAM.

Secondly, the MI performance of GS-128QAM and R-128QAM after fiber link transmission are also tested and compared. After 160 km, i.e., two spans of SMF transmission, the MI results corresponding to the two modulation formats under different channel conditions are shown in Figure 14. For the R-128QAM, due to the serious incompatibility between it and the PCPE algorithm, even at the optimal launched optical power value, the corresponding MI performance is only 4.84 bit/symbol after fiber transmission, which is far below the threshold required for reliable transmission. In comparison, the proposed GS-128QAM signal achieves the highest performance value of 6.62 bit/symbol at 0 dBm. When the launched optical power gradually increases from 0 dBm to 5 dBm, the MI performance value of GS-128QAM will gradually decrease, but even in the case of the most severe nonlinear distortion, the MI value can still reach 5.91 bit/symbol, which exceeds the threshold corresponding to 20% SD-FEC.

Based on the above results, it can be seen that the GS-32QAM and GS-128QAM proposed in this paper also show good tolerance to ASE noise and nonlinear distortion, and can achieve MI performance significantly better than traditional odd-bit QAM signals even after 100 km of fiber link transmission.

4.3. Compatibility Analysis with the Cascading PCPE-BPS Algorithm

In this subsection, the compatibility of the proposed scheme with the cascading PCPE-BPS algorithm is simulated and analyzed. The PCPE-BPS algorithm is a two-stage phase recovery algorithm proposed in [17] to further improve the MI performance of the PCPE algorithm in the high SNR range. The main idea is to use the PCPE algorithm for coarse phase recovery in the first stage, and then perform a BPS algorithm for second stage recovery with test phases of [17]:

$${⌀}_b=\eta \pi \left(\frac{2b-1}{4B_2}-\frac{1}{4}\right), b\in \left\{\mathrm{1,2},\dots ,B_2\right\},$$

(10)

where $B_2$ represents the total number of test phases and $\eta$ represents the phase aperture in which the fine phase recovery will be carried out.

In order to explore the compatibility of the PCPE-BPS algorithm with the proposed GS scheme, the simulation system in Section 3.1 is used again, and the performance is studied and compared under the same parameter settings. The obtained simulation results shown in Figure 15 indicate that, for both 32QAM and 128QAM modulation formats, the MI performance of the PCPE-BPS algorithm can be significantly improved by using the GS scheme proposed in this paper. Taking the 20% SD-FEC threshold as a reference, SNR gains of 0.82 dB and 1.82 dB can be achieved, respectively.

5. Conclusions and Future Research Details

Key research contributions in this paper include: In the context that the existing literature only focuses on the application of PCPE algorithm to even-bit QAM [17] or even-bit QAM based on different probabilistic shaping models [19,20] signals, this study investigates the compatibility between the PCPE algorithm and traditional cross-shaped odd-bit QAM signals for the first time. Results from qualitative and quantitative analyses confirm that the absence of high-power constellation points at the four corners of the constellation diagram is the primary reason for the decline in recovery accuracy of the PCPE algorithm when applied to odd-order QAM signals. Building upon the aforementioned analysis, this paper introduces, for the first time, a geometric shaping optimization scheme tailored for the PCPE algorithm. Inspired by previous works [22,24], the proposed scheme achieves optimization by defining the objective function in the PO iteration process as the MI performance index post-recovery of the PCPE algorithm under specific channel conditions. This approach ensures an optimized design that comprehensively considers the visibility of the Post-Squared Constellation’s PC vector while ensuring that the MED of the constellation is not excessively small. After the PO iterative convergence, the overall structure of the optimized GS-QAM constellations obtained tends to be closer to a square, rather than to a circle like the existing schemes optimized for AWGN only [28,29,30], so it can be suitable for the working principle of the PCPE algorithm. The simulation results have also proved that the scheme in this paper can extend the application scope of the PCPE algorithm from only supporting even-bit QAM to full QAM orders.

In summary, this paper studies for the first time the compatibility between the PCPE algorithm and odd-bit QAM signals, and points out that the traditional cross-shaped constellation structure will lead to a serious decrease in the estimation accuracy of the PCPE algorithm under high SNR range. Correlation quantitative analysis shows that even for the R-128QAM signal that is only subjected to a fixed π/6 phase rotation, the PCPE algorithm is seriously incompatible with it in the high SNR range, resulting in an MI loss of up to 0.99 bit/symbol. In response to this question, we present a GS approach tailored for odd-bit QAM constellations, demonstrated through rigorous validation via numerical simulation and VPItransmissionMaker simulation. Notably, this study marks the pioneering application of the GS technique to achieve compatibility with the PCPE algorithm. The geometrically shaped signals, when compared to traditional cross-shaped constellation structures, exhibit a substantial enhancement in mutual information performance across the entire measured SNR range. By applying the GS-32QAM and GS-128QAM constellation structures designed in this paper to replace the traditional cross-shaped odd-bit QAM signals, the maximum achievable MI enhancement can reach 0.5 bit/symbol and 1.62 bit/symbol, respectively.

The proposed constellation design scheme effectively extends the applicability of the low-complexity PCPE algorithm, offering a viable alternative for practical optical communication systems utilizing odd-bit QAM formats. Specifically, due to the clear pursuit of high speed and low power consumption in DCI interconnection scenarios, the proposed scheme can make it possible to reliably apply the low-complexity PCPE algorithm in the odd-bit QAM communication system with high spectral efficiency. Therefore, the proposed scheme will bring significant advantages in this particular communication system.

In future investigations, our focus will delve into the following aspects: 1. Streamlining the iterative optimization process integral to GS, aiming for expeditious and efficient constellation structure optimization tailored for the PCPE algorithm; 2. Studying the Odd-bit QAM GS scheme based on 4D modulation [30], to realize the constellation structure design that can be adapted to the dual-polarization joint PCPE algorithm; 3. Optimizing the implementation of the PCPE algorithm, and attempting to achieve compatibility with odd-bit QAM signals through the improvement of the algorithm implementation side.