Enhanced PON and AMCC Joint Transmission with GMM-Based Probability Shaping Techniques

Abstract

In ITU-T standards, auxiliary management and control channels (AMCCs), as defined, facilitate the rapid deployment and efficient management of wavelength division multiplexing passive optical network (WDM-PON) systems. The super-imposition of an AMCC introduces additional interference to a PON signal, resulting in the degradation of the performance of the overall transmission. In prior research, we proposed employing a Gaussian mixture model (GMM) to fit a baseband-modulated AMCC signal. Following the analysis of the interference model and the distribution characteristics of received signal errors, we propose a combined optimization method for a transmitter and receiver in this paper. This method, grounded in probabilistic shaping (PS) techniques, optimizes the probability distribution of the transmitted signal based on the AMCC interference model, with the objective of reducing the error rate in PON signal transmission. We have validated this approach within a 50G-PON experimental system by utilizing PAM4 modulation. The experimental results demonstrate the effectiveness of this method for mitigating the impact of baseband-modulated AMCC, thereby reducing the error rate in PON signal transmission. The approach presented in this paper can further minimize the performance degradation introduced by baseband-modulated AMCC in WDM-PON systems, enhancing the efficiency of WDM-PON deployment.

1. Introduction

According to the latest whitepapers from mobile network operators [1,2], the current demand for internet capacity has experienced explosive growth. In the next 3–5 years, there will still be significant demand for mobile internet, as evidenced by the planned upgrade from the fifth-generation (5G) mobile network architecture to the sixth-generation (6G). This upgrade is expected to potentially increase network capacity by up to 100 times, with end-to-end latency reduced to less than 1 millisecond. Optical communication stands out as an excellent choice for high-speed communication networks, and wavelength division multiplexing passive optical network (WDM-PON) is a particularly attractive optical communication technology that is applicable in various scenarios, including 5G mobile fronthaul (MFH) networks [3,4]. The deployment of WDM-PON in networks requires an auxiliary management and control channel (AMCC) to achieve efficient network deployment. This channel has been defined in the ITU-T G.989 series standards [5,6,7]. In recent years, extensive research has been conducted on AMCC by various institutions. This research encompasses modulation and transmission techniques, as well as methods to enhance transmission performance [8,9,10,11,12,13,14,15]. Through these advancements, the transmission speed of an AMCC has been elevated to the level of 20 Mbps under specific conditions. Studies in the realm of system applications suggest that an AMCC can play a role in wavelength management and control [16,17], with NTT conducting research on the application of AMCCs in all optical networks [18]. However, as future networks demand higher capacity and lower transmission latency, it is necessary to increase the transmission speed of WDM-PON systems further. Research indicates that PON systems are currently in a new upgrade cycle, and as early as 2020, ITU-T proposed initiating the development of mobile-centric WDM-PON standards [19]. Studies on standardization progress suggest that the introduction of forward error correction (FEC) coding will raise the line rate of PON systems to over 50 Gbps, providing widespread benefits for both regular and latency-sensitive wireless applications [20]. In order to achieve the goal of enhancing PON transmission rates, various methods can be considered, including increasing channel bandwidth, applying simplified coherent techniques, and employing advanced modulation schemes, such as four-level pulse amplitude modulation (PAM4). Consequently, the AMCC responsible for operation administration and maintenance (OAM) data also requires new technologies to support PON upgrades. Research indicates the feasibility of transmitting AMCCs in a 50 Gbps PAM4 network [21,22]. An AMCC can also be employed in simplified coherent systems, as suggested in [23], which introduces a block-based digital signal processing method used to extract AMCC signals in a 25 Gbps QPSK coherent communication system, resulting in a power penalty reduction of 0.2 dB for 128 kbps AMCC signals. Compared to on-off keying (OOK) signals, PAM4 signals carry double information per symbol, reducing the link bandwidth requirements at the same bit rate but making them more susceptible to interference. Thus, the signal attenuation caused by the overlay of AMCC on a PAM4 system is more severe than in an OOK system at the same rate. In order to achieve channel management functions, high-speed AMCC signals need to be correctly demodulated, requiring the maintenance of sufficient AMCC signal amplitude to meet the signal-to-noise ratio (SNR) requirements. However, the larger the amplitude of the AMCC signal, the more interference it introduces to the PON, making it challenging to meet the power penalty requirements specified by the ITU standards [6,7]. In response to these challenges, in [24], we proposed a modeling method for interference signals to assess mixed signals, and we introduce a novel joint demodulation receiver structure capable of simultaneously demodulating PON and AMCC signals while maintaining excellent demodulation performance. In this article, we conduct an analysis of the distribution parameters of the received signal using the AMCC interference model in a multi-level modulation PON system. Based on this analysis, we propose an enhanced transmission system for PON-AMCC signals. This improved method is grounded in probability shaping (PS) techniques and is optimized using a Gaussian mixture model (GMM). By employing a GMM fitting method to acquire signal distribution parameters, we scrutinize interference intensity and error symbol probabilities at different positions in the channel. This adjustment aims to strategically position more signals in areas with lower interference, thereby reducing the transmission error rate and enhancing the overall system’s transmission performance. We further apply this method to optimize the joint demodulation receiver proposed in [24]. The effectiveness of this approach is validated in an experimental system with a 50G-PON with an AMCC superimposed, demonstrating improved transmission performance compared to signals transmitted with equal probability.

2. Interference Modeling and Error Symbol Analysis

The transmission of baseband AMCC signals in a multi-level modulation PON signal system can be achieved using a distributed feedback laser (DFB) and a Mach-Zehnder modulator (MZM) [22]; its basic structure is illustrated in Figure 1. The blue segment represents electrical signals, while the green segment represents optical signals.

The PON signal is input through the RF port of the modulator, while the AMCC signal is connected to the DC bias port. Under appropriate operating conditions, a superimposed baseband AMCC can be achieved, resulting in the output signal being superimposed in the form of Equation (1).

$$\begin{array}{c}\hfill \begin{array}{c}\hfill P_O\left(t\right)=\frac{G\ast P_I\left(t\right)}{2}\left(1+cos\left[\frac{\pi }{V_{\pi }}[A_P\ast S_{PON}\left(t\right)+V_{BAIS}+A_M\ast S_{AMCC}\left(t\right)]\right]\right)+n\left(t\right).\end{array}\end{array}$$

(1)

In the equation, $S_{PON}\left(t\right)$ and $S_{AMCC}\left(t\right)$ are the values of the PON signal and AMCC signal, respectively, and $A_P$ and $A_M$ represent the amplitude of the PON and AMCC signals, respectively. The modulation index of AMCC can be expressed as $A_M/A_P$. The parameter G is the insertion loss of MZM, with a value of less than 1, and $V_{\pi }$ is the half-wave voltage of MZM. $V_{BIAS}$ indicates the DC bias voltage of the modulator, which determines the operating conditions of the modulator. $P_I\left(t\right)$ donates the input optical signal power of the modulator.

Through mathematical derivation, it can be inferred that the distortion caused by AMCC varies for different values of the PON signal. The essential reason for this phenomenon is that the relationship between the input and output signals of the MZM is a cosine mapping rather than a linear mapping. This characteristic results in a significantly lower distortion value for PON signals near the top and bottom of the cosine curve when AMCC is superimposed when compared to signals near the middle of the curve. This feature can be validated through the eye diagram of the signal. For example, Figure 2 illustrates an eye diagram of a PAM4-PON signal overlaid with OOK-AMCC. In the eye diagram, it can be observed that the 4-level PON signal splits into eight levels. However, it is evident that the splitting amplitudes of the signal levels at the top and bottom are much smaller than those at the two middle levels. This observation aligns with the mathematical analysis presented earlier in Equation (1).

With the increasing bandwidth of AMCC, traditional interference elimination methods have limited effectiveness. After analyzing the interference characteristics of AMCC, a new interference model based on GMM was proposed in [24]. By analyzing the interference model, we understand that introducing an AMCC results in the splitting of each level of the PAM4 signal into multiple levels. Consequently, the statistical characteristics of the corresponding signals are more complex, impacting the demodulation of the signals given by the receiver. In a joint demodulation receiver, the receiving end utilizes a GMM fitting module to model the received signal and estimate the distribution parameters. For an additive white Gaussian noise (AWGN) channel, the received signal conforms to a Gaussian mixture model, which can be expressed as [25]

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \begin{array}{cc}\hfill p\left(x\right)& =\sum _{k=1}^K{\rho }_{k}N\left(x\right|{\mu }_k,{\sigma }_k),\hfill \end{array}\end{array}\end{array}$$

(2)

where $N\left(x\right|{\mu }_k,{\sigma }_k)$ is called the k-th component in the mixture model, and ${\rho }_k$ corresponds to the weight of that component and satisfies

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \sum _{k=1}^K{\rho }_k=10\le {\rho }_k\le 1.\end{array}\end{array}$$

(3)

The ${\mu }_k$ and ${\sigma }_k$ for each level reflect the channel’s impact on the signal at that position, representing the SNR of the channel at that location. By evaluating the parameters of the GMM for the received signal, we can assess the SNR for different levels. When the SNR is high at a particular position, signals appearing at that position should exhibit good transmission performance, characterized by a relatively low error rate. Conversely, when the SNR is poor at a particular position, the signals appearing at that position should exhibit a higher error rate. For instance, in a transmission system with a PAM4-PON signal overlaid with OOK-AMCC, according to Equation (1), the received signal should conform to a Gaussian mixture distribution with eight peaks. After fitting a GMM to the received signal using Equation (2), the signal distribution might resemble the one shown in Figure 3. From the figure, it can be observed that when the signal levels are PAM4 level 2 and level 3, the received signal has a higher SNR. Accordingly, a received signal that conforms to this model and has its level at positions 2 to 3 should exhibit better transmission performance, with a lower error rate compared to signals at other positions. Therefore, by transforming the transmitted signal to place more information at positions with a higher SNR, it is possible to effectively reduce the error rate and enhance the overall transmission performance of the system. In order to achieve this purpose, a feasible method is to use probability-shaping techniques to adjust the distribution probability of the transmitted signal.

3. Enhanced Transmission System Utilizing GMM-Based Probability Shaping Techniques

3.1. GMM-Based Probabilistic Shaping Techniques

Probabilistic Shaping (PS) is a technique utilized in digital communication systems to improve information transmission performance over noisy channels. This approach involves meticulous signal design, incorporating probabilistic considerations to mitigate the impact of noise and interference. The core concept is to assign probabilities to different symbol outputs, prioritizing symbols with lower error rates for transmission. The foundational implementation method of the PS techniques has been extensively discussed in previous research, with constant composition distribution matching (CCDM) being a common approach for PS [26,27]. In a PAM modulation system, each symbol represents a specific amplitude level, and the probability distribution of symbol points is uniform in traditional systems, meaning each symbol point has an equal probability of occurrence. However, PS techniques optimize the probability distribution and reduce the probability of occurrence for levels with inferior signal-to-noise ratios (SNRs) while elevating the probability of levels with superior SNRs. Assuming that a PAM-L-modulated signal is generated from the level set $S=\{1,2,\cdots ,L\}$, the probability density function (PDF) $D_S\left(x\right)$ tends to align the signal distribution with a Maxwell-Boltzmann (M-B) distribution [28],

$$\begin{array}{c}\hfill \begin{array}{c}\hfill D\left(x\right)=\frac{e^{-{v\left|x\right|}^2}}{{\sum }_{x^{\prime }\in S}{e}^{-v|x^{\prime }{|}^2}},\end{array}\end{array}$$

(4)

where v is a rate parameter utilized to control the kurtosis of MB distribution. Due to the change in signal distribution, the value of v also undergoes a change, indicating that the non-uniform probability distribution reduces the entropy of the transmitted signal. PS technology has been shown to improve transmission performance in optical communication systems [29,30,31,32].

In the PON-AMCC system, we have re-modeled the signal, constructing it as a Gaussian mixture model. In the GMM-based signal model, as described in Equation (2), each PAM level has an associated parameter, ${\sigma }_l$, reflecting the noise level at the l-th PAM amplitude. Through this parameter, we can assess the quality of the channel. Therefore, we can directly use ${\sigma }_l$ to adjust the transmitted signal, reducing the probability of higher levels of ${\sigma }_l$ occurring. For the l-th level in PAM-L, its PDF should satisfy

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \begin{array}{c}\hfill D\left(l\right)=\frac{e^{-{\sigma }_l^2{\left|l\right|}^2}}{{\sum }_{l^{\prime }\in S}{e}^{-{\sigma }_l^2{\left|l^{\prime }\right|}^2}},\end{array}\end{array}\end{array}$$

(5)

The noise level parameter ${\sigma }_l$ can be extracted from the results of the GMM fitting. Additionally, $D\left(l\right)$ needs to satisfy another condition:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \sum _{l=1}^{L}D\left(l\right)=1,0\le D\left(l\right)\le 1,\end{array}\end{array}$$

(6)

which means the total probability across all levels is equal to 1. With GMM-based PS techniques, the system can be optimized by appropriately selecting the parameter ${\sigma }_l$ of the Gaussian distribution according to the channel conditions and transmission requirements. A lower value ${\sigma }_l$ increases the occurrence probability of more likely symbol points, thus improving the system capacity. Conversely, a smaller ${\sigma }_l$ value makes specific amplitude levels more likely to be used, improving transmission reliability.

3.2. Enhanced Transmission Systems with Joint Transmitter-Receiver Optimization

After proposing a new AMCC interference model in [24], a joint demodulation receiver based on this new interference model is also introduced. The GMM-based PS technique can be used to optimize the transmitter and receiver together in this system, as illustrated in Figure 4. At the transmitter, the module responsible for generating PAM signals is replaced with a PS-PAM signal generation module. $S_{PON}\left(x\right)$ and $S_{AMCC}\left(x\right)$ are combined using the method shown in Figure 1 through an MZM modulator to produce the transmitted signal $S\left(x\right)$. $S\left(x\right)$ undergoes the channel treatment to become $R\left(x\right)$ and enters the receiver. The basic structure of the receiver is based on a GMM-HMM joint demodulation receiver, consisting of a parameter estimation workflow and joint demodulation workflow. The training sequence from $R\left(x\right)$ enters the parameter estimation workflow. The parameter estimation module first fits the signal model using the GMM method to obtain the model parameters for the received signal. The probability distribution parameters of the signal will be entered into the demodulation process as the $\mathbf{P}$ matrix mentioned in Equation (7) for PON signal demodulation. The $\sigma$ parameter in the GMM parameters is used to analyze the channel, and according to the method mentioned in Equation (5), the amplitude distribution of the PAM signal is calculated. The parameters obtained, $D\left(l\right)$, are then passed to both the HMM transition matrix estimation module in the receiver and the PS-PAM mapping adjustment module in the transmitter. Initially, the transmitter still generated PON signals, $S_{PON}\left(x\right)$, with an equiprobable distribution. After obtaining the new parameters, $D\left(l\right)$, the PS-PAM mapping adjustment module in the transmitter adjusts the PAM signal using the CCDM technique proposed in [26] to generate a new PON signal. After receiving the $D\left(l\right)$ parameters, the HMM transition matrix estimation module in the receiver calculates the new transition matrix parameters, $\mathbf{H}$, and inputs them into the joint demodulation process. The joint demodulation of PON-AMCC signals using the HMM method can be defined by the following formula [33,34]:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \begin{array}{cc}\hfill \lambda & =\left(\Pi ,\mathbf{H},\mathbf{P}\right)\hfill \\ \hfill & =\left({\left[{\pi }_m\right]}_{N\ast 1},{\left[h_{mn}\right]}_{N\ast N},{\left[p_m\left(x\right)\right]}_{N\ast 1}\right),\hfill \end{array}\end{array}\end{array}$$

(7)

where $\pi$ is the initial probability distribution, $\mathbf{H}$ is the state transition probability matrix, and $\mathbf{P}$ is the observation probability matrix. In our interference model, the observation probability distribution is obtained by the GMM fitting process; therefore, $p_m\left(x\right)$ in the P matrix should be expressed as Equation (2). The transition matrix $\mathbf{H}$ can be described as

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \mathbf{H}={\left[h_{mn}\right]}_{N\ast N},1\le m,n\le N.\end{array}\end{array}$$

(8)

Considering the randomness of the signals, the transition probabilities are the same when $m,n$ belongs to different groups. Meanwhile, the probability of occurrence for L levels in a single group is equal. Thus, the jump probability for each level needs to be further divided by L. On the other hand, when m and n belong to the same group, the transition probabilities are also equal. Together with the restriction that ${\sum }_n{h}_{mn}=1$, we can obtain

$$\begin{array}{c}\hfill \begin{array}{c}\hfill h_{mn}=\left\{\begin{array}{c}D\left(l\right)\times \frac{1}{K}\times \frac{R_{AMCC}}{R_{PON}},m\not\equiv n\left(modK\right)\hfill \\ D\left(l\right)\times (1-\frac{R_{AMCC}}{R_{PON}}\times \frac{K-1}{K}),m\equiv n\left(modK\right).\hfill \end{array}\right.\end{array}\end{array}$$

(9)

When the transmitter uses an equiprobable distribution of L-order PAM signals, the coefficient $D\left(l\right)$ can be represented as the constant $1/L$. However, when the transmitted signal undergoes the GMM-based PS method, the probabilities of occurrence for each level are no longer equal. Consequently, the coefficients, $D\left(l\right)$, in the transition parameters, $h_{mn}$, used in the HMM demodulation module of the joint demodulation receiver also need to be adjusted accordingly. As mentioned above, these coefficients are updated to reflect the actual probabilities of level occurrence, which are derived from the GMM fitting step of the receiver and are consistent with the probability parameters used in the transmitter. After updating the transition matrix, $\mathbf{H}$, in the HMM module of the receiver, we proceed with the joint demodulation process for PON and AMCC, where the PON Viterbi processing uses the demodulation method described by Equation (7) to demodulate the received signal, resulting in the output signal $R_{PON}\left(x\right)$. The transition parameters during the Viterbi process are input into the AMCC post-processing module for demodulating the AMCC signal $R_{AMCC}\left(x\right)$.

In practical transmission systems, the channel conditions may vary over time, and using fixed parameters may not meet the requirements of all scenarios. The parameters for probability shaping at the transmitter can be obtained through the GMM fitting step of the joint demodulation receiver. If we continuously feed back these parameters to the transmitter in real time, the transmitter can adjust the probability shaping parameters based on the current channel conditions. After adjusting the parameters of the transmitted signal, the transmitter can notify the receiver through the AMCC to synchronize and update the demodulation parameters.

4. Experimental Results and Discussion

4.1. Experimental System

In order to validate the signal adjustment method proposed in this article, we constructed an experimental platform, as illustrated in Figure 5. In this platform, we transmitted a 50G-PON signal mixed with an AMCC. The received signals were subjected to offline processing using the joint demodulation method at the receiver.

On the transmitter side, the shaped and bandwidth-limited electrical AMCC data are sent to the DC bias port of the modulator via a signal generator (Rigol DG992; 250 MSps), and a pre-generated 50 Gbps PAM4 PON signal is sent from an arbitrary waveform generator (AWG, Keysight M8195A, 65 GSps) to the RF port of the modulator. These two signals are combined in a LiNbO3 intensity modulator (Ixblue MXAN-LN series). An electrical amplifier (EA; Ixblue DR-AN-40-MO) is used in the RF port as a signal driver. The optical power into the fiber is set to 5 dBm. The combined signal is transmitted over a 10 km standard single-mode fiber (SSMF). Simultaneously, a variable optical attenuator (VOA) was employed to control the received optical power (ROP) at the receiver. This was carried out to vary line attenuations to assess the system’s performance under different attenuations. Link attenuation will not affect the signal distribution characteristics, and the received signals are consistent with our proposed model under different link losses. At the receiver, the optical signal is detected by the photodetector (Thorlabs DXM30BF), and the electrical signal is connected to a real-time oscilloscope (Keysight UXR0334A, 33 GHz bandwidth). The channel is sampled at 128 GSps to capture the information of the PON signal. The captured signal undergoes processing through the digital signal processing flow, as illustrated in Figure 4, to derive the interference model and distribution parameters. These parameters are essential for the joint demodulation process in the receiver. Simultaneously, they play a pivotal role in tuning and optimizing the transmitted signals. This dual functionality aims to improve the system’s overall transmission performance.

4.2. Distribution and Error Symbol

In order to validate the relationship between error symbols and signal distribution, we first analyze the case of an equal probability distribution for the received signals in the experimental system. When the AMCC modulation index in the experimental system was set to 10% and the received signal power was −10 dBm, we performed a statistical analysis of all demodulation errors in the received PON signals. The results of this analysis are shown in Figure 6, where the horizontal axis represents the PAM levels of the received signals, and the vertical axis represents the number of error symbols corresponding to each level.

Due to the presence of the AMCC, the original PON signal, which was PAM4-modulated, becomes a Gaussian mixture distribution with eight peaks at the receiver, where every two peaks correspond to a PON signal with the same amplitude value. From the graph, it is evident that the error rate for the PAM4 signal levels of −1 and +1 is significantly lower than for the PAM signal levels of −3 and +3. This observation aligns with the signal-to-noise ratio relationship depicted in Figure 3 based on the Gaussian mixture model. For this channel, we calculated the probability of the PON transmission signal as [0.33, 0.17, 0.17, 0.33] using the proposed method. Because the symbol probability of PAM4 has been modified, resulting in a decrease in the entropy of the non-uniform signal, in order to maintain the same effective bit rate for PON signals, we adjusted the signal rate after using probability shaping. According to the probability distribution, we adjusted the transmission rate of the PON signal to 26 GBaud while maintaining an effective data rate of 50 Gbps.

4.3. System Performance

After optimizing the transmission signal using probability shaping techniques, we performed tests on the transmission performance of PON signals. By using the new method, Figure 7a illustrates the relationship curve between the bit error rate (BER) and the ROP of the PON signal with the different modulation indices of the AMCC super-imposed. The points in the figure represent the raw data obtained from the experiments, while the curves are the result of fitting these data points. As the modulation index of AMCC increases, the interference received by the PON also increases. Therefore, the performance of the PON signal is best when there is no interference from AMCC. In the ITU-T standard, it is mentioned that 50G-PON introduces FEC to meet higher transmission requirements, including both Reed–Solomon (RS) and low-density parity-check (LDPC) codes [20]. When RS coding is used, the PHY layer needs to meet a BER of ${10}^{-3}$ to achieve error-free transmission by FEC, and this requirement can be reduced to ${10}^{-2}$ when LDPC is used. Under this condition, error-free transmission can be realized by using FEC if the proposed method can keep the BER of the PON signal below this threshold. Therefore, we use $3.8\times {10}^{-3}$ (7% hard-decision forward error correction, HD-FEC), which is the BER threshold required for RS coding, as a comparison criterion in the following discussion. This is shown (black curve) in the figure. Compared to the BER curve of the joint demodulation receiver under equal probability transmitted signals, as presented in [24], the new method proposed in this article results in a reduction in the BER of the PON signal. Additionally, there is a slight increase in the power budget within the 1 dB penalty range. Simultaneously, under these conditions, the BER curve of the AMCC is shown in Figure 7b. The greater modulation depth of the AMCC translates to stronger signal strength and a higher corresponding signal-to-noise ratio (SNR), leading to improved performance. A modulation depth of 5% represents the minimum for the AMCC, thus resulting in the poorest performance. All the modulation indices of the AMCC meet the BER threshold requirements for HD-FEC, indicating that error-free transmission can be achieved.

The curves shown in Figure 8 were obtained under the conditions of an AMCC signal rate of 10 Mbps and a modulation index of 10%. Similar to Figure 7, the points in the figure represent the raw data obtained from the experiments, while the curves are obtained through fitting. The blue curve represents the relationship between the BER of the PON signal, demodulated by the joint demodulation receiver, and the ROP when transmitting signals with an equal probability distribution. The red curve illustrates the BER vs. ROP when the GMM-based PS technique proposed in this article is applied to adjust the transmitted signals, with the transition matrix in the joint demodulation receiver corrected. It can be observed that by employing the GMM-based PS technique for joint transmitter-receiver optimization, the power range of the received PON signal can be increased by nearly 1 dB at an error rate of ${10}^{-3}$.

5. Conclusions

In this paper, we propose a method to enhance PON signal transmission performance through an analysis of the model of received signals in the PON-AMCC transmission system and the distribution characteristics of transmission error symbols. Firstly, we conduct GMM fitting regarding the received signals and analyze their distribution characteristics. Subsequently, we analyze the relationship between the error symbols in the received signals and the GMM distribution, which allows us to establish the objectives for adjusting the probabilities of the transmitted signals. Following these objectives, we employed probability-shaping techniques to modify the signal transmission probability. Due to the changes in the transmitted signals within the joint demodulation receiver, it is necessary to update the HMM transition matrix based on the modified transmission signal distribution characteristics. Finally, we conducted experiments using the proposed method in a 50G-PON experimental system with PAM4 modulation. The experimental results indicate that this method can reduce the error rate of PON signals at an equivalent data rate. This method can further mitigate the impact of AMCCs on PON, thus facilitating the efficient deployment of WDM-PON systems with AMCC. This approach also enables dynamic optimization tailored to the changing channel conditions, facilitating the rapid deployment of PON transmission systems in different channels.