Orbital Angular Momentum Multiplexed Free-Space Optical Communication Systems Based on Coded Modulation

In this paper, we experimentally investigate the turbulence mitigation methods in free-space optical communication systems based on orbital angular momentum (OAM) multiplexing. To study the outdoor atmospheric turbulence environment, we use an indoor turbulence emulator. Adaptive optics, channel coding, Huffman coding combined with low-density parity-check (LDPC) coding, and spatial offset are used for turbulence mitigation; while OAM multiplexing and wavelength-division multiplexing (WDM) are applied to boost channel capacity.


Introduction
Optical communication systems are usually deployed over fiber-optic links [1][2][3], free-space optical (FSO) links [4][5][6], or hybrid FSO-fiber links [7].Recent advances in photonics integrated circuits (PIC) have been greatly pushing forward the worldwide application of optical communications [8][9][10].Although they have enabled a capacity-approaching communication [11][12][13], fiber-optic links may be too fragile or costly to be deployed in some environments, e.g., seismic belts.As a result, FSO links are more favorable due to their easy and fast communication link reconstruction.Although the channel loss of FSO links is not stable, and typically higher than that of fiber-optic links, FSO communication systems provide free-scalable channels for spatial mode division (SDM) multiplexing, e.g., orbital angular momentum (OAM) multiplexing [14][15][16][17].Despite their free-scalable characteristics, spatially multiplexed modes hardly preserve their orthogonality when transmitting over the atmospheric FSO links, resulting in dynamic inter-mode crosstalk [18][19][20].
There are several ways to mitigate the inter-mode crosstalk, including wavefront sensor (WFS) or wavefront sensorless based adaptive optics (AO) systems [21][22][23], digital multi-input multi-output (MIMO) equalization [24][25][26], and advanced forward error correction (FEC) based channel coding techniques [27][28][29].AO systems are usually implemented in satellite communications to mitigate wavefront distortion, while their commercial application in near-Earth FSO links is greatly limited by the expensive WFS and deformable mirror (DM).In some instances, a wavefront sensorless AO is used as a trade-off between cost and reliability.MIMO equalization is also often used to relieve inter-mode crosstalk among the multiplexed spatial modes.It is, however, preferentially used in the FSO links affected by weak-to-medium atmospheric turbulence.When spatial modes are transmitted in a strong atmospheric turbulence environment, the unwanted inter-mode crosstalk is not only limited to adjacent spatial modes, but also spread widely across other spatial modes.As a result, the use of more mode detectors is expected at the receiver side to capture the distorted signals in correlated modes, followed by more computationally complex MIMO equalization.If the quantity of mode detector is not sufficient, MIMO equalization may fail to work due to data loss.FEC based channel coding techniques have been fully developed over decades, and extensively used in error-free digital fiber-optic communication systems.However, FEC coding-only solution can't guarantee a reliable data transmission over long reach FSO links, especially in the strong atmospheric turbulence environment.
In this paper, we discuss the high-speed OAM multiplexed FSO communication systems, enabled by AO, low-density parity-check (LDPC) coding, spatial offset (SO), and joint Huffman and LDPC coding.First, we experimentally study an AO assisted, LDPC coded, OAM-based FSO communication system.Briefly, four OAM multiplexed mode channels that in total carry 500 Gbps quadrature phase shift keying (QPSK) signals are transmitted over wavelength-division multiplexed (WDM) channels with 50 GHz spacing.The turbulence-induced inter-mode crosstalk is compensated by a wavefront sensorless AO setup.Subsequently, error-free communication can be achieved with a strong LDPC coding scheme.The minimum coding gain of 3.9 dB is achieved at BER = 2 × 10 −2 for OAM states ±2 and ±6.
Second, we present another inter-mode crosstalk mitigation solution in an atmospheric turbulence limited OAM multiplexed FSO links.The OAM mode crosstalk is first relieved by optical spatial mode offset, and then further resolved by coded modulation technology.Huffman coding and optimal constellation design techniques are applied to generate the quadrature amplitude modulation (QAM) formats, i.e., 5/9-QAM formats.Meanwhile, the GF(5) based LDPC coding is implemented for 5-QAM symbol sequences, and GF(32) based LDPC coding is implemented for 9-QAM symbol sequences [30,31].Unlike the classical OAM multiplexed FSO links, where all spatial mode channels are centrally aligned, binary FEC coding is used for error correction.Furthermore, uniformly distributed M-QAM formats are used for data modulation, e.g., QPSK and 8-QAM.The proposed two-stage OAM mode crosstalk mitigation solution can largely enhance the communication reliability in atmospheric FSO links.

Experimental Setup
Figure 1 shows the experimental setup for an AO enabled FSO communication system.The five continuous wave (CW) laser beams are generated with the inter-channel spacing of 50 GHz (1549.32-1550.92nm).The wavelength channels are multiplexed together and used as the optical input of an I/Q modulator.The pseudorandom binary sequence (PRBS) signals are encoded using a binary LDPC code with the code rate of 0.8.The data streams pass to the arbitrary waveform generator (AWG) and drive the I/Q modulator to generate a 15.6 GBaud optical QPSK signal.An optical interleaver (IL) is applied to separate odd and even channel signals, which are then decorrelated by 350 symbols and recombined together.The resulting WDM QPSK optical signals with the decorrelated adjacent wavelength channels are boosted by an Erbium-doped fiber amplifier (EDFA), followed by an optical tunable filter (OTF) to suppress amplified spontaneous emission (ASE) noise.The optical signals are separated by a coupler, and one path is later decorrelated before recombination.The optical Gaussian modes are collimated by fixed fiber optic collimator and converted to OAM modes (OAM states ±2 and ±6) by using a high-resolution spatial light modulator (SLM).The resulting OAM modes are then centrally aligned by a beam splitter (BS).Another 1548.9 nm Gaussian probe beam is used to assist AO compensation.The Gaussian probe beam is expanded using a beam expander (BE) to reach a diameter larger than the widest OAM beams (OAM states ±6) generated in this experiment.At this stage, the probe beam is also centrally aligned with the data-carrying OAM modes.The expanding telescope is applied to adjust the diameters of the collimated OAM beam and Gaussian probe beam, which are then sent to the turbulence emulator.The SLM2 and SLM3 are continuously and randomly updating phase patterns to be modelled on the dynamic atmospheric turbulence environment [22].The accuracy of the turbulence model can be validated in terms of onaxis Gamma-Gamma intensity distribution and intensity correlation function.The atmospheric turbulence emulator used here is designed according to the Rytov variance [32] of   2 = 2.
The size of the distorted OAM and probe beams are decreased by a compressing telescope.The distorted beams then pass to the DM for wavefront correction.The distorted Gaussian probe beam functions as a stimulus in this wavefront sensorless AO setup, with the assumption that the probe beam and OAM beams are affected by the similar wavefront distortion.Partial optical beams are segregated via a BS and collected by a single-mode fiber (SMF) patch cable.In this experiment, the OTF3 with a central wavelength of 1548.9 nm is implemented to only capture the Gaussian probe beam, followed by a photodiode (PD) for power monitoring.The detected analog voltage is digitized by an analog-to-digital converter (ADC) to update the DM pixels based on stochastic parallel gradient descent algorithm [33].It is noteworthy that the performance of the FSO transmission system is dominated by linear mode crosstalk.within comparison to fiber-optic transmission systems, FSO links will not have notable nonlinear effects in principle.The AO used in our experiment will not bring nonlinear distortions, since the processing time of the AO is far longer than the data rate.
Following AO compensation, the less distorted OAM modes are detected by SLM4.This is used to convert one OAM mode back to the Gaussian-like mode, which is then collected by another SMF patch cable.The collected optical signals are pre-amplified by EDFA2, followed by a variable optical attenuator (VOA) for optical power tuning.Additional ASE noise is generated and adjusted via a sub-system configured by EDFA3, EDFA4, and VOA2.Such ASE noise is added to the optical signal using an optical coupler, after which an OTF with the central wavelength of 1550.12 nm is applied to single out the corresponding optical wavelength channel.In the coherent receiver, the local oscillator (LO) light and the optical signal are mixed in an optical 90 o hybrid, detected by two PDs, and digitized by a real-time oscilloscope.After the off-line digital signal processing (DSP) signal recovery, the sum-product algorithm is used in the LDPC decoding procedure with a maximum of 50 iterations [34].The expanding telescope is applied to adjust the diameters of the collimated OAM beam and Gaussian probe beam, which are then sent to the turbulence emulator.The SLM2 and SLM3 are continuously and randomly updating phase patterns to be modelled on the dynamic atmospheric turbulence environment [22].The accuracy of the turbulence model can be validated in terms of on-axis Gamma-Gamma intensity distribution and intensity correlation function.The atmospheric turbulence emulator used here is designed according to the Rytov variance [32] of σ 2 R = 2.The size of the distorted OAM and probe beams are decreased by a compressing telescope.The distorted beams then pass to the DM for wavefront correction.The distorted Gaussian probe beam functions as a stimulus in this wavefront sensorless AO setup, with the assumption that the probe beam and OAM beams are affected by the similar wavefront distortion.Partial optical beams are segregated via a BS and collected by a single-mode fiber (SMF) patch cable.In this experiment, the OTF3 with a central wavelength of 1548.9 nm is implemented to only capture the Gaussian probe beam, followed by a photodiode (PD) for power monitoring.The detected analog voltage is digitized by an analog-to-digital converter (ADC) to update the DM pixels based on stochastic parallel gradient descent algorithm [33].It is noteworthy that the performance of the FSO transmission system is dominated by linear mode crosstalk.within comparison to fiber-optic transmission systems, FSO links will not have notable nonlinear effects in principle.The AO used in our experiment will not bring nonlinear distortions, since the processing time of the AO is far longer than the data rate.
Following AO compensation, the less distorted OAM modes are detected by SLM4.This is used to convert one OAM mode back to the Gaussian-like mode, which is then collected by another SMF patch cable.The collected optical signals are pre-amplified by EDFA2, followed by a variable optical attenuator (VOA) for optical power tuning.Additional ASE noise is generated and adjusted via a sub-system configured by EDFA3, EDFA4, and VOA2.Such ASE noise is added to the optical signal using an optical coupler, after which an OTF with the central wavelength of 1550.12 nm is applied to single out the corresponding optical wavelength channel.In the coherent receiver, the local oscillator (LO) light and the optical signal are mixed in an optical 90 • hybrid, detected by two PDs, and digitized by a real-time oscilloscope.After the off-line digital signal processing (DSP) signal recovery, the sum-product algorithm is used in the LDPC decoding procedure with a maximum of 50 iterations [34].

Results and Analysis
We begin by investigating the atmospheric turbulence-induced mode crosstalk and the merits of AO assisted wavefront correction.The power ratio between the target OAM modes (OAM states ±2, ±6) and their adjacent modes are used here as a metric to evaluate the effect of mode crosstalk.As illustrated in Figure 2a, the power of target OAM mode measured was similar to the adjacent OAM modes, indicating that the data originating from mode crosstalk will severely interfere with the desirable data after mode detection.Data in Figure 2b shows that the average extinction ratio (ER) after AO assisted wavefront correction reaches a 6-dB improvement.The blue bars in Figure 2a and navy bars in Figure 2b represent the desirable transmitted OAM modes, and the green bars represents the unwanted OAM modes caused by OAM mode crosstalk.

Results and Analysis
We begin by investigating the atmospheric turbulence-induced mode crosstalk and the merits of AO assisted wavefront correction.The power ratio between the target OAM modes (OAM states ±2, ±6) and their adjacent modes are used here as a metric to evaluate the effect of mode crosstalk.As illustrated in Figure 2a, the power of target OAM mode measured was similar to the adjacent OAM modes, indicating that the data originating from mode crosstalk will severely interfere with the desirable data after mode detection.Data in Figure 2b shows that the average extinction ratio (ER) after AO assisted wavefront correction reaches a 6-dB improvement.The blue bars in Figure 2a  Figure 3a shows the average bit-error rate (BER) vs. OSNR performance with or without atmospheric turbulence effects.The data clearly demonstrates that the BER curves do not drop quickly, even with the increasing OSNR values.It is caused by the unperfect mode generation and detection patterns, which will also introduce unwanted inter-mode crosstalk effects.Note that the worse BER performance of OAM states ±6 compared to OAM state ±2 is due to high-order OAM mode sensitivity to the boundary effect.Figure 3a shows the average bit-error rate (BER) vs. OSNR performance with or without atmospheric turbulence effects.The data clearly demonstrates that the BER curves do not drop quickly, even with the increasing OSNR values.It is caused by the unperfect mode generation and detection patterns, which will also introduce unwanted inter-mode crosstalk effects.Note that the worse BER performance of OAM states ±6 compared to OAM state ±2 is due to high-order OAM mode sensitivity to the boundary effect.

Results and Analysis
We begin by investigating the atmospheric turbulence-induced mode crosstalk and the merits of AO assisted wavefront correction.The power ratio between the target OAM modes (OAM states ±2, ±6) and their adjacent modes are used here as a metric to evaluate the effect of mode crosstalk.As illustrated in Figure 2a, the power of target OAM mode measured was similar to the adjacent OAM modes, indicating that the data originating from mode crosstalk will severely interfere with the desirable data after mode detection.Data in Figure 2b shows that the average extinction ratio (ER) after AO assisted wavefront correction reaches a 6-dB improvement.The blue bars in Figure 2a  Figure 3a shows the average bit-error rate (BER) vs. OSNR performance with or without atmospheric turbulence effects.The data clearly demonstrates that the BER curves do not drop quickly, even with the increasing OSNR values.It is caused by the unperfect mode generation and detection patterns, which will also introduce unwanted inter-mode crosstalk effects.Note that the worse BER performance of OAM states ±6 compared to OAM state ±2 is due to high-order OAM mode sensitivity to the boundary effect.Figure 3b shows that distinct average OSNR gain can be reached after AO assisted wavefront correction and LDPC coding.We observe that the BER curves after AO assisted wavefront correction can be lower than 0.04, which is the error correction threshold of the used LDPC code.Furthermore, the performance of the AO assisted wavefront correction on OAM states ±2 are better than that of OAM states ±6.This is caused not only by the boundary effect brought by the limited sizes of the used optical components, but also the features of the used probe beam.This beam is able to fully cover the small-size OAM modes (OAM states ±2) after the turbulent FSO transmission, rather than the large-size OAM modes (OAM states ±6).After the AO correction is performed, LDPC coding/decoding can then be applied to efficiently eliminate the post-FEC error floor.Figure 3b shows that the BER curves of all OAM modes can drop quickly as long as the OSNR is higher than 8 dB.More specifically, when the BER is 2 × 10 −2 the coding gains of 3.9, 4.1, 5.2, and 5 dB are reached for OAM states 2, −2, 6, and −6, respectively.
In this study, we implement pre-compensation algorithms at the transmitter side and post-equalization algorithms at the receiver side to minimize the implementation penalty.Turning off the turbulence emulator by sending blank phase patterns to the SLMs in the turbulence emulator does not eliminate all OAM mode distortions caused by the imperfect SLM screens, especially at the OAM mode generation and detection steps.

Experimental Setup
Figure 4 shows the experimental setup for joint Huffman and LDPC coding enabled FSO communication system.In the transmitter, a 1550 nm CW light is generated as the optical carrier, and passes to an optical I/Q modulator.The PRBS signals are coded by Huffman procedure to the symbol sequences with alphabet sizes of 5 and 9 respectively [30]; or uniformly mapped to symbol sequences with alphabet sizes of 4 and 8, respectively.Then the GF(5), GF(3 2 ) based nonbinary LDPC encoding is used, followed by mapping procedures from the coded sequences to 5-QAM and 9-QAM signals, respectively.Classical binary LDPC encoding is implemented for QPSK (or 4-QAM) and 8-QAM sequences.The used Huffman trees and the 5/9-QAM formats with corresponding bit labeling are provided (Figure 4(a1,a2,b1,b2)).When 12.5 G Baud electronic signals are generated, they drive the I/Q modulator.The optical signals are boosted by EDFA1, and filtered with an OTF.Then optical signals are separated by an optical coupler, decorrelated, and converted to OAM states 2 and −6 by SLM1.The formed OAM modes are combined and centrally aligned by BS1.BS2 is used to separate the multiplexed OAM modes, and re-combined by the BS3.The optical signals in one optical path bounce off a mirror one time to generate the opposite OAM modes, i.e., OAM states −2 and 6.The optical signals in the other path bounce off four times to keep the original mode states and decorrelate the carried optical signals.The desired SO between the two optical paths can be achieved by adjusting the BS3 position.In this setup, the limited SLM screen size restricts the SO freedom.To reduce the side effect-induced inter-mode crosstalk and mode power loss, OAM states 2 and −6 are centrally aligned and transmitted in one optical path, while OAM states −2 and 6 are combined and launched onto another optical path.
The offset OAM beams are expanded by a BE, and then distorted in the designed turbulence emulator.The current turbulence emulator is designed according to the Rytov variance of 0.5.The distorted OAM beams are captured by a compressing telescope, demultiplexed by SLM4, and back-convert the target OAM mode to the Gaussian mode.The Gaussian beam is then coupled from free space into a fiber cable and pre-amplified by EDFA2.The ASE noise is loaded onto the signal in the 3-dB coupler, and later the out-of-band noise is removed by the OTF2.The optical signal is detected by a coherent receiver, and equalized by DSP signal recovery.The symbol log-likelihood ratio (LLR) estimation is executed before the GF(2)/GF(5)/GF(3 2 ) based LDPC decoding procedures.Ultimately, BER values are calculated after LDPC decoding to determine system performance.

Results and Analysis
The signal-to-crosstalk ratio (SCTR) gains achieved by SO are presented in Figure 5.The SCTR is measured after the OAM mode detection.It is defined as the power ratio of the desirable OAM mode and neighboring OAM modes, in the scenario where the desired OAM mode is generated exclusively at the transmitter.The insets in Figure 5a1,a2 show the measured SCTR gains.The SCTR improvement of >1.6 dB is achieved for OAM state 2; and the SCTR gain of >1 dB is available for OAM state −6.The data shows that the best SO is at 6 mm and 5 mm for OAM states 2, and −6, respectively.For simplicity, the SO will be set to 5 mm for all following cases in this paper.In Figure 6, we analyze the effects of the nonuniform signaling.The data represents BER performance in the atmospheric turbulence-free environment.Figure 6a,b shows that the pre-FEC performance of 5-QAM is worse than that of the QPSK due to the implementation and DSP penalties.These data also indicate that the pre-FEC performance of 9-QAM can outperform the 8-QAM.However, the post-FEC OSNR penalties between the 5-QAM and QPSK are measured to be <0.

Results and Analysis
The signal-to-crosstalk ratio (SCTR) gains achieved by SO are presented in Figure 5.The SCTR is measured after the OAM mode detection.It is defined as the power ratio of the desirable OAM mode and neighboring OAM modes, in the scenario where the desired OAM mode is generated exclusively at the transmitter.The insets in Figure 5(a1,a2) show the measured SCTR gains.The SCTR improvement of >1.6 dB is achieved for OAM state 2; and the SCTR gain of >1 dB is available for OAM state −6.The data shows that the best SO is at 6 mm and 5 mm for OAM states 2, and −6, respectively.For simplicity, the SO will be set to 5 mm for all following cases in this paper.

Results and Analysis
The signal-to-crosstalk ratio (SCTR) gains achieved by SO are presented in Figure 5.The SCTR is measured after the OAM mode detection.It is defined as the power ratio of the desirable OAM mode and neighboring OAM modes, in the scenario where the desired OAM mode is generated exclusively at the transmitter.The insets in Figure 5a1,a2 show the measured SCTR gains.The SCTR improvement of >1.6 dB is achieved for OAM state 2; and the SCTR gain of >1 dB is available for OAM state −6.The data shows that the best SO is at 6 mm and 5 mm for OAM states 2, and −6, respectively.For simplicity, the SO will be set to 5 mm for all following cases in this paper.In Figure 6, we analyze the effects of the nonuniform signaling.The data represents BER performance in the atmospheric turbulence-free environment.Figure 6a,b shows that the pre-FEC performance of 5-QAM is worse than that of the QPSK due to the implementation and DSP penalties.These data also indicate that the pre-FEC performance of 9-QAM can outperform the 8-QAM.However, the post-FEC OSNR penalties between the 5-QAM and QPSK are measured to be <0.In Figure 6, we analyze the effects of the nonuniform signaling.The data represents BER performance in the atmospheric turbulence-free environment.Figure 6a,b shows that the pre-FEC performance of 5-QAM is worse than that of the QPSK due to the implementation and DSP penalties.These data also indicate that the pre-FEC performance of 9-QAM can outperform the 8-QAM.However, the post-FEC OSNR penalties between the 5-QAM and QPSK are measured to be <0.3 dB and 0.2 dB, in the respective OAM states 2 and −6, when the BER is 10 The BER performance affected by atmospheric turbulence is shown in Figure 7.The data indicate that the pre-FEC BER performance gap between QPSK and 5-QAM shrinks in an atmospheric turbulence limited environment; while the pre-FEC BER performance of 9-QAM is better than that that of the 8-QAM.Figure 7a shows that, if OAM state 2 is under testing, the average OSNR gains of >1.6 dB and 5.6 dB are reached by 5-QAM and 9-QAM, respectively, when the post-FEC BER is 10 −4 , compare to coded QPSK and 8-QAM.In addition, if OAM state −6 is detected, as depicted in Figure 7b, the coding gains of >1.1 dB and 5.4 dB can be obtained when the post-FEC BER is 10 −4 respectively by comparing 5-QAM with QPSK, and 9-QAM with 8-QAM.It is noteworthy that we only measure the performances of OAM states 2 and −6, this is due to the symmetry between OAM modes with positive and negative states.In other words, OAM state 2 will have a very similar performance as OAM state −2; while OAM state 6 will also have a similar performance as OAM state −6.In order to clarify the performance improvements achieved by the coded modulation, the average OSNR requirements at BER = 10 −4 for 4/5/8/9-QAM formats are shown in Table 1.These data are under conditions where OAM states 2 and 6 are detected, and with/without atmospheric turbulence.The OSNR penalties brought by atmospheric turbulence for OAM state 2, are concluded The BER performance affected by atmospheric turbulence is shown in Figure 7.The data indicate that the pre-FEC BER performance gap between QPSK and 5-QAM shrinks in an atmospheric turbulence limited environment; while the pre-FEC BER performance of 9-QAM is better than that that of the 8-QAM.Figure 7a shows that, if OAM state 2 is under testing, the average OSNR gains of >1.6 dB and 5.6 dB are reached by 5-QAM and 9-QAM, respectively, when the post-FEC BER is 10 −4 , compare to coded QPSK and 8-QAM.In addition, if OAM state −6 is detected, as depicted in Figure 7b, the coding gains of >1.1 dB and 5.4 dB can be obtained when the post-FEC BER is 10 −4 respectively by comparing 5-QAM with QPSK, and 9-QAM with 8-QAM.It is noteworthy that we only measure the performances of OAM states 2 and −6, this is due to the symmetry between OAM modes with positive and negative states.In other words, OAM state 2 will have a very similar performance as OAM state −2; while OAM state 6 will also have a similar performance as OAM state −6.The BER performance affected by atmospheric turbulence is shown in Figure 7.The data indicate that the pre-FEC BER performance gap between QPSK and 5-QAM shrinks in an atmospheric turbulence limited environment; while the pre-FEC BER performance of 9-QAM is better than that that of the 8-QAM.Figure 7a shows that, if OAM state 2 is under testing, the average OSNR gains of >1.6 dB and 5.6 dB are reached by 5-QAM and 9-QAM, respectively, when the post-FEC BER is 10 −4 , compare to coded QPSK and 8-QAM.In addition, if OAM state −6 is detected, as depicted in Figure 7b, the coding gains of >1.1 dB and 5.4 dB can be obtained when the post-FEC BER is 10 −4 respectively by comparing 5-QAM with QPSK, and 9-QAM with 8-QAM.It is noteworthy that we only measure the performances of OAM states 2 and −6, this is due to the symmetry between OAM modes with positive and negative states.In other words, OAM state 2 will have a very similar performance as OAM state −2; while OAM state 6 will also have a similar performance as OAM state −6.In order to clarify the performance improvements achieved by the coded modulation, the average OSNR requirements at BER = 10 −4 for 4/5/8/9-QAM formats are shown in Table 1.These data are under conditions where OAM states 2 and 6 are detected, and with/without atmospheric turbulence.The OSNR penalties brought by atmospheric turbulence for OAM state 2, are concluded In order to clarify the performance improvements achieved by the coded modulation, the average OSNR requirements at BER = 10 −4 for 4/5/8/9-QAM formats are shown in Table 1.These data are under conditions where OAM states 2 and 6 are detected, and with/without atmospheric turbulence.The OSNR penalties brought by atmospheric turbulence for OAM state 2, are concluded as 3.3 dB, 1.4 dB, 5.7 dB, and 2.8 dB for 4/5/8/9-QAM, respectively.Similarly, the mode-crosstalk penalties for OAM state −6, are measured to be 3.1 dB, 1.8 dB, 5 dB, and 2.8 dB for 4/5/8/9-QAM, respectively.Thereafter, the nonuniform 5/9-QAM schemes have a higher inter-mode crosstalk tolerance over uniform QPSK and 8-QAM.

Concluding Remarks
We have investigated the high-speed OAM multiplexed FSO communication system, enabled by AO based wavefront correction and LDPC coding.The inter-mode crosstalk was first compensated by the wavefront sensorless AO setup, and the residual mode crosstalk induced data interference was later solved by sufficiently strong LDPC coding.
We also presented a crosstalk-resistance solution in an OAM multiplexed FSO link based on SO and coded modulation.More than 1 dB SCTR improvement has been shown for the used OAM modes.Moreover, the 5/9-QAM schemes exhibit a better crosstalk tolerance than regular QPSK and 8QAM schemes.
Sometimes, more advanced modulation formats are used to further increase channel capacity.When high-order modulation formats are used, FSO communication systems are more sensitive to atmospheric turbulence.Some turbulence compensation solutions may not work well.Stronger FEC coding may be used to protect systems reliability.Optical domain turbulence compensation solutions, like adaptive optics, are suggested for implementation before mode detection.Nonuniform coded modulation may still work, but should not bring much performance improvement like 5/9QAM formats.MIMO processing solutions will face a more severe challenge, due to the increasing computation complexity and the reduced robustness of the MIMO equalization.

Figure 2 .
Figure 2. Power distributions of orbital angular momentum (OAM) modes: (a) without AO assisted wavefront correction and (b) with AO assisted wavefront correction.

Figure 3 .
Figure 3. (a) Average BER vs. OSNR performance in cases with or without atmospheric turbulence effects.(b) Average BER vs. OSNR performance after AO assisted wavefront correction and lowdensity parity-check (LDPC) coding.

Figure 2 .
Figure 2. Power distributions of orbital angular momentum (OAM) modes: (a) without AO assisted wavefront correction and (b) with AO assisted wavefront correction.

Figure 2 .
Figure 2. Power distributions of orbital angular momentum (OAM) modes: (a) without AO assisted wavefront correction and (b) with AO assisted wavefront correction.

Figure 3 .
Figure 3. (a) Average BER vs. OSNR performance in cases with or without atmospheric turbulence effects.(b) Average BER vs. OSNR performance after AO assisted wavefront correction and lowdensity parity-check (LDPC) coding.

Figure 3 .
Figure 3. (a) Average BER vs. OSNR performance in cases with or without atmospheric turbulence effects.(b) Average BER vs. OSNR performance after AO assisted wavefront correction and low-density parity-check (LDPC) coding.

Figure 7 .
Figure 7. BER vs. average OSNR under an atmospheric turbulence limited environment, when (a) OAM state 2 is under testing; (b) OAM state −6 is under testing.

Figure 6 .
Figure 6.BER vs. OSNR performance in the atmospheric turbulence-free environment, when (a) OAM state 2 is under testing; (b) OAM state −6 is under testing.

encoded 9 -Figure 6 .
Figure 6.BER vs. OSNR performance in the atmospheric turbulence-free environment, when (a) OAM state 2 is under testing; (b) OAM state −6 is under testing.

Figure 7 .
Figure 7. BER vs. average OSNR under an atmospheric turbulence limited environment, when (a) OAM state 2 is under testing; (b) OAM state −6 is under testing.

Figure 7 .
Figure 7. BER vs. average OSNR under an atmospheric turbulence limited environment, when (a) OAM state 2 is under testing; (b) OAM state −6 is under testing.