Analysis of Noise Propagation Mechanisms in Wireless Optical Coherent Communication Systems

Abstract

This paper systematically analyzes the propagation, transformation, and accumulation mechanisms of multi-source noise and device non-idealities within the complete signal chain from the transmitter through the channel to the receiver, focusing on wireless optical coherent communication systems from a signal propagation perspective. It establishes the stepwise propagation process of signals and noise from the transmitter through the atmospheric turbulence channel to the coherent receiver, clarifying the coupling mechanisms and accumulation patterns of various noise sources within the propagation chain. From a signal propagation viewpoint, the study focuses on analyzing the impact mechanisms of factors, such as Mach–Zehnder modulator nonlinear distortion, atmospheric turbulence effects, 90° mixer optical splitting ratio imbalance, and dual-balanced detector responsivity mismatch, on system bit error rate performance and constellation diagrams under conditions of coexisting multiple noises. Simultaneously, by introducing differential and common-mode processes, the propagation and suppression characteristics of additive noise at the receiver end within the balanced detection structure were analyzed, revealing the dominant properties of different noise components under varying optical power conditions. Simulation results indicate that within the range of weak turbulence and engineering parameters, the impact of modulator nonlinearity on system bit error rate is relatively minor compared to channel noise. Atmospheric turbulence dominates system performance degradation through the combined effects of amplitude fading and phase perturbation, causing significant constellation spreading. Imbalanced optical splitting ratios and mismatched responsivity at the receiver weaken common-mode noise suppression, leading to variations in effective signal gain and constellation stretching/distortion. Under different signal light power and local oscillator light power conditions, the system noise exhibits distinct dominant characteristics.

1. Introduction

Wireless optical coherent communication systems combine the advantages of coherent detection with free-space optical transmission, featuring high bandwidth, strong anti-interference capabilities, and high spectral efficiency [1]. They represent one of the key candidate solutions for high-capacity, long-distance, high-speed transmission. However, compared to fiber-optic channels, wireless optical links are more sensitive to atmospheric conditions and device non-idealities. From a system-level perspective, the propagated and coupled noise at the transmitter, channel, and receiver directly impacts the accuracy of amplitude and phase recovery through coherent detection, thereby constraining the system’s actual performance [2].

From the perspective of signal propagation, wireless optical coherent communication systems can typically be analyzed by categorizing noise sources into three types based on their location and impact within the transmission link: transmitter noise, channel noise, and receiver noise. At the transmitter, nonlinear distortion in Mach–Zehnder modulators generates higher-order harmonic components in the modulated signal [3]. Simultaneously, phase noise [4] and relative intensity noise [5] in the laser introduce initial perturbations to the phase and amplitude of the optical field. During channel propagation, atmospheric turbulence induces optical intensity scintillation and wavefront distortion through refractive index fluctuations [6], manifesting as multiplicative amplitude fading and additional phase noise. These effects couple with transmit-side noise and propagate toward the receiver. At the receiver, distortions introduced by the noise of the local oscillator laser, the non-ideal characteristics of the 90° mixer and dual-balanced detector, along with detector noise such as shot noise [7], thermal noise [8], and dark current noise [9], collectively determine the equivalent signal-to-noise ratio level of the received signal.

The coupling mechanisms of the aforementioned multi-source noise during propagation are complex. Different noise sources introduce perturbations at the transmitter, channel, and receiver with distinct physical mechanisms, locations, and timescales. During propagation, these sources couple through modulation, superposition, and transformation, ultimately jointly affecting the accuracy of coherent detection in recovering signal amplitude and phase information. Particularly in the presence of atmospheric turbulence, both the phase and amplitude of the optical field undergo simultaneous random perturbations, resulting in nonlinear superposition characteristics of multi-source noise effects at the receiver.

Existing research on the impact of noise in wireless optical coherent communication has primarily focused on isolated analyses of specific noise sources or individual system modules. For instance, studies often examine only the effects of laser phase noise, atmospheric turbulence amplitude fading, or additive noise at the receiver on system performance. In such investigations, other noise factors are frequently neglected or simplified into equivalent Gaussian noise models. In contrast, a systematic analysis from a unified signal propagation perspective is lacking regarding the propagation, transformation, and accumulation processes of multi-source noise within the complete signal chain, as well as its relative dominance over system performance under coexisting conditions.

Based on the above understanding, this paper investigates noise propagation and accumulation in wireless optical coherent communication systems. Building upon existing noise models, it conducts a unified analysis of the stepwise propagation behavior of signals and multi-source noise at the transmitter, channel, and receiver, following the signal propagation process as the main thread. The paper systematically elucidates the mechanism by which different types of noise interact with the signal in multiplicative or additive forms, ultimately accumulating into the system performance evaluation. Building upon this foundation, the paper critically examines the impact of non-ideal factors—including Mach–Zehnder modulator nonlinear distortion, atmospheric turbulence effects, 90° mixer optical splitting ratio imbalance, and dual-balanced detector responsivity mismatch—on bit error performance and constellation distribution. Simultaneously, by introducing differential and common-mode processes, the propagation and suppression characteristics of additive noise at the receiver end within the balanced detection structure were analyzed. Finally, numerical simulations validate the aforementioned analysis and quantitatively evaluate the relative dominance of different noise sources on system performance under coexisting multi-noise conditions, providing reference for the engineering design and parameter optimization of wireless optical coherent communication systems.

2. System Structure and Noise Propagation Model

2.1. Wireless Optical Coherent Communication System Architecture

Wireless optical coherent communication systems represent a high-speed optical communication technology that utilizes the amplitude and phase of an optical carrier to carry information. By introducing a local oscillator light source at the receiver end to coherently mix with the signal light, the amplitude and phase information of the signal can be precisely recovered, thereby enhancing the system’s sensitivity. The basic structure of the system, as shown in Figure 1, primarily consists of three components: the transmitter, the atmospheric turbulence channel, and the coherent receiver.

At the transmitter end, a Mach–Zehnder modulator modulates the signal to be transmitted onto an optical carrier, achieving both amplitude and phase modulation. The modulated optical signal is transmitted through the atmospheric turbulence channel to the receiver end. At the receiver, the local oscillator light and received signal light undergo interference mixing in a 90° mixer, yielding four optical signals. These signals then undergo electro-optic conversion and differential processing via a dual-balanced detector, producing I and Q electrical signals. This ultimately achieves coherent demodulation and decision-making of the modulated optical signal.

2.2. Signal and Noise Propagation Analysis

To systematically analyze the propagation, transformation, and coupling mechanisms of multi-source noise in wireless optical coherent communication systems at the system level, this paper introduces a unified signal propagation perspective based on existing theoretical models of modulation, channels, and coherent reception. It analyzes the stepwise propagation process of signals and their associated noise across the transmitter, channel, and receiver.

The output signal at the transmitter end can be represented as

$$E_{tx}(t)=E_s{T}_{MZM}\left[s(t)\right]e^{j({\omega }_{s}t+{\varphi }_s(t))}(1+{\delta }_s(t))$$

(1)

In the equation, $E_s$ represents the signal light amplitude, $T_{MZM}\left[\cdot \right]$ denotes the transmission function of the Mach–Zehnder modulator (including nonlinear distortion), $s(t)$ is the modulated signal, ${\omega }_s$ is the signal light frequency, ${\varphi }_s(t)$ is the signal light phase noise, and ${\delta }_s(t)$ is the signal light relative intensity noise.

After propagation through the atmospheric turbulence channel, the received signal is

$$E_{rx}(t)=E_{tx}(t)h_{turb}(t)=E_{tx}(t)\chi (t)e^{j{\varphi }_{turb}(t)}$$

(2)

In the equation, $h_{turb}(t)=\chi (t)e^{j{\varphi }_{turb}(t)}$ represents the complex gain of the atmospheric turbulent channel, $\chi (t)$ denotes the amplitude fading factor, and ${\varphi }_{turb}(t)$ signifies the turbulent phase noise.

At the coherent receiver, the signal light and local oscillator light are mixed in a 90° mixer and then converted into an electrical signal by a dual-balanced detector. Assuming the local oscillator light signal is

$$E_{lo}(t)=E_{lo}{e}^{j({\omega }_{lo}t+{\varphi }_{lo}(t))}(1+{\delta }_{lo}(t))$$

(3)

In the equation, $E_{lo}$ represents the amplitude of the local oscillator light, ${\omega }_{lo}$ denotes the frequency of the local oscillator light, ${\varphi }_{lo}(t)$ signifies the phase noise of the local oscillator light, and ${\delta }_{lo}(t)$ indicates the relative intensity noise of the local oscillator light.

The ideal balanced detection output signals for the I and Q channels can be expressed as

$$I_I(t)\propto \chi (t)\left|T_{MZM}\right|\cos (\mathrm{\Delta }\omega t+{\varphi }_{\mathrm{\Delta }}(t))$$

(4)

$$I_Q(t)\propto \chi (t)\left|T_{MZM}\right|\sin (\mathrm{\Delta }\omega t+{\varphi }_{\mathrm{\Delta }}(t))$$

(5)

In the equation, $\mathrm{\Delta }\omega ={\omega }_s-{\omega }_{lo}$ represents the intermediate frequency, ${\varphi }_{\mathrm{\Delta }}(t)$ denotes the total phase noise, and ${\varphi }_{\mathrm{\Delta }}(t)={\varphi }_s(t)-{\varphi }_{lo}(t)+{\varphi }_{turb}(t)$.

The final received signal, after superimposing various types of additive noise, can be expressed as

$$I(t)=I_{signal}(t)+n_{shot}(t)+n_{the}(t)+n_{dark}(t)+n_{bg}(t)+n_{RIN,leak}(t)$$

(6)

In the equation, $n_{shot}(t)$, $n_{the}(t)$, $n_{dark}(t)$, $n_{bg}(t)$, and $n_{RIN,leak}(t)$ represent shot noise, thermal noise, dark current noise, background light noise, and residual relative intensity noise.

3. Transmitter Noise Analysis

Transmitter noise primarily consists of nonlinear distortion from the Mach–Zehnder modulator and laser noise. The former originates from the modulator’s inherent transmission characteristics and constitutes the dominant distortion source at the transmitter end; the latter stems from the laser’s spontaneous emission process and has a relatively minor impact under the conditions of this system.

3.1. Mach–Zehnder Modulator Nonlinear Distortion

The Mach–Zehnder modulator achieves electro-optic conversion based on the electro-optic effect, with its electric field transfer function given by [3].

$$E_{\mathrm{mod}}(t)=E_{in}(t)\cos ({\varphi }_b+\beta \sin {\omega }_{m}t)$$

(7)

In the equation, $E_{in}(t)$ represents the input light field, ${\varphi }_b=\pi V_{bias}/2V_{\pi }$ denotes the bias phase, $\beta =\pi V_m/2V_{\pi }$ indicates the modulation depth, $V_{\pi }$ corresponds to the half-wave voltage, $V_{bias}$ signifies the bias voltage, and $V_m(t)$ denotes the modulation peak voltage.

The cosine modulation function in Equation (7) can be expanded into a Bessel function series using the Jacobi–Anger identity [10].

$$\cos ({\varphi }_b+\beta \sin {\omega }_{m}t)={\displaystyle \sum _{n=-\infty }^{\infty }{c}_n{e}^{jn{\omega }_{m}t}}$$

(8)

In the equation, $c_n$ represents the complex coefficient of the $n$th harmonic. When the modulator operates at the orthogonal bias point (${\varphi }_b=\pi /2$), even-order harmonic components are suppressed, and the output spectrum contains only odd-order harmonics.

$$c_1=J_1(\beta ),c_3=-J_3(\beta ),c_5=J_5(\beta ),\cdot \cdot \cdot$$

(9)

In the equation, $J_n(\beta )$ denotes the Bessel function of the first kind of order $n$.

Ideally, only the fundamental component $c_1$ should be retained as the useful signal. However, as the modulation depth $\beta$ increases, the power of third-order and higher harmonics will rise, resulting in in-band nonlinear distortion. The signal distortion ratio $\xi$ is defined as the ratio of fundamental power to total harmonic distortion power.

$$\xi ={\displaystyle \frac{{\left|c_1\right|}^2}{{\displaystyle {\sum }_{n=3,5\dots }{\left|c_n\right|}^2}}}={\displaystyle \frac{J_1^2(\beta )}{J_3^2(\beta )+J_5^2(\beta )+\cdot \cdot \cdot }}$$

(10)

Under small-signal conditions ($\beta \ll 1$), utilizing the asymptotic expansion $J_n(\beta )\approx {(\beta /2)}^n/n!$ of the Bessel function yields the approximate relationship $\xi \approx 576/{\beta }^4$. This equation indicates that the signal distortion ratio is inversely proportional to the fourth power of the modulation depth. Therefore, moderately reducing the modulation depth can effectively suppress nonlinear distortion.

3.2. Laser Phase Noise

The phase noise in lasers originates from the random phase perturbations introduced by spontaneously emitted photons into the stimulated radiation field. Spontaneous emission is a stochastic process, wherein each spontaneously emitted photon introduces a random phase increment into the laser field, causing the output phase to undergo random walk. The laser output signal can be expressed as

$$E(t)=E_0{e}^{j(\omega t+\varphi (t))}$$

(11)

In the equation, $E_0$ represents the signal amplitude, $\omega$ represents the signal frequency, and $\varphi (t)$ represents the signal phase noise.

The phase noise of a laser can be modeled as a Wiener process, where the phase change rate is constant Gaussian white noise with a constant power spectral density.

$$\mathrm{S}(f)=2\pi \mathrm{\Delta }v$$

(12)

In the equation, $\mathrm{\Delta }v$ represents the laser linewidth.

According to the properties of the Wiener process, the phase increment $\mathrm{\Delta }{\varphi }_s(t)={\varphi }_s(t+\tau )-{\varphi }_s(t)$ within any time interval τ follows a zero-mean Gaussian distribution [4].

$$\mathrm{\Delta }{\varphi }_s(t)~\mathrm{{\rm N}}(0,2\pi \mathrm{\Delta }{v}_s\left|\tau \right|)$$

(13)

To determine the relative impact of laser phase noise within this system, it must be quantitatively compared with turbulent phase noise. For a system with a symbol rate of $R_s$, the variance of laser phase noise over each symbol period $T_s=1/R_s$ is

$${\sigma }_{\varphi ,laser}^2=2\pi \mathrm{\Delta }\nu T_s={\displaystyle \frac{2\pi \mathrm{\Delta }\nu }{R_s}}$$

(14)

The turbulent phase variance can be expressed as (see Section 3.2)

Table 1. Quantitative Comparison of Laser Phase Noise and Turbulence Phase Noise.

Scenario	${\mathit{C}}_{\mathit{n}}^2$(m−2/3)	L(km)	$\mathrm{\Delta }\mathit{\nu }$(kHz)	${\mathit{\sigma }}_{\mathit{\varphi },\mathit{l}\mathit{a}\mathit{s}\mathit{e}\mathit{r}}^2$(rad2)	${\mathit{\sigma }}_{\mathit{\varphi },\mathit{t}\mathit{u}\mathit{r}\mathit{b}}^2$(rad2)	Ratio
Weak turbulence, short distance	${10}^{-17}$	$1$	$100$	$6.3\times {10}^{-5}$	$1.5\times {10}^{-3}$	$2.45\times {10}^1$
Weak turbulence, long distance	${10}^{-17}$	$10$	$1$	$6.3\times {10}^{-7}$	$1.5\times {10}^{-2}$	$2.45\times {10}^4$
Moderate turbulence	${10}^{-15}$	$5$	$10$	$6.3\times {10}^{-6}$	$0.77$	$1.23\times {10}^5$
Strong turbulence, short distance	${10}^{-13}$	$1$	$100$	$6.3\times {10}^{-5}$	$15.4$	$2.5\times {10}^5$

presents quantitative comparisons of the two types of phase noise under different turbulence intensities, transmission distances, and laser linewidth conditions for typical system parameters ($\lambda =1550 \mathrm{nm}$, $D=0.1 m$, $R_s=10 \mathrm{GBaud}$). The ratio is defined as ${\sigma }_{\varphi ,turb}^2/{\sigma }_{\varphi ,laser}^2$.

Under the parameters of wireless optical coherent communication systems, the variance of turbulent phase noise remains greater than that of laser phase noise even under conditions of weak turbulence and short distances. Under conditions of moderate to strong turbulence, the variance difference between the two on the symbol time scale can reach 3 to 5 orders of magnitude. Therefore, within the parameter range of interest in this paper, the system phase error is primarily dominated by turbulent phase noise, with laser phase noise being a secondary factor.

3.3. Relative Intensity Noise of Lasers

Relative intensity noise originates from random fluctuations in laser output power and is defined as the ratio of the power spectral density of optical power fluctuations to the square of the average power [5].

$$RIN={\displaystyle \frac{S(f)}{{\overline{P}}^2}}$$

(15)

Typically denoted by $\mathrm{dB}/\mathrm{Hz}$, $RI{N}_{dB}=10{\log }_{10}RIN$. The relative intensity noise of the signal laser causes amplitude fluctuations in the modulated signal, introducing noise power of

$${\sigma }_{RIN,s}^2={(RP)}^{2}RI{N}_{s}B$$

(16)

In the equation, $R$ represents the detector responsivity, $P$ denotes the signal power, and $B$ indicates the electrical bandwidth.

In coherent reception systems, the impact of relative intensity noise must be considered separately for both the signal light and the local oscillator light. The noise current variance introduced by the relative intensity noise of the signal light can be expressed as

$${\sigma }_{RIN,s}^2={(R{P}_s)}^{2}RIN\cdot B$$

(17)

In coherent reception systems employing balanced detection structures, the relative intensity noise of the local oscillator light ideally manifests as common-mode noise, which can be completely suppressed through differential detection. However, in practical systems, due to non-ideal factors such as imbalance in the 90° mixer’s optical splitting ratio or mismatched responsivity in dual-balanced detectors, residual forms of the local oscillator light’s relative intensity noise will enter the received signal. The variance of this noise current can be expressed as

$${\sigma }_{RIN,leak}^2={(R{P}_{lo})}^{2}RIN\cdot B\cdot \chi$$

(18)

Under system parameter conditions (signal light power $P_s=-30 \mathrm{dBm}$, local oscillator light power $P_{lo}=10 \mathrm{dBm}$, relative intensity noise $RIN=-150 \mathrm{dB}/\mathrm{Hz}$, electrical bandwidth $B=10 \mathrm{GHz}$, $\chi$ denotes the common-mode rejection ratio, $\chi =0$ indicates complete rejection of common-mode noise, detector responsivity $R=1 \mathrm{A}/\mathrm{W}$), quantitative calculations indicate that the relative intensity noise of the signal light is significantly smaller than the residual relative intensity noise of the local oscillator light. Furthermore, the residual relative intensity noise of the local oscillator light is below the level of shot noise, meaning the entire system still operates within the shot noise-limited region. Therefore, subsequent analyses neglect the relative intensity noise of the signal light, considering only the residual influence of the local oscillator light’s relative intensity noise under non-ideal device conditions.

4. Channel Noise Analysis

Atmospheric turbulence channels are a key characteristic distinguishing wireless optical communication from fiber optic communication. This section establishes a mathematical model of atmospheric turbulence effects and analyzes their impact on signal amplitude and phase. Atmospheric turbulence originates from fluctuations in refractive index caused by random distributions of temperature and pressure within the atmosphere. When a laser beam traverses turbulent air, variations in refractive index at different spatial locations induce random phase delays in the light wave. This manifests macroscopically as beam amplitude fading and phase fluctuations.

The refractive index structure constant $C_n^2$ is a key parameter characterizing turbulence intensity, typically ranging from ${10}^{-17}~{10}^{-13} {\mathrm{m}}^{-2/3}$. For plane waves, the Rytov variance [11] is given by

$${\sigma }_R^2=1.23C_n^2{k}^{7/6}{L}^{11/6}$$

(19)

In the equation, $k=2\pi /\lambda$ represents the wave number of light, and $L$ denotes the link distance. Based on the magnitude of the Rytov variance, turbulence intensity can be classified into three intervals: weak turbulence (${\sigma }_R^2<0.3$), moderate turbulence ($0.3\le {\sigma }_R^2\le 1.0$), and strong turbulence (${\sigma }_R^2>1.0$).

4.1. Amplitude Decay Model

Turbulence-induced amplitude fading causes random fluctuations in received optical power. Let the amplitude fading factor $\chi (t)$ be defined as the ratio of the instantaneous received signal amplitude to its non-turbulent amplitude, whose statistical distribution depends on turbulence intensity.

Under weak turbulence conditions, according to the Rytov approximation theory, $\chi$ follows a log-normal distribution [12].

$$f(\chi )={\displaystyle \frac{1}{\chi \sqrt{2\pi {\sigma }_x^2}}}\exp \left(-{\displaystyle \frac{\left(\ln \chi +{\sigma }_x^2/2\right)}{2{\sigma }_x^2}}\right)$$

(20)

In the equation, the logarithmic amplitude variance is denoted by ${\sigma }_x^2={\sigma }_R^2/4$, and the flicker index (light intensity normalized variance) is denoted by ${\sigma }_I^2=e^{4{\sigma }_x^2}-1$.

Under conditions of moderate turbulence, the coupled interaction between large-scale and small-scale turbulent vortices renders the log-normal model inapplicable. In such cases, the Gamma-Gamma distribution [13] provides a more accurate representation.

$$f(\chi )={\displaystyle \frac{2{(\alpha \beta )}^{\frac{\alpha +\beta }{2}}}{\mathrm{\Gamma }(\alpha )\mathrm{\Gamma }(\beta )}}{\chi }^{{\displaystyle \frac{\alpha +\beta }{2}}-1}{K}_{\alpha -\beta }(2\sqrt{\alpha \beta \chi })$$

(21)

In the equation, $\mathrm{\Gamma }(·)$ and $K_n(·)$ represent the Gamma function and the Bessel function of the second kind of order $n$, respectively. $\alpha$ and $\beta$ denote the effective turbulence parameters for small-scale and large-scale turbulence, respectively, expressed as follows

$$\alpha ={\left(\exp \left({\displaystyle \frac{0.49{\sigma }_R^2}{{\left(1+1.11{\sigma }_R^{12/5}\right)}^{7/6}}}\right)-1\right)}^{-1}, \beta ={\left(\exp \left({\displaystyle \frac{0.51{\sigma }_R^2}{{\left(1+1.11{\sigma }_R^{12/5}\right)}^{5/6}}}\right)-1\right)}^{-1}$$

(22)

4.2. Turbulent Phase Noise

Atmospheric turbulence not only causes signal amplitude fading but also induces random phase perturbations. Such wavefront distortions can be equivalently characterized as additional phase noise ${\varphi }_{turb}$, whose statistical properties are typically approximated as a zero-mean Gaussian distribution.

$${\varphi }_{turb}~\mathrm{{\rm N}}(0,{\sigma }_{\varphi ,turb}^2)$$

(23)

Phase variance and Fried parameter [6] $r_0$ correlation

$${\sigma }_{\varphi ,turb}^2=1.03{({\displaystyle \frac{D}{r_0}})}^{5/3}$$

(24)

In the equation, D denotes the diameter of the receiving aperture. The Fried parameter [6] is defined as

$$r_0={(0.423k^2{C}_n^{2}L)}^{-3/5}$$

(25)

This parameter reflects the variations in atmospheric refractive index along the propagation path, determining the intensity of phase fluctuations caused by turbulence. Combining amplitude fading and phase noise, the complex gain of the atmospheric turbulence channel is

$$h=\chi (t)e^{j{\varphi }_{turb}}$$

(26)

5. Receiver Noise Analysis

The noise sources at the receiving end are relatively complex, including imbalance in the 90° mixer’s optical splitting ratio, mismatch in the response of the dual-balanced detectors, noise from the local oscillator laser, and detector noise. This section focuses on analyzing the impact of the first two non-ideal factors on noise propagation and establishes a unified model for physical noise.

5.1. Non-Ideal Characteristics of 90° Mixers and Dual-Balance Detectors

The 90° mixer is the core optical component of a coherent receiver, performing optical mixing between the signal light and the local oscillator light with a 90° phase difference. It outputs four optical signals for subsequent balanced detection. Let the input signal light be $E_s(\mathrm{t})$ and the local oscillator light be $E_{lo}(\mathrm{t})$. The four outputs of the 90° mixer are [14].

$$\left\{\begin{array}{l}{E}_1(t)=\left(\sqrt{1-{\alpha }_1}{E}_s(t)+\sqrt{{\alpha }_1}{E}_{Io}(t)\right)\hfill \\ E_2(t)=\left(\sqrt{{\alpha }_1}{E}_s(t)-\sqrt{1-{\alpha }_1}{E}_{Io}(t)\right)\hfill \\ E_3(t)=\left(\sqrt{1-{\alpha }_2}{E}_s(t)+j\sqrt{{\alpha }_2}{E}_{Io}(t)\right)\hfill \\ E_4(t)=\left(\sqrt{{\alpha }_2}{E}_s(t)-j\sqrt{1-{\alpha }_2}{E}_{Io}(t)\right)\hfill \end{array}\right.$$

(27)

In the equation, ${\alpha }_k\in [0,1]$ represents the mixer’s optical splitting ratio, and $k=1,2$.

The four outputs of the 90° mixer are fed into two balanced detectors for photoelectric conversion. Let the responsivities of the four photodiodes be $R_1$, $R_2$, $R_3$, and $R_4$, respectively. Then the photocurrent for each channel is [15].

$$i_k(t)=R_k{\left|E_k(t)\right|}^2,k=1,2,3,4$$

(28)

The output of the balance detector is the difference in current between the paired photodiodes, and the corresponding I and Q channel outputs can be expressed as [16].

$$I_I(t)=i_1(t)-i_2(t),I_Q(t)=i_3(t)-i_4(t)$$

(29)

Substituting Equations (28) and (29) into Equation (30) yields the expanded expressions for the I and Q channel outputs.

$$I_I(t)=\left[R_1(1-{\alpha }_1)-R_2{\alpha }_1\right]P_s+\left[R_1{\alpha }_1-R_2(1-{\alpha }_1)\right]P_{lo}+2\sqrt{{\alpha }_1(1-{\alpha }_1)}(R_1+R_2)\Re \left\{E_s(t)E_{lo}^{*}(t)\right\}$$

(30)

$$I_Q(t)=\left[R_3(1-{\alpha }_2)-R_4{\alpha }_2\right]P_s+\left[R_3{\alpha }_2-R_4(1-{\alpha }_2)\right]P_{lo}+2\sqrt{{\alpha }_2(1-{\alpha }_2)}(R_3+R_4)\Im \left\{E_s(t)E_{lo}^{*}(t)\right\}$$

(31)

Equations (31) and (32) demonstrate that response mismatch and optical splitting ratio imbalance lead to DC residual and common-mode noise leakage. The core advantage of ideal balanced detection lies in its ability to eliminate DC components and common-mode noise from both the signal beam and the local oscillator beam. When optical splitting ratio imbalance and response mismatch occur, the DC residual term is non-zero, and the relative intensity noise of the local oscillator beam cannot be completely suppressed.

Neglecting the DC component, the effective signal in the I and Q paths can be expressed as

$$I_I(t)\approx 2\sqrt{{\alpha }_1(1-{\alpha }_1)}(R_1+R_2)\Re \left\{E_s(t)E_{lo}^{*}(t)\right\}$$

(32)

$$I_Q(t)\approx 2\sqrt{{\alpha }_2(1-{\alpha }_2)}(R_3+R_4)\Im \left\{E_s(t)E_{lo}^{*}(t)\right\}$$

(33)

The gains of the I and Q channels can be expressed as

$$G_I=2\sqrt{{\alpha }_1(1-{\alpha }_1)}(R_1+R_2),G_Q=2\sqrt{{\alpha }_2(1-{\alpha }_2)}(R_3+R_4)$$

(34)

Under ideal conditions, the gain ratios of the ${\alpha }_1={\alpha }_2=0.5$, $R_1=R_2=R_3=R_4=R$, I, and Q channels are equal to $G_I/G_Q=1$. When the optical splitting ratio is imbalanced or the responsivity is mismatched, the gains of the $G_I/G_Q\ne 1$, I, and Q channels differ, causing inconsistent scaling of the constellation diagram in the I and Q directions.

5.2. Additive Noise at the Receiver

While accounting for device non-idealities, multiple additive noise sources exist at the receiver end of wireless optical coherent communication systems. These noises superimpose during reception, collectively forming the system’s inherent noise. This section introduces differential and common-mode processes to provide a unified analysis of additive noise at the receiver under a dual-balanced detection structure. Based on this analysis, an equivalent noise power expression after balanced detection is derived, and the propagation and suppression characteristics of various additive noises within the balanced detection structure are discussed.

5.2.1. Differential and Common-Mode Processes

Let the two noise components of the balanced detector be random processes $x(t)$ and $y(t)$. Define the differential process $D(t)$ and the common-mode process $C(t)$ as follows [17]

$$D(t)=x(t)-y(t)$$

(35)

$$\mathrm{C}(t)=x(t)+y(t)$$

(36)

In actual systems, constrained by the common-mode rejection ratio, the total output noise is jointly determined by both differential and common-mode processes. Its expression can be represented as

$$N(t)=D(t)+\chi C(t)$$

(37)

In the equation, $\chi$ denotes the common-mode rejection ratio, and $\chi =0$ represents complete suppression of common-mode noise.

For any two random processes $x(t)$ and $y(t)$, introduce the cross-covariance function [17].

$$Co{v}_{xy}(\tau )\triangleq E[x(t)-{\eta }_x]E[y(t+\tau )-{\eta }_y]$$

(38)

In the equation, ${\eta }_x$ and ${\eta }_y$ represent the mean values of random processes $x(t)$ and $y(t)$, respectively.

When the random process is a zero-mean process, Equation (39) can be simplified to

$$Co{v}_{xy}(\tau )\triangleq E[x(t)]E[y(t+\tau )]=R_{xy}(\tau )$$

(39)

In the equation, $R_{xy}(\tau )$ denotes the cross-correlation function, and its relationship with the normalized cross-correlation coefficient can be expressed as [17].

$$r_{xy}(\tau )={\displaystyle \frac{R_{xy}(\tau )}{{\sigma }_x{\sigma }_y}}$$

(40)

In the equation, $r_{xy}$ is the correlation coefficient between random processes $x(t)$ and $y(t)$, while ${\sigma }_x$ and ${\sigma }_y$ are the standard deviations of random processes $x(t)$ and $y(t)$, respectively.

The average power of the differential process can be expressed as

$$P_D=E[D^2(t)]=P_x+P_y-2E\left[x(t)y(t)\right]$$

(41)

Substituting Equation (41) into Equation (42) yields

$$P_D=P_x+P_y-2r_{xy}{\sigma }_x{\sigma }_y-2{\eta }_x{\eta }_y$$

(42)

Similarly, the average power of the common-mode process can be expressed as

$$P_C=E[C^2(t)]=P_x+P_y+2E\left[x(t)y(t)\right]$$

(43)

Substituting Equation (41) into Equation (44) yields

$$P_C=P_x+P_y+2r_{xy}{\sigma }_x{\sigma }_y+2{\eta }_x{\eta }_y$$

(44)

The total noise power can be expressed as

$$P_N=E[N^2(t)]=P_D+{\chi }^2{P}_C+2\chi (P_x-P_y)$$

(45)

Substituting Equations (43) and (45) into Equation (46) yields

$$P_N={(1+\chi )}^2{P}_x+{(1-\chi )}^2{P}_y-2(1-{\chi }^2)(r_{xy}{\sigma }_x{\sigma }_y+{\eta }_x{\eta }_y)$$

(46)

5.2.2. Background Light Noise

Background light noise is defined as external light sources (such as sunlight or artificial light) entering the receiving device, which approximately follow a Gaussian distribution at high photon counts. Let the background light noise variances [18] entering the two photodetectors be denoted as ${\sigma }_{bg,1}^2$ and ${\sigma }_{bg,2}^2$, respectively. Then,

$${\sigma }_{bg,i}^2=2q{R}_i{P}_{bg}{B}_n, i=1,2$$

(47)

In the equation, $q$ represents the electron charge, $R_i$ denotes the responsivity of the photodetector, $P_{bg}$ signifies the background light power, and $B_n$ indicates the receiver bandwidth.

Since the background light typically illuminates both photodetectors simultaneously, their noise components are highly correlated, i.e., $r_{xy}=1$. Substituting Equation (48) into Equation (47) yields the output noise power after balanced detection of the background light noise

$$P_{bg}={(1+\chi )}^2{\sigma }_{bg,1}^2+{(1-\chi )}^2{\sigma }_{bg,2}^2-2(1-{\chi }^2){\sigma }_{bg,1}{\sigma }_{bg,2}$$

(48)

When $\chi =0$, the output noise power of the background light noise after balanced detection can be expressed as

$$P_{bg}={\sigma }_{bg,1}^2+{\sigma }_{bg,2}^2-2{\sigma }_{bg,1}{\sigma }_{bg,2}$$

(49)

Due to the background light entering the two photodetectors, ${\sigma }_{bg,1}={\sigma }_{bg,2}$ is present, yielding the background light noise output power $P_{bg}=0$. This indicates that background light noise, as a typical common-mode noise component, can be completely suppressed under ideal balanced detection conditions. However, under non-ideal conditions with limited common-mode rejection ratio, residual background light noise will leak to the differential output terminals.

5.2.3. Shot Noise

Shot noise refers to fluctuations in the output current of photodetectors caused by the random arrival or emission of photons [7], which follow a Poisson distribution and approximate a Gaussian distribution at high photon counts. Let the variance of the shot noise generated by the two detectors be denoted as ${\sigma }_{shot,1}^2$ and ${\sigma }_{shot,2}^2$, respectively. Then,

$$\left\{\begin{array}{c}{\sigma }_{shot,1}^2=2q{R}_1\alpha (P_s+P_{lo})B_n\\ {\sigma }_{shot,2}^2=2q{R}_2(1-\alpha )(P_s+P_{lo})B_n\end{array}\right.$$

(50)

In the equation, $q$ represents the electron charge, $\alpha \in [0,1]$ denotes the spectral ratio, $R_k$ signifies the photodetector’s responsivity ($k=1,2$), $P_s$ indicates the signal optical power, $P_{lo}$ represents the local oscillator optical power, and $B_n$ denotes the receiver bandwidth.

5.2.4. Dark Current Noise

Dark current noise is defined as the noise caused by the current (dark current) generated by a photodetector even in the absence of illumination, with its fluctuations approximating a Gaussian distribution. Let the variance of the dark current noise for two photodetectors be ${\sigma }_{dark,1}^2$ and ${\sigma }_{dark,2}^2$, respectively [8]. Then,

$${\sigma }_{dark,i}^2=2q{I}_{d,i}{B}_n, i=1,2$$

(51)

In the equation, $q$ represents the electron charge, $I_{d,i}$ represents the dark current, and $B_n$ represents the receiver bandwidth.

5.2.5. Thermal Noise

Thermal noise refers to the noise current generated by the random thermal motion of internal charge carriers (typically electrons) under thermal fluctuations. According to the central limit theorem, it can be approximated as a Gaussian random process with zero mean. Let the thermal noise variances [9] of two photodetectors be ${\sigma }_{the,1}^2$ and ${\sigma }_{the,2}^2$, respectively. Then,

$${\sigma }_{the,i}^2={\displaystyle \frac{4kT}{R_{L,i}}}{B}_n, i=1,2$$

(52)

In the equation, $k$ is the Boltzmann constant, $T$ is the temperature in kelvins, $R_{L,i}$ is the load resistance, and $B_n$ is the receiver bandwidth.

Thermal noise, dark current noise, and shot noise all originate independently within the photodetector, and the noise from the two channels is mutually independent, meaning their correlation coefficient is zero ($r_{xy}=0$). When the common-mode rejection ratio is suboptimal ($\chi \ne 0$), substituting Equations (51)–(53) into Equation (47) yields the output noise powers of thermal noise, dark current noise, and shot noise after balanced detection as follows

$$P_{the}={(1+\chi )}^2{\sigma }_{the,1}^2+{(1-\chi )}^2{\sigma }_{the,2}^2$$

(53)

$$P_{dark}={(1+\chi )}^2{\sigma }_{dark,1}^2+{(1-\chi )}^2{\sigma }_{dark,2}^2$$

(54)

$$P_{shot}={(1+\chi )}^2{\sigma }_{shot,1}^2+{(1-\chi )}^2{\sigma }_{shot,2}^2$$

(55)

As can be seen from the above, thermal noise, dark current noise, and scattered photon noise are all output through the differential node in a superimposed manner. Their propagation paths are direct: generated by the photodetector, they are superimposed onto the differential output terminal and remain independent from the signals and noise propagating from the preceding stage.

6. Comprehensive Performance Analysis

This paper employs a power-based symbol signal-to-noise ratio (SNR) to analyze system performance. The instantaneous SNR is defined as the ratio of the useful signal’s instantaneous power to the equivalent total noise power at the receiver’s sampling decision point, expressed as

$${\gamma }_0={\displaystyle \frac{{(2R\sqrt{P_s{P}_{lo}})}^2}{2{\sigma }_n^2}}={\displaystyle \frac{2R^2{P}_s{P}_{lo}}{{\sigma }_n^2}}$$

(56)

In the equation, ${\sigma }_n^2$ represents the equivalent total noise power.

The total noise variance of the system is composed of all additive noise components.

$${\sigma }_n^2={\sigma }_{bg}^2+{\sigma }_{shot}^2+{\sigma }_{dark}^2+{\sigma }_{the}^2+{\sigma }_{RIN,leak}^2+{\sigma }_{nl}^2$$

(57)

In the equation, ${\sigma }_{bg}^2$, ${\sigma }_{shot}^2$, ${\sigma }_{dark}^2$, ${\sigma }_{the}^2$, ${\sigma }_{RIN,leak}^2$ and ${\sigma }_{nl}^2$ represent the background light noise, shot noise, dark current noise, thermal noise, residual relative intensity noise, and equivalent noise due to modulator nonlinear distortion, respectively, where H ${\sigma }_{nl}^2={\chi }^2(t){(2R\sqrt{P_s{P}_{lo}})}^2/\xi$.

Assuming the receiver employs ideal carrier recovery, where phase errors are fully compensated during coherent demodulation, the impact of turbulence on system performance manifests solely through amplitude fading. Thus, the effective signal-to-noise ratio after turbulence is considered to be

$${\gamma }_{eff}={\gamma }_0{\chi }^2(t)$$

(58)

It should be noted that the aforementioned ideal carrier recovery assumption applies to operating conditions where the laser has a narrow linewidth, the phase tracking loop performs well, and the turbulence phase noise variance remains within compensatable limits. Under conditions of strong turbulence or significant phase fluctuations, residual phase errors may not be fully eliminated. Their impact on system performance will no longer manifest solely as amplitude fading, necessitating the explicit introduction of a phase noise term for joint modeling.

For QPSK modulation, its conditional bit error rate under a given channel state can be expressed as [19].

$$P_e=Q(\sqrt{{\gamma }_{eff}})={\displaystyle \frac{1}{2}}erfc(\sqrt{{\displaystyle \frac{{\gamma }_{eff}}{2}}})$$

(59)

In the equation, $Q(\cdot )$ represents the $Q$ function, and $erfc(\cdot )$ represents the complementary error function.

By statistically averaging the turbulent distribution and other random channel conditions, the system’s average bit error rate is obtained [20].

$${\overline{P}}_e={\displaystyle {\int }_0^{\infty }{P}_{e}f(\chi )}d\chi$$

(60)

7. Simulation Results and Analysis

This section analyzes the impact of system parameters and channel parameters on the performance of wireless optical coherent communication systems.

Table 2. Simulation Parameters Used.

Parameters	Symbol	Value (Unit)
Temperature	$T$	$300 (\mathrm{K})$
Load Resistance	$R_L$	$50 (\mathsf{\Omega })$
Bandwidth	$B_n$	$0.155\times {10}^9 (\mathrm{GHz})$
Dark Current	$I_d$	$1\times {10}^{-9} (\mathrm{A})$
Backlight Power	$P_{bg}$	$1\times {10}^{-8} (\mathrm{W})$
RIN	$RIN$	$-155 (\mathrm{dB}/\mathrm{Hz})$
Detector Responsivity	$R$	$0.6~0.9 (\mathrm{A}/\mathrm{W})$
optical splitting ratio	$\alpha$	$0.3~0.7$
Electron Charge	$q$	$1.602\times {10}^{-19} (\mathrm{C})$
Boltzmann Constant	$k$	$1.38\times {10}^{-23} (\mathrm{J}/\mathrm{K})$

lists the relevant parameters used in the simulations. To validate the accuracy of the analytical expressions, numerical simulations were conducted using the Monte Carlo method. For each signal-to-noise ratio (SNR) point, no fewer than 10^5 mutually independent channel realizations were generated. Each channel realization transmitted a single independent modulated symbol, ensuring that the bit error rate (BER) statistics for each SNR point were based on at least 10^5 independent symbol samples. This configuration effectively suppresses statistical fluctuations in the low BER region, guaranteeing that the BER estimation results exhibit high confidence and reproducibility.

7.1. Nonlinear Distortion in the Mach–Zehnder Modulator

This section conducts a simulation analysis of the nonlinear characteristics of the Mach–Zehnder modulator, focusing on the effects of modulation depth $\beta$ on output harmonic components, signal distortion ratio, and system bit error rate performance. modulation depth is uniformly sampled within the $\beta \in [0.01,2]$ interval.

Figure 2 shows the variation curves of the fundamental wave and third- and fifth-harmonic amplitudes with modulation depth $\beta$. The fundamental amplitude increases significantly with increasing $\beta$, peaking near $\beta \approx 1.7~1.8$ (approximately 0.58) before slightly declining. In contrast, the third and fifth harmonic amplitudes remain substantially smaller than the fundamental, rising only gradually at high modulation depths. At point $\beta =2$, the third harmonic is approximately 0.1, while the fifth harmonic approaches 0. This indicates that within the practical operating range of $0~2$, especially $\beta \le 1$, the output of the Mach–Zehnder modulator is predominantly dominated by fundamental components. Higher-order harmonics constitute a very small proportion, and nonlinear harmonic distortion is relatively weak. Therefore, it can be concluded that the modulator exhibits good linearity within this range.

Figure 3 shows the variation of the signal distortion ratio with modulation depth $\beta$. It can be observed that as $\beta$ increases, the proportion of higher harmonic energy gradually rises, and the signal distortion ratio exhibits a monotonically decreasing trend. At low modulation depths ($\beta \le 0.3$), the third and fifth harmonics are minimal, and the signal distortion ratio is generally greater than $45 \mathrm{dB}$. When $\beta$ increases to approximately $0.5$, the signal distortion ratio drops to around $40 \mathrm{dB}$. approaching $\beta =1.0$, the signal distortion ratio further decreases to approximately $30 \mathrm{dB}$. This indicates that deeper modulation results in higher energy ratios of third and fifth harmonics relative to the fundamental wave, leading to more pronounced nonlinear distortion. However, within the practical operating range of $\beta \le 1.0$, the signal distortion ratio remains above approximately $30 \mathrm{dB}$. The power of higher-order harmonic distortion is significantly smaller than the useful signal power, making the linearity of the Mach–Zehnder modulator acceptable for system performance.

Figure 4 shows the BER–SNR curves at different modulation depths. Although the corresponding signal distortion ratio decreases from approximately $67.6 \mathrm{dB}$ to approximately $27.0 \mathrm{dB}$, the BER curves at different modulation depths almost completely overlap within the SNR range of $0~25 \mathrm{dB}$. The required SNR at typical metrics such as $BER={10}^{-6}$ is essentially identical. Therefore, the equivalent noise introduced by the nonlinearity of the Mach–Zehnder modulator is far smaller than the channel noise. Its impact on the system’s error performance is negligible, meaning system performance is primarily determined by channel noise rather than modulator nonlinearity.

7.2. Channel Noise Simulation Analysis

Under specified system parameters (wavelength $\lambda =1550 \mathrm{nm}$, transmission distance $L=1 \mathrm{km}$, receiving aperture diameter $D=0.1 \mathrm{m}$), simulations were conducted on the bit error performance and constellation characteristics of wireless optical coherent communication systems for different atmospheric structure constants $C_n^2$, based on the log-normal and Gamma–Gamma turbulence models. The analysis primarily focuses on the relationship between the average bit error rate and normalized signal-to-noise ratio under varying turbulence intensities, the variation in phase noise standard deviation with turbulence intensity, and the evolution of constellation patterns under different turbulence conditions.

Figure 5 shows the variation in the system’s average bit error rate (BER) with signal-to-noise ratio (SNR) under different atmospheric structure constants $C_n^2$. Turbulence intensity is characterized by the structure constant $C_n^2$, where $C_n^2=1\times {10}^{-16} {\mathrm{m}}^{-2/3}$, $C_n^2=1\times {10}^{-15} {\mathrm{m}}^{-2/3}$, and $C_n^2=1\times {10}^{-14} {\mathrm{m}}^{-2/3}$ represent weak, moderate, and strong turbulence, respectively. As shown in the figure, the average BER decreases monotonically with increasing average SNR under all three turbulence conditions. However, at the same SNR, stronger turbulence (higher $C_n^2$) results in a higher system BER, indicating that turbulence significantly degrades the system’s error performance.

Although similar simulations have been conducted in [21,22], they did not account for the influence of total additive noise. To address this gap, we introduce residual relative intensity noise, background light noise, scattered particle noise, dark current noise, and thermal noise, whose expressions are given by Equations (19) and (36) to Equation (39), respectively.

As shown in Figure 6, under weak turbulence ($C_n^2=3.5\times {10}^{-15} {\mathrm{m}}^{-2/3}$) conditions, different types of noise exert a significant influence on the average bit error rate. Simulation results indicate that as the average received optical power increases, the average bit error rate for all noise types exhibits a monotonically decreasing trend, with local oscillator light-correlated noise playing a dominant role in system performance. For example, when the average received optical power is $P=-20 \mathrm{dBm}$, the system’s average bit error rate is approximately $1.6\times {10}^{-3}$ when considering all noise sources. At this point, the average bit error rate when considering only the local oscillator’s photonic shot noise is approximately $1.1\times {10}^{-3}$, which is nearly identical to the result under total noise conditions.

In contrast, the impact of the remaining noise on system performance is significantly weaker. When considering only thermal noise, the average bit error rate (ABER) is approximately $2.1\times {10}^{-6}$, about one order of magnitude lower than the total noise scenario. When considering only residual local oscillator optical intensity noise, the ABER is approximately $1.7\times {10}^{-6}$. When only background noise is present, the ABER further decreases to the $6.7\times {10}^{-12}$ level. The impact of signal shot noise is minimal, with a corresponding ABER of approximately $3.3\times {10}^{-16}$, which is negligible across the entire received power range. As the received optical power continues to increase, the overall average bit error rate curve consistently follows the variations in the local oscillator light shot noise, indicating that the system has entered a local oscillator light shot noise-limited operating state.

Furthermore, under the system parameter conditions considered in this paper, dark current typically resides at the nanoampere level or even lower. Its corresponding noise variance is significantly smaller than that of the local oscillator’s photon shot noise and thermal noise. Consequently, its impact on the system’s bit error performance can be neglected across the entire received power range.

Figure 7 and Figure 8 further analyze the impact of turbulence on coherent QPSK from the perspectives of phase noise and constellation distribution. Simulation results for different atmospheric structure constants $C_n^2$ indicate that as turbulence intensity increases, the phase noise standard deviation exhibits a significant monotonically increasing nonlinear relationship with both ${\sigma }_{\varphi }$ and $C_n^2$. Within the ${10}^{-17}~{10}^{-16} {\mathrm{m}}^{-2/3}$ interval, ${\sigma }_{\varphi }$ increases only from a few degrees to over ten degrees. The corresponding constellation diagram still maintains four distinct clusters, exhibiting only slight dispersion. When $C_n^2=5\times {10}^{-17} {\mathrm{m}}^{-2/3}$, ${\sigma }_{\varphi }\approx {5.0}^{\circ }$, the four constellations are compactly distributed within their respective quadrants, far from the judgment boundary, and can be regarded as a “clearly distinguishable” state under weak turbulence conditions; When $C_n^2$ increases to $5\times {10}^{-16} {\mathrm{m}}^{-2/3}$ and ${\sigma }_{\varphi }\approx {15.9}^{\circ }$, the constellation clusters begin to stretch along the circumference, expanding their clustering range while remaining predominantly distributed within their respective quadrants. With a limited number of points near the decision axis, the constellation diagram structure remains easily recognizable, exhibiting only “mild blurring.” As turbulence continues to increase to level $C_n^2=5\times {10}^{-15} {\mathrm{m}}^{-2/3}$, the phase standard deviation reaches level ${\sigma }_{\varphi }\approx {50.3}^{\circ }$, exceeding half the interval between adjacent constellation points (${45}^{\circ }$). At this point, the constellation point cloud essentially forms a closed loop, with numerous points crossing the decision boundary. The symbol regions in the four quadrants overlap significantly, causing the constellation to lose its original “four-point” structure. This stage can be considered the onset of “difficulty in distinguishing.” As turbulence intensifies further to $C_n^2=5\times {10}^{-14} {\mathrm{m}}^{-2/3}$, ${\sigma }_{\varphi }\approx {159.1}^{\circ }$, the constellation points exhibit an approximately isotropic circular distribution across the entire complex plane. Different symbols become statistically indistinguishable, corresponding to the complete failure of QPSK under strong turbulence. Using QPSK phase spacing ${90}^{\circ }$ as a reference, when ${\sigma }_{\varphi }\ge {45}^{\circ }$ (corresponding to $C_n^2\approx {10}^{-14}~2\times {10}^{-14} {\mathrm{m}}^{-2/3}$) occurs, the constellation diagram becomes visually indistinguishable and difficult to interpret for decision-making purposes.

7.3. Optical Splitting Ratio Imbalance and Responsivity Mismatch

To analyze the effects of imbalance in the optical splitting ratio of a 90° mixer and mismatch in the responsivity of dual-balanced detectors on the received constellation diagram, this section simulates the relevant non-ideal factors. The simulation employs specified system parameters ($\lambda =1550 \mathrm{nm}$, $P_s=-26.81 \mathrm{dB}$, $P_{lo}=0 \mathrm{dB}$), disregarding atmospheric turbulence while retaining only receiver noise and device non-idealities. By varying the mixer’s optical splitting ratio and detector responsivity independently, changes in constellation diagram morphology are observed to analyze signal degradation patterns under different non-ideal conditions.

Under the premise that all four detectors in the dual-balanced detector have a response of $0.9 \mathrm{A}/\mathrm{W}$, simulate the following four sets of spectral ratio scenarios: I channel optical splitting ratio imbalance (${\alpha }_1=0.3,{\alpha }_2=0.5$), Q channel optical splitting ratio imbalance (${\alpha }_1=0.5,{\alpha }_2=0.3$), and I and Q channels optical splitting ratio imbalance (${\alpha }_1=0.3,{\alpha }_2=0.7$, ${\alpha }_1=0.7,{\alpha }_2=0.3$).

As shown in Figure 9, at the ideal optical splitting ratio (${\alpha }_1={\alpha }_2=0.5$), the mixing efficiency reaches its theoretical maximum, and the gains in the I and Q channels are perfectly matched. At this point, the four constellation point clouds are symmetrically distributed along the I and Q axes. Each constellation cluster is approximately circular, with similar dispersion in both the I and Q directions. The constellation clusters exhibit the smallest radius, resulting in the most compact point cloud distribution.

Compared to the ideal spectral ratio, the received constellation diagrams under all non-ideal spectral ratios shown in Figure 10 share the following common characteristics: the four QPSK constellation clusters remain distributed across the I and Q quadrants, but the radius of each cluster increases, the point clouds spread outward, and the edges of the clusters become more blurred. This indicates that at the same transmit power, the equivalent SNR at the receiver decreases.

With the optical splitting ratio fixed at the ideal value (${\alpha }_1={\alpha }_2=0.5$), modifications to the photodetector responsivity were made to analyze the following three scenarios: $R_1=R_2=R_3=R_4=0.9$. Complete responsivity matching $R_1=R_2=0.9,R_3=R_4=0.6$. Overall low responsivity in the Q channel $R_1=R_2=0.6,R_3=R_4=0.9$. Overall low responsivity in the I channel The corresponding constellation diagrams are shown in Figure 11.

When the responsivity is perfectly matched, the I and Q differential channels exhibit symmetry. The received constellation forms an approximately circular point cloud distributed across four quadrants, with similar expansion in both I and Q directions. This pattern remains largely undistorted, reflecting only noise-induced effects. When the overall responsivity of the Q channel is low, its output is uniformly scaled down under the same light field. After normalization, the higher-response I channel is relatively amplified, while the lower-response Q channel is further compressed. The four points in the constellation diagram are elongated along the I-axis and flattened along the Q-axis, forming horizontal ellipses. When the overall response of the I channel is low, the pattern mirrors the previous scenario: the constellation diagram exhibits vertical ellipses, with expansion in the Q direction and compression in the I direction.

7.4. Variation in Noise Components at Different Optical Powers

Set the signal light power to $-90~0 \mathrm{dBm}$ and fix the local oscillator light power at $10 \mathrm{dBm}$. Simulate and analyze the relationship between different noise components and signal light power in a wireless optical coherent communication system. The results are shown in Figure 12.

Simulation results indicate that different noise components exhibit distinct dominant characteristics across varying signal optical power ranges: at low signal optical power levels, the system’s total noise is primarily determined by receiver-side additive noise—such as oscillator shot noise and thermal noise—which is independent of signal power. As signal optical power increases, relative intensity noise gradually becomes dominant, nearly completely dictating the system’s total noise level at high signal optical power levels. Therefore, in practical system design, weak-signal scenarios should prioritize optimizing receiver noise performance, while higher signal optical power conditions require prioritizing suppression of the light source’s relative intensity noise to further enhance the system’s signal-to-noise ratio and overall performance.

Furthermore, when the local oscillator light power is set to $-10~20 \mathrm{dBm}$ and the fixed signal light power is set to $-40 \mathrm{dBm}$, a simulation analysis was conducted on the relationship between different noise components and the local oscillator light power in a wireless optical coherent communication system. The results are shown in Figure 13.

Simulation results indicate that system noise exhibits distinct zone characteristics with variations in local oscillator optical power: at low local oscillator optical power levels, system noise is primarily constrained by receiver-side inherent noise such as thermal noise; whereas at higher local oscillator optical power conditions, the local oscillator’s shot noise rapidly increases and becomes the dominant noise source, determining the overall noise level of the system. Therefore, a reasonable upper limit exists for selecting the local oscillator optical power. Excessively high local oscillator power not only fails to further enhance system performance but also limits signal-to-noise ratio improvement due to amplified shot noise.

8. Conclusions

This paper addresses the propagation and accumulation of multi-source noise in wireless optical coherent communication systems. From a signal propagation perspective, it systematically analyzes the stepwise mechanism of noise effects at the transmitter, channel, and receiver. Combined with numerical simulations, it quantitatively evaluates the relative dominance of different noise sources on system performance under coexisting multi-noise conditions. Based on the system architecture and parameter settings employed in this study, the following conclusions can be drawn:

(1)

Noise Propagation Mechanisms and Pathways: From the perspective of noise propagation pathways, receiver noise can be broadly categorized into three types, each exhibiting fundamentally distinct propagation characteristics. Additive noise (thermal noise, dark current noise, and shot noise) originates within the photodetector itself. These two noise components operate independently and, after balanced detection, manifest at the differential output as a power superposition, constituting the system’s unavoidable inherent noise floor. Common-mode noise (e.g., relative intensity noise and background light noise) can be completely suppressed at the differential node when device characteristics are ideal, effectively blocking its propagation path. However, when devices exhibit non-ideal characteristics, common-mode rejection capability diminishes, leading to common-mode leakage that enters the differential output as residual noise. Multiplicative noise (laser phase noise and atmospheric turbulence noise) directly modulates the signal in amplitude or phase. After coherent mixing, it directly affects the beat frequency signal, exhibiting a “pass-through” propagation characteristic. This is the most challenging noise factor to suppress through the receiving structure and ultimately limits system performance.

(2)

Transmitter Analysis: The nonlinear distortion of the Mach–Zehnder modulator was analyzed using a Bessel function expansion. Within the practical operating range, the signal distortion ratio remains above $30 \mathrm{dB}$. The equivalent distortion component introduced by the modulator becomes secondary to atmospheric turbulence noise after channel propagation and interaction with other noise sources. This result indicates that under conditions of coexisting multiple noises, the impact of modulator nonlinear distortion on system bit error performance can be neglected within a certain parameter range.

(3)

Channel Analysis: Atmospheric turbulence is the dominant factor in the noise propagation chain, with its multiplicative amplitude fading and additional phase perturbations having a decisive impact on coherent detection performance. When the phase noise standard deviation approaches 45°, the demodulation performance of QPSK systems degrades significantly, rendering them nearly inoperable.

(4)

Receiver Analysis: Imbalances in the 90° mixer’s splitting ratio and mismatches in the dual-balanced detector’s responsivity primarily affect noise propagation by reducing mixing efficiency, causing I/Q gain mismatches, and weakening common-mode noise suppression capabilities. Their impact on system performance manifests mainly as constellation spread and distortion.