Matched Filtering for MIMO Coherent Optical Communications with Mode-Dependent Loss Channels

The use of digital signal processors (DSP) to equalize coherent optical communication systems based on spatial division multiplexing (SDM) techniques is widespread in current optical receivers. However, most of DSP implementation approaches found in the literature assume a negligible mode-dependent loss (MDL). This paper is focused on the linear multiple-input multiple-output (MIMO) receiver designed to optimize the minimum mean square error (MMSE) for a coherent SDM optical communication system, without previous assumptions on receiver oversampling or analog front-end realizations. The influence of the roll-off factor of a generic pulse-amplitude modulation (PAM) transmitter on system performance is studied as well. As a main result of the proposed approach, the ability of a simple match filter (MF) based MIMO receiver to completely eliminate inter-symbol interference (ISI) and crosstalk for SDM systems under the assumption of negligible MDL is demonstrated. The performance of the linear MIMO fractionally-spaced equalizer (FSE) receiver for an SDM system with a MDL-impaired channel is then evaluated by numerical simulations using novel system performance indicators, in the form of signal to noise and distortion ratio (SNDR) loss, with respect to the case without MDL. System performance improvements by increasing the transmitter roll-off factor are also quantified.


Introduction
The increasing demand of higher bit rates, combined with the environmental requirement of energy-efficient communication systems, is driving the development of ultra-highcapacity fiber optic communications. In this context, recent advances in spatial division multiplexing (SDM) using multimode or multicore fibers in long-and short-distance links [1,2] cannot be possible without the extensive use of multiple-input multiple-output (MIMO) signal processing.
Since the initial proposal to use polarization-division multiplexing (PDM) in a single mode fiber (SMF) [3] to double the capacity of a coherent optical communication system, MIMO signal processing [4,5] has become necessary to process and recover the parallel transmitted data streams even before the signal processing used was identified as a MIMO equalizer [6]. The channel model for PDM in SMF and its relation with the non-linear Schrödinger equation [7], its representation by means of the 2 × 2 Jones matrix [8] and as a multi-section system [9] has been extensively discussed in previous works. Multiple contributions to adaptive MIMO equalizers using the flexibility of digital signal processors (DSP) have been developed [7,10,11], where normally the equalization is divided into two parts: The first one, with an invariant chromatic dispersion (CD) compensation for each

Notation
Matrices are represented as M, and vectors as v. Vectors are column vectors unless otherwise noted. x denotes the largest integer less than or equal to x. x * represents the conjugate of x, M H denotes the Hermitian of M, and v T represents the transpose of the vector v. E[x] is the expectation operator applied to the random variable x. i ∈ {1, . . . , D} is used to index a mode among the D modes used in the fiber and * represents the convolution operator. The result of the convolution operator applied to a D 1 × D 2 matrix a(t) and a D 2 × D 3 matrix b(t) is a D 1 × D 3 matrix denoted as c(t) and given by: where each of the elements of c(t), denoted as c ij (t), are obtained as in a simple matrix multiplication, but substituting the product by the convolution operator: Similarly, the result of the convolution operator applied to a D 1 × D 2 matrix a(t) and a time dependent signal y(t) is a D 1 × D 2 matrix denoted as d(t) where d(t) = a(t) * y(t).
Each of the elements of the matrix d(t) are obtained as in a multiplication of a matrix with a scalar, however substituting the product by the convolution operator: F {y(t)} denotes the Fourier transform of the continuous-time signal y(t) and F −1 {Y(ω))} the denotes inverse Fourier transform of Y(ω). Similarly, for the discrete-time signal y[n] we denote its corresponding discrete Fourier transform as Y(Ω).

Long-Haul Optical Link MIMO Channel Model
In this section we describe the multi-section optical channel model used in this work. The effect of the channel noise is discussed separately in Section 4. The relationship between the input vector x(ω) = [x 1 (ω), x 2 (ω), . . . , x D (ω)] T of complex electric field amplitudes of each of the D modes propagating along the fiber, and the corresponding output vector y(ω) = [y 1 (ω), y 2 (ω), . . . , y D (ω)] T can be modeled, after neglecting non-linear effects, as a multiple-input multiple-output linear system H tot (ω) [2]: where H tot (ω) is a D × D matrix that models the signal propagation along the channel. For D = 2, the system is equivalent to a classical PDM over a SMF, and H tot (ω) takes the form of the Jones matrix [8]. For D > 2, extensions to the Jones matrix have been proposed to be adequate for the SDM model [19,20].
In the case of long-haul systems, H tot (ω) can be further modeled as a concatenation of K amp spans composed of the optical fiber and an optical amplifier [2,18,33,34]. Hence, the whole channel transfer function can be written as: where H CD (ω) = e − j 2 ω 2β 2 tot is a single-input single-output (SISO) term that models the mode-averaged distortion due to CD,β 2 represents the mode-averaged CD per unit length, and tot denotes the total link length. The matrix H(ω) includes inter-mode cross-talk, MDL and MD effects of the complete MIMO system. Equation (5) can be written as a product over the K amp spans: where H (k) (ω) is the channel response of the kth span. We use k ∈ {1, . . . , K amp } to index the spans in the optical channel. We can write out H (k) (ω) as [20,34]: where the diagonal matrix Λ (k) (ω) for a given span k includes the MDL effects and the MD of each mode w.r.t. the mode-averaged value [33], and can be expressed as: being D ] the uncoupled modal gains and D ] uncoupled modal group delays. We assume that the uncoupled modal group-velocity dispersion is equal to zero for all the k spans [33].
The k-th span mode coupling is modeled by the frequency-independent V (k) and U (k) matrices. It is important to note that, by considering that all the modes propagating through the fiber experience the same attenuation, both matrices are unitary, i.e., Alternatively, H(ω) can also be written by applying a singular value decomposition (SVD), as the product of two unitary matrices U (tot) (ω) and V (tot) (ω), and a diagonal matrix Λ (tot) (ω) as [35]: where now, the diagonal matrix Λ (tot) (ω) is given by: where g (tot) = [g Note that in (10), both U (tot) (ω) and V (tot) (ω) unitary matrices have in general frequency dependence, in contrast to U (k) and V (k) in (7) that have not [27,33].

SDM Communication System Model
This section describes the model employed to represent the communication system established over the optical channel with multiple spans. The SVD of the channel in (10) can be useful for designing a transmitter based on a precoding matrix combined with a linear receiver, as used in wireless systems [36]. However, this approach becomes unfeasible for long-haul optical communication systems, since the end-to-end channel side information needed to build the transmitter precoding matrix changes faster than the time needed for the system to collect, send, and process that information [33]. Therefore we focus on a SDM system with no channel side information that uses a linear receiver to cope with the channel impairments as shown in Figure 1 [37].
The binary data symbols, s[n] = [s 1 [n], s 2 [n], . . . , s D [n]] T , are PAM modulated in parallel for each of the i ∈ {1, . . . , D} optical modes using the same transmitter pulse P(ω) to get the PAM signals, denoted by the column vector x(t) = [x 1 (t), x 2 (t), . . . , x D (t)] T . In Figure 1, T is the transmitted symbol period and the first block represent D parallel PAM modulators working at a symbol rate (and, hence, it includes the discrete-time to continuous-time conversion). The transmitted signal is distorted by ISI and crosstalk introduced by the MIMO channel, modeled with the H tot (ω) matrix described in Section 3. It has been shown that the noise in a MDL-impaired system is additive and spatially white [21]. Therefore, in this work we add before the receiver, as part of the channel, an additive white Gaussian noise (AWGN) vector n(t) = [n 1 (t), n 2 (t), . . . , n D (t)] T , which, for a certain mode i, has a variance equal to N 0 2 . The resulting continuous-time signal vector y(t) = [y 1 (t), y 2 (t), . . . , y D (t)] T is processed by a MIMO receiver to obtain the estimation of the transmitted symbols s[n], denoted asŝ In this work we focus on linear MIMO receivers and so, the estimation part in the receiver is depicted in Figure 1 with a generic linear filter of response O(ω), which is followed by a sampler working at the symbol rate. In the following, we propose linear MIMO receiver structures based on the MMSE criterion of an estimated symbol vector.

Transmitter
The transmitted data in each of the D modes are modulated using a PAM with a square-root raised cosine pulse p(t) with roll-off factor equal to α and normalized power, which can be expressed as [38]: Hence, we can write the sequence of PAM pulses for a given mode i as: where s i [n] is a random variable with values taken from the set defined by the PAM modulation scheme. Let us define the global impulse response q(t) as the convolution of the transmitting pulse p(t) and the optical channel impulse response matrix h tot (t) as: so that q(t) is a D × D matrix of impulse responses. This way, q ij (t) describes the impulse response between the transmitter mode i and the receiver mode j. Therefore, we can write the relationship between the transmitted symbols s j [n] and the received signal in mode i, y i (t), as: with n i (t) as the noise in the i-th receiver mode.

Linear MMSE MIMO Receiver
The most widely used linear MIMO receiver for SDM systems is based on the design of filter O(ω) in Figure 1 to minimize the mean squared error (MSE), which is called a linear MMSE MIMO receiver [28,37,39]. Mathematically, the MSE for the linear MIMO receiver under the MMSE criterion is defined as: where withŝ[n] as the output vector of the linear MIMO receiver O(ω).
It is well known that the structure of a linear MMSE MIMO receiver can be divided into a matched filter Q H (ω) operating in continuous time, a sampler operating at the symbol rate, and a discrete-time equalizer of response W(Ω), as presented in Figure 2. We denote as y [n] the vector of samples after the match filter Q H (ω) = F {q H (−t)} in the receiver and a symbol rate sampling, being q(t) = F −1 {Q(ω)} defined in (14). Now, we define the sampled impulse response at t = nT of the convolution of q(t) and its matched filter q H (−t), which represents the equivalent discrete channel response as: and its discrete Fourier transform pair as: Hence, the optimal discrete-time MIMO equalizer W opt (Ω) according to the MMSE criterion becomes: where we are considering a normalized transmission power equally distributed in each of the D modes. When G(Ω) satisfies the Nyquist criterion for MIMO systems G(Ω) = I, there is neither ISI nor cross-talk at the matched filter output, further equalization would not be needed, and the optimum linear receiver consists only in the matched filter. However, if such a criterion is not fulfilled, the equalizer W(Ω) is essential and some SNDR loss at the output will be unavoidable w.r.t. the ideal case.

Matched Filter-Based Receiver for SDM
In this subsection we will explore the optical channel requirements to reduce the linear MIMO receiver O(ω) in Figure 1 to a simple matched filter-based receiver. Furthermore, we will show that the resulting receiver is optimal in the sense that the discrete-time system response of the SDM communication system is the identity matrix, followed by the addition of the AWGN noise.
Let us first write out: where H tot (ω) and P(ω) are the Fourier transforms of h tot (t) and p(t), respectively and according to what is plotted in Figure 2. It follows that: The signal at each of the D branches y i (t), defined in (15), is processed before sampling by the continuous-time filter Q H (ω). The equivalent scheme for this matched filter-based MIMO receiver is shown in Figure 3a. By using the linearity of the system we can rearrange Figure 3a to obtain Figure 3b. Then, elaborating the expression Q(ω)Q H (ω) we obtain that: where we have used that: From (10) we have that: And the diagonal matrix: that does not allow simplifying (25) unless the following holds: This latter condition is equivalent to assuming that: or, in other words, that the MDL is negligible for the K amp -th span. When the condition expressed in (27) is satisfied for all the spans of the system, we can commute the terms in (25), and therefore, we can obtain: Revisiting (23), and plugging in (29) under the assumption of a negligible MDL in the optical channel, we can write: Therefore, without loss of generality, e ∑ Kamp k=1 g (k) = 1 can be assumed and, after sampling at the symbol rate, (30) can be written as: where we have used that p(t) defined in Equation (12) is a square-root raised cosine pulse satisfying the Nyquist criterion.
Regarding the filtered noise waveforms z 1 (t) to z D (t) in Figure 4a, they have an autocorrelation function matrix R zz (t), whose Fourier transform pair S z (ω) can be expressed as: where S n (ω) is the Fourier transform of the autocorrelation function matrix R nn (t) = I · N 0 2 · δ(t) of the received noise vector n(t) = [n 1 (t), n 2 (t), . . . , n D (t)] T . We remind that the noise components of the noise vector n(t) were assumed uncorrelated with identical power in each mode equal to N 0 /2. Using (30) and (32) leads to: Using the previous assumptions about gains g (k) and P(ω) made before, we obtain that the sampled noise vector z[n] = [z 1 [n], . . . , z D [n]] T has an autocorrelation matrix function R zz [n] = I · N 0 2 · δ[n]. Therefore, we can conclude that a D × D MIMO coherent optical communication system using a continuous-time matched filter as a receiver completely eliminates channel ISI and crosstalk when the MDL in the channel is negligible. Moreover, the equivalent discrete-time system model reduces to D discrete parallel AWGN channels as shown in Figure 4b. Hence, there would be no loss of performance w.r.t. the AWGN channel without distortion.

Numerical Simulation of Linear MIMO FSE Receiver for MDL-Impaired Optical Channel
In this section, we assess the performance of the ideal MMSE linear receiver when MDL is present. Specifically, we study the SNDR degradation at the receiver output w.r.t. the case when the MDL is negligible. To carry out this study, we will use a FSE-based receiver, as shown in Figure 5, which is the most common implementation of the ideal linear filter in discrete-time systems (see O(ω)in Figure 1). Note that this scheme is only valid for integer oversampling rates r ov . The FSE oversampling rate r ov has been set to two [1,12] and the discrete-time equalizer, with a W FSE (Ω) response, has been designed with a number of taps N taps large enough so that any further increase does not lead to a significantly better SNDR at the receiver output. The decimated output of W FSE (Ω), by a r ov factor, are the estimated symbolŝ[n].
We define the receiver performance metric for each mode i, L(i), as the difference in dB between the output SNR of an ISI and crosstalk-free system with D parallel AWGN channels (see Figure 4b), and the output SNDR of the FSE-based receiver, denoted as SNDR out .

Channel Model
We decide to carry out the numerical simulations of the H(ω) channel model described in (6) in the time domain, so that the relative delays of the different modes can be easily described as a time shift between them. The different mode amplitudes can also be handled simply by a diagonal matrix. The chromatic dispersion, which have a SISO frequency response H CD (ω) that does not depend on the mode i ∈ {0, . . . , D}, is represented as [7]: where β =β 2 tot , and tot = K amp span when all spans are considered of equal length. The MDL effect is modeled with an amplification factor for each mode and each optical amplifier (located at the end of each span). These factors are considered time-invariant for a given channel realization in the form of a vector for the k-th span g (k) = [g (k) . . , D} is expressed in dB and taken from a Gaussian distribution with zero mean and standard deviation (STD) σ g . The sum of all factors i is set to 0 for normalization purposes. Hence, the amplitudes matrix of the k-th span, frequency independent, is given by: Alternatively, for each span k of the communication link we have the delays matrix: being D ] the vector that models the MD with group delays for each mode of the k-th span.
To obtain the delays, we generate the first D/2 values of τ (k) from a Gaussian distribution with STD σ gd , and the second D/2 values are taken as the opposite of these, which = 0, since we consider that the system uses polarization multiplexing as part of the SDM [40].
The time-domain impulse response for each of the spans k is calculated by applying the inverse Fourier transform to (7) and can be expressed as: where and we have used that the matrices A (k) , U (k) , and V (k) are constant. Equation (37) describes that incoming signal at the kth span is multiplied by the unitary matrix U (k) , then each modal impulse response is delayed by τ (k) i , the amplification factor is set by the diagonal matrix A (k) and the mode-mixing unitary matrix V (k) is applied. Finally, the impulse response of the complete channel is given by: where h CD (t) = F −1 {H CD (ω)}. Note that, due to the random nature of g (k) and τ (k) in each k span, we can generate an arbitrary number N ch of channel realizations of H tot (ω) = F {h tot (t)} for a given value of σ g and σ gd .
Since we consider all the modes to be strongly coupled, the U (k) and V (k) matrices of each span k are modeled as unitary Gaussian random matrices obtained from a QR factorization of a complex random matrix whose elements have a zero mean and STD equal to 1. The two orthogonal matrices after QR factorization of two independent realizations of the random matrix are used as U (k) and V (k) , respectively.
We consider a total number of K amp = 100 spans, each span = 50 km long. For the fiber parameters, we used the multi-core fiber data reported in [41], considering the number of modes D = 6 and the central wavelength λ c = 1469 nm. The selection of this multi-core fiber allow us to compare the results of this work with those presented in [28], and to obtain the fiber parameters needed for the numerical simulations from [41].
We take 2% as the underestimation dispersion factor that is applied to the dispersion coefficient D CD to obtain the residual CD experienced by the receiver. For the gain STD σ g , we considered several values in the range of the systems referenced in [27]. For the numerical simulations, we compute a total of N ch = 10,000 realizations of the channel frequency response H tot (ω) defined in (5).

Transmitter and Linear MIMO FSE Receiver Parameters
As described in Section 4, the transmitter uses a generalized PAM modulation and an square-root raised cosine for pulse shaping with a roll-off factor equal to α. We will show results of the numerical simulation for several values of α. The symbol period T has been set for a symbol rate R s = 64 GBaud. The FSE-based MIMO receiver has an oversampling factor r ov = 2, and a number of equalizer taps N taps = 1000 has been selected to ensure that it does not limit the receiver performance for the considered channel MDL.

Signal-to-Noise at the Input of the Receiver
The signal-to-noise ratio at the input of each mode S N in The signal-to-noise at the input of the receiver SNR in in dB can be written as: and it is taken from the set of values in Table 1. P in (i) is the receiver input power in the mode i for the current channel realization.

Performance Loss Metric for FSE-Based MIMO Receiver
We define the performance loss metric (in dB) of the FSE-based MIMO receiver in MDL-impaired channels for certain mode i as: where SNR (in) (i) = 10 · log 10 S N in (i), and SNDR out = [SNDR out (1), SNDR out (2), . . . , SNDR out (D)] T is calculated as defined in [42] and Equation (28) in [43] for MIMO implementation of the FSE. Given a set of system model parameters, the numerical simulation will generate a total of D · N ch values of L(i). The average loss AL is calculated for each channel realization of among the available N ch as: Two FSE-based MIMO receiver performance metrics can be extracted from the D · N ch calculated values of L(i): • ML 95 is defined as the 95th percentile of the L(i) distribution obtained for any optical channel realization and mode; • AML 95 is defined as the 95th percentile of the AL distribution obtained for any optical channel realization.
The values for the parameters used in the simulation are summarized in Table 1.

Numerical Simulation Results
The first result is focused on the impact of PAM pulses roll-off factor α and MDL level, represented by σ g , on the SDM optical system performance. Figure 6 shows that for systems with transmitters using a higher α, the degradation is a bit lower. The effect is higher with increasing σ g for systems working at a high regime of SNR in , as seen in Figure 6b. In practical systems, the allowable loss of SNDR out in a channel with elements introducing MDL w.r.t. an ideal channel without MDL is around 1-2 dB. We can observe that, assuming a maximum degradation of 2 dB in the system with a 95% confidence, σ g values of the amplifiers should not exceed 0.2 dB. These results are in agreement with the capacity limits of a MIMO MMSE receiver and MDL channel calculated in [27].
A second result is presented in Figure 7, where the probability distribution of AL and SNR in estimated from the analysis of all channel realizations is plotted. We are comparing different levels of SNR regimes, with SNR in = 5, 6.2, and 10 dB (Figure 7a) and SNR in = 15, 30, and 60 dB (Figure 7b), for a roll-off factor of α = 0.9. The upper and lower limits of the blue boxes represent the 25th and 75th percentiles respectively. The red line inside the box indicates the median of the metric. In case the distribution of values was Gaussian, the whisker bounds correspond to 2.7 times the STD of the metric or, in other words, the number of values between the upper and lower bounds of the whiskers contains 99.3% of the values. Values outside these limits are considered outliers and are individually represented by red crosses.
We make the following observations from Figure 7: • The distribution of AL is not Gaussian, as we can observe by comparing the difference between upper and lower outliers for higher σ g values and their asymmetry; • There are no negative values of AL, since the FSE MIMO receiver cannot improve on average the SNR in . However, by taking all values of L(i) for any received mode i, we can find that, for certain channels and modes, the FSE MIMO receiver can locally improve the SNR in (i) of a particular mode i, but always at the cost of another mode of the receiver; • The performance degradation depends on the SNR in . In a low SNR in regime (5 dB, Figure 7a), the degradation is measured lower in absolute values when compared to the high SNR in regime (60 dB, Figure 7b); • The STD of the performance degradation also depends on the SNR in . In the low SNR in regime (5 dB, Figure 7a), the STD of the degradation is lower when compared to the high SNR in regime (60 dB, Figure 7b); • The performance degradation measured as AL is milder than measured as L(i) when more than 95% coverage of the channels is considered. Note that the difference is negligible when median values are taken into account.  (42) (down) for SNR in = 5, 6.2, and 10 dB (a) and SNR in = 15, 30, and 60 dB (b). α = 0.9 for all graphs. Note that σ g = 0 corresponds to a channel without MDL.

Conclusions
This work explored long-haul fiber-optic SDM coherent systems with PAM raisedcosine pulses. We investigated the overall system performance under different configurations of the optical channel. For that purpose, we modeled this channel with a MIMO multi-span structure that included the several dispersion terms and modal losses factors.
It was demonstrated that the linear MMSE MIMO receiver completely eliminated ISI and crosstalk when the number of taps was sufficiently high and the optical channel was free of MDL. Moreover, the generic structure of the linear MMSE MIMO receiver could be simplified to a continuous-time matched filter and still retained the same properties. This observation paves the way for analog receivers that simply implement the matched filter of the optical channel, eliminating all channel impairments when MDL is negligible.
We also defined performance metrics to assess the losses of a linear MIMO receiver implemented using a MIMO FSE with an oversampling factor of two for an optical channel that exhibited significant MDL. We have shown that such loss depends on the transmitter PAM pulses roll-off factor and the SNR level at the receiver input. We also determined that the performance degradation could be limited by processing the D output modes together by averaging the D output SNDR values at the receiver.
This fact opens a way to exploit this loss compensation between modes at the receiver output. The design of specific forward error correction codes taking into account this aspect could improve the final performance of the system in terms of bit error rate. Constructing the message to be encoded, including bits or signals belonging to all modes, could improve system performance w.r.t. constructing messages with bits or signals from only one mode.