Dispersion Compensation Scheme with a Simple Structure in Ultra-High-Speed Optical Fiber Transmission Systems

Abstract

With the explosive growth of global data traffic, long-distance fiber optic transmission systems are continuously evolving towards higher capacity and longer distances. However, to overcome the high complexity of fiber dispersion compensation algorithms, various dispersion compensation techniques have emerged. This paper aims to systematically review and summarize dispersion compensation algorithms in long-distance fiber optic transmission. First, we briefly introduce the physical mechanism of fiber dispersion. Then, this paper focuses on digital domain compensation algorithms, dividing them into two major categories: compensation algorithms without penalty and with penalty. For compensation algorithms without penalty, we elaborate on traditional block processing strategies such as Overlap-Save (OLS), and various enhanced strategies combining intelligent filter segmentation and optimized frequency domain workflows. For compensation algorithms with penalty, we focus on analyzing a scheme that redesigns chromatic dispersion compensation (CDC) algorithm into a hardware-friendly structure using geometric clustering of taps, and finite-impulse-response (FIR) filters based on frequency response approximating the ideal inverse chromatic dispersion (CD) transfer function. By numerical simulation, we analyze the core principles, computational complexity, and compensation performance of each type of algorithm. Finally, this paper summarizes the limitations and development trends of existing dispersion compensation algorithms, pointing out that low-complexity and small-scale deployment algorithm structures will be an important research direction in the future.

1. Introduction

Long-haul coherent optical fiber transmission systems form the core of the physical layer of modern high-capacity communication networks and are one of the key technologies supporting global digital interconnection. These systems are widely used in critical scenarios such as the global internet backbone, transoceanic submarine optical cables, large enterprise data center interconnections, and national telecommunications backbone networks, undertaking the task of high-speed, stable, and long-distance data transmission [1].

From metropolitan area networks and core network links spanning hundreds of kilometers to ultra-long-distance intercontinental connections, these systems cover communication needs from regional to global, possessing high speed, large capacity, low bit error rate, and strong anti-interference capabilities, making them an indispensable infrastructure for modern information society. Especially against the backdrop of continuously increasing data traffic and the rapid development of emerging applications such as cloud computing and artificial intelligence, the performance and reliability of long-haul optical fiber systems have become key factors in ensuring global communication quality [2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17]. Although wireless communication and access network technologies have made significant progress in recent years [18,19,20,21,22,23,24,25], long-haul optical fiber transmission systems still play an irreplaceable role in actual deployment environments. Its advantages in transmission distance, bandwidth capacity, and stability have kept it at the core of the global communication system. This situation further highlights the urgency and importance of continuous innovation in optical signal processing, modulation and demodulation technologies, and system architecture optimization. To meet the demands of future communication systems for higher speeds, lower energy consumption, and greater robustness, it is essential to continuously drive breakthroughs and evolution in optical communication technology at the levels of algorithm design, device performance, and system integration [26,27,28,29,30].

In fiber optic communication systems, chromatic dispersion is a typical static linear impairment that significantly impacts the transmission quality of high-speed optical signals. The essence of dispersion lies in the frequency response characteristics of the fiber material; that is, optical signals of different frequencies do not propagate at completely uniform speeds within the fiber. This frequency dependence causes the optical pulse to broaden during propagation, similar to the multipath effect in wireless communication, thus inducing time-domain distortion of the signal [31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46].

Specifically, chromatic dispersion in optical fibers consists of two main parts: material dispersion and waveguide dispersion. Material dispersion originates from the physical characteristic of the fiber’s refractive index varying with wavelength, while waveguide dispersion is related to the fiber’s geometry, such as the core-to-cladding ratio and refractive index distribution. In single-mode fibers, these two types of dispersion jointly determine the propagation characteristics of the optical signal, especially in long-distance transmission or high-speed systems, where the dispersion effect is particularly significant [47,48,49,50,51,52,53].

Dispersion has multifaceted effects on system performance. First, it causes the optical pulse to broaden along the time axis, resulting in inter-symbol interference (ISI), making it difficult for the receiver to accurately determine the signal, thereby increasing the bit error rate. Secondly, dispersion can blur eye diagrams, cause clock components to disappear, and in severe cases, even lead to complete signal distortion. Therefore, in practical system design, effective dispersion compensation is essential to ensure communication quality [54,55,56,57,58,59,60,61].

Traditional dispersion compensation methods primarily rely on optical means. For example, in early direct modulation-direct detection (IM-DD) systems, dispersion-compensating fibers (DCF) or fiber gratings (FBG) were commonly used for compensation. These devices typically have negative dispersion coefficients, which can cancel out the positive dispersion of ordinary fibers, thus achieving overall dispersion balance. Furthermore, photonic crystal fibers (PCF) have been widely studied due to their tunable dispersion characteristics and are used to construct transmission links with specific dispersion curves [62,63].

With the development of digital coherent optical communication technology, digital signal processing (DSP) has gradually become the mainstream method for dispersion compensation. Compared to optical compensation, DSP offers greater flexibility and programmability. By modeling the fiber transfer function, reverse compensation in the time or frequency domain can be achieved at the receiver. For example, in the frequency domain, the Fast Fourier Transform (FFT) can be used to convert the signal to the frequency domain, phase compensation can be applied, and then the inverse Fourier Transform (IFFT) can be used to recover the time-domain signal, thereby effectively eliminating phase distortion caused by dispersion [64,65,66].

In summary, chromatic dispersion, as a key impairment mechanism in optical fiber communication, has undergone an evolution in compensation techniques from optical devices to digital algorithms. In modern optical communication systems, dispersion compensation is not only a necessary means to improve system performance but also an important guarantee for promoting transmission rate increases and system capacity expansion. In the future, as optical communication develops towards higher bandwidth and longer distances, dispersion compensation technology will continue to be optimized and deeply integrated with other signal processing technologies to build a more efficient and stable transmission system.

The remaining sections of this work are structured as follows. Section 2 presents the physical mechanism of fiber dispersion and its negative impact on transmission systems. Section 3 presents compensation algorithm without penalty. Section 4 presents Compensation algorithm with penalty. Section 5 presents the paper’s conclusion at the end.

2. The Principle of Dispersion

The transmitted signals in optical fibers contain different frequency or mode components. After passing through the optical fibers, the signal pulses will broaden due to the different group velocities, resulting in signal distortion. This physical phenomenon is called dispersion. From a mechanism perspective, optical fiber dispersion is divided into material dispersion, waveguide dispersion, and mode dispersion. The first two types of dispersion occur because the signals are not of a single frequency, while the latter type of dispersion occurs because the signals are not of a single mode. The widespread existence of optical fiber dispersion causes signal pulse distortion during transmission, thereby limiting the transmission capacity and bandwidth of optical fibers. For single-carrier coherent optical communication systems, both the non-optical compensation method using DCF and the digital signal processing algorithm can achieve compensation for optical fiber dispersion. However, the latter method has a lower implementation cost and higher tolerance for optical nonlinear effects compared to the former [67]. Therefore, the optical fiber dispersion compensation module is an indispensable part of the digital signal processing process [67,68,69,70,71,72,73,74,75,76].

Assuming that after being processed by the front-end orthogonalization and normalization module as well as the clock synchronization module, the amplitude and phase mismatches in the two polarization directions have been fully compensated and corrected, and the sampling clocks at the transmitter and receiver ends are completely synchronized. On this basis, efforts will be made to compensate for the insertion loss introduced by the optical fiber (including linear and nonlinear losses). Here, to simplify the structure of the coherent receiver, the nonlinear losses of the optical fiber (such as self-phase modulation, nonlinear phase noise) are temporarily ignored, and only the linear losses of the optical fiber such as chromatic dispersion and polarization mode dispersion (PMD) are considered. Under typical operating conditions with launch powers below 11 dBm, the impact of nonlinear effects on system performance is relatively small, which justifies the use of a linear dispersion compensation model. However, at higher launch powers, longer transmission distances, or when employing higher-order modulation formats, nonlinear effects can no longer be neglected. In such cases, the performance of dispersion compensation algorithms may degrade, and joint processing with nonlinear equalization may be required.

After determining the adoption of the double-layer structure, the following will elaborate on the compensation algorithm from the perspective of the essence of the fiber dispersion effect. Firstly, from the physical properties of dispersion, when light, as an electromagnetic wave, interacts with the bound electrons in the dielectric, the response of the medium is usually related to the frequency $\omega$ of the light wave. This indicates the dependency of the refractive index $n(\omega )$ on the frequency. Mathematically, the dispersion effect of the fiber can be described by expanding it into a Taylor series of the propagation constant $\beta$ at the central frequency ${\beta }_0$, as shown in Equation (1):

$$\beta (\omega )=n(\omega )\cdot {\displaystyle \frac{\omega }{c}}={\beta }_0+{\beta }_1(\omega -{\omega }_0)+{\displaystyle \frac{1}{2}}{\beta }_2{(\omega -{\omega }_0)}^2+\cdots , {\beta }_m={\left({\displaystyle \frac{d^m\beta }{d{\omega }^m}}\right)}_{\omega ={\omega }_0}(m=0,1,2,\dots )$$

(1)

The parametric ${\beta }_1$, ${\beta }_2$ is related to the refractive index $n$, and their relationship is as follows:

$${\beta }_1={\displaystyle \frac{n_g}{c}}={\displaystyle \frac{1}{v_g}}={\displaystyle \frac{1}{c}}(n+\omega {\displaystyle \frac{dn}{d\omega }}) {\beta }_2={\displaystyle \frac{1}{c}}(2{\displaystyle \frac{dn}{d\omega }}+\omega {\displaystyle \frac{d^{2}n}{d{\omega }^2}})$$

(2)

where $n_g$ is the group refractive index and $v_g$ is the group velocity, the envelope of the light pulse moves at the group velocity. The parameter ${\beta }_2$ represents group velocity dispersion and is related to pulse broadening. This phenomenon is called group velocity dispersion (GVD), and the parameter ${\beta }_2$ is the GVD parameter.

The dispersion coefficient $D$ is generally used to quantify the degree of pulse broadening caused by fiber dispersion, with units of ps/nm/km. The dispersion coefficient $D$ gives the degree of pulse broadening $\Delta T$ (in ps) of a 1 nm bandwidth optical signal pulse transmitted through 1 km of fiber. It can be expressed by the following formula:

$$\Delta T=D\cdot \xi \cdot L$$

(3)

Meanwhile, the dispersion coefficient $D$ and the GVD parameter ${\beta }_2$ satisfy the following relationship:

$$D={\displaystyle \frac{d{\beta }_1}{d\lambda }}=-{\displaystyle \frac{2\pi c}{{\lambda }^2}} {\beta }_2\approx -{\displaystyle \frac{\lambda }{c}}{\displaystyle \frac{d^{2}n}{d{\lambda }^2}}$$

(4)

Reference [71] gives the nonlinear Schrödinger equation for propagation within a single-mode optical fiber, as follows:

$$j{\displaystyle \frac{\partial A}{\partial z}}=-j{\displaystyle \frac{\alpha }{2}}A+{\displaystyle \frac{{\beta }_2}{2}}{\displaystyle \frac{{\partial }^{2}A}{\partial T^2}}-\gamma |A{|}^{2}A$$

(5)

where $A$ is the slowly varying amplitude of the pulse envelope, and $T=t-z/v_g$ is the time measure that moves with the pulse at a group velocity $v_g$ in the reference frame. The three terms on the right-hand side of Equation (5) correspond to the absorption, dispersion, and nonlinear effects of the optical pulse propagating in the optical fiber, respectively. The initial pulse width $T_0$ and peak power $P_0$ determine whether dispersion or nonlinearity plays a dominant role in the pulse’s propagation in the optical fiber. Here, we ignore the nonlinear effects of the optical fiber and assume that dispersion plays a dominant role, introducing a normalized time measure for the initial pulse width $T_0$:

$$\tau ={\displaystyle \frac{t-\frac{z}{v_g}}{T_0}}$$

(6)

At the same time, the normalized amplitude $U$ is introduced using the following definition:

$$A(z,t)=\sqrt{P_0}\cdot e^{-\alpha z/2}\cdot U(z,\tau )$$

(7)

The exponential factor in the formula reflects the fiber loss. Using Equations (5)–(7) and ignoring the nonlinear term in the latter part of the equation, the normalized amplitude $U(z,\tau )$ of the optical pulse should satisfy

$$j{\displaystyle \frac{\partial U(z,\tau )}{\partial z}}={\displaystyle \frac{{\beta }_2}{2T_0^2}}{\displaystyle \frac{{\partial }^{2}U(z,\tau )}{\partial {\tau }^2}}$$

(8)

Substituting Equation (4) into the above equation and taking $T_0=1$, we obtain

$${\displaystyle \frac{\partial U(z,\tau )}{\partial z}}=-j{\displaystyle \frac{D{\lambda }^2}{2\pi c}}{\displaystyle \frac{{\partial }^{2}U(z,\tau )}{\partial {\tau }^2}}=j{\displaystyle \frac{D{\lambda }^2}{4\pi c}}{\displaystyle \frac{{\partial }^{2}U(z,\tau )}{\partial {\tau }^2}}$$

(9)

The above theoretical derivation yields the partial differential equation for the influence of fiber dispersion on the signal envelope $U(z,\tau )$, which is also the basis of all fiber dispersion equalization algorithms. Here, $z$ represents the transmission distance, $\tau$ represents the normalized time parameter, $D$ represents the fiber dispersion coefficient, $\lambda$ represents the wavelength of the light wave, and $c$ represents the speed of light.

A direct solution to the partial differential Equation (9) is to perform a FFT on it to obtain the frequency domain transmission equation $G(z,\omega )$:

$$G(z,\omega )=\exp \left(-j{\displaystyle \frac{D{\lambda }^2}{4\pi c}}{\omega }^2\right)$$

(10)

where $\omega$ represents any frequency component.

As can be seen from Equation (10), the all-pass filter $1/G(z,\omega )$ can be approximated by a digital filter with an infinite impulse response (IIR) recursive structure [72] or an FIR non-recursive structure [73], thus achieving direct compensation for dispersion in the frequency domain. Although the number of filter taps required for dispersion compensation using an IIR filter is much smaller than that of an FIR filter, the inherent recursive feedback structure of the IIR filter makes it almost impossible to implement in high-speed parallel signal processing. At the same time, it is difficult to design an IIR filter that fully meets the conditions based on the phase response as shown in Equation (10). Therefore, FIR filters are generally used to compensate for fiber dispersion in the frequency domain.

In addition to the direct dispersion equalization in the frequency domain as described above, a further Fourier transform of Equation (10) easily yields the time-domain impulse response, as shown below:

$$g(z,t)=\sqrt{{\displaystyle \frac{c}{jD{\lambda }^{2}z}}}\exp \left(j{\displaystyle \frac{\pi c}{D{\lambda }^{2}z}}{t}^2\right)$$

(11)

Therefore, a filter that matches the impulse response can also be designed in the time domain to perform time-domain equalization of fiber dispersion.

From an algorithmic implementation perspective, frequency domain equalization appears more intuitive and easier to implement. By inverting the sign of the dispersion coefficient D in the dispersion frequency domain transfer function given in Equation (10), the frequency domain transfer function of the dispersion frequency domain compensation filter is obtained:

$$G(z,\omega )=\exp \left(j{\displaystyle \frac{D{\lambda }^2}{4\pi c}}{\omega }^2\right)$$

(12)

3. Compensation Algorithm Without Penalty

In long-haul optical fiber transmission systems, dispersion compensation is commonly realized using large-scale digital filters to recover signal integrity. However, direct implementation of conventional FIR structures entails formidable challenges, including prohibitive computational complexity, elevated power consumption, and substantial hardware resource requirements, which collectively impede real-time processing feasibility. To address these limitations, frequency-domain filtering—most notably architectures leveraging the FFT—has emerged as the dominant solution. Nevertheless, for high-speed and long-distance transmission scenarios, the large signal bandwidths and long processing windows significantly complicate real-time frequency-domain operations. The corresponding FFT size requirements also intensify hardware constraints, a problem that becomes particularly critical in resource-limited or cost-sensitive deployment environments.

Consequently, block-processing strategies such as OLA and OLS methods have been extensively adopted in practical systems. By partitioning long input sequences into manageable segments, these schemes alleviate direct FFT size restrictions, yet still incur non-negligible resource overhead due to block overlaps and repeated transform operations.

In recent years, a variety of enhanced strategies combining intelligent filter segmentation with optimized frequency-domain workflows have been proposed to overcome these bottlenecks. For instance, Ref. [56] introduced the vanilla OLS-based segmented filter (VOSF) and low-complexity OLS-based segmented filter (LOSF) frameworks, which emphasize efficient filter partitioning, whereas Ref. [77] proposed the time-segmented overlap-free (TS-OLF) scheme to eliminate inter-block overlap entirely. These approaches substantially reduce hardware resource utilization while maintaining dispersion-compensation accuracy, thereby offering practical, scalable, and energy-efficient solutions for real-time high-speed optical fiber communication systems.

3.1. OLA

To apply a dispersion-compensating filter $h\left[n\right]$ of length $L_h$ to a discrete-time signal $x\left[n\right]$ of length $L_x$, the theoretically ideal dispersion-compensated output is obtained via the linear convolution of the two sequences.

$$y[n]=x[n]\times h[n].$$

(13)

However, direct computation of long linear convolutions is computationally intensive. To address this, the OLA method offers an efficient alternative by partitioning the long input sequence into non-overlapping blocks and performing FFT-based convolution on each block independently, as shown in Figure 1.

Let the input sequence x[n] be divided into N non-overlapping segments:

$$P=⌈{\displaystyle \frac{L_x}{N}}⌉,$$

(14)

$$x_i[m]=x[m+iN], for m=0,1,\dots ,N-1,$$

(15)

where $i=0,1,\dots ,P-1$, and $P$ is the number of signal blocks. By exploiting the computational efficiency of the FFT, the linear convolution for each segment is reformulated as a circular convolution. To avoid time-domain aliasing inherent in circular convolution, both the signal segment $x_i[m]$ and the filter $h[m]$ are zero-padded to a length $B\ge N+L_h-1$:

$$x_i^{\prime }[m]=\left\{\begin{array}{ll}{x}_i[m],& 0\le m\le N-1\\ 0,& N\le m\le B-1\end{array}\right.,$$

(16)

$$h^{\prime }[m]=\left\{\begin{array}{ll}h[m],& 0\le m\le L_h-1\\ 0,& L_h\le m\le B-1\end{array}\right..$$

(17)

The circular convolution is then computed in the frequency domain:

$$Y_i[k]=\mathrm{FFT}\left\{x_i^{\prime }\right\}[k]\cdot \mathrm{FFT}\left\{h^{\prime }\right\}[k], k=0,1,\dots ,\mathrm{B}-1,$$

(18)

$$y_i[m]=\mathrm{IFFT}\{Y_i[k]\}[m].$$

(19)

Due to adequate zero padding, the resulting circular convolution is equivalent to the desired linear convolution. The final output signal is synthesized by appropriately delaying and summing the individual segment outputs:

$$y[n]=\sum _{i=0}^{P-1}{y}_i[n-iN],$$

(20)

note that adjacent segments have an overlap length of $\mathrm{B}-\mathrm{N}$. These overlapping samples must be coherently summed to ensure continuity and correctness of the reconstructed output.

3.2. OLS

The OLS method performs block-based filtering by segmenting the input signal into overlapping blocks. It utilizes circular convolution in the frequency domain and discards the aliased samples at the beginning of each output block, retaining only the valid portions which are subsequently concatenated to reconstruct the final output signal., as shown in Figure 2.

The input sequence $x[n]$ is partitioned into overlapping segments of length $B$, where each adjacent segment shares $L_h-1$ points, with $L_h$ denoting the length of the filter. The $i$-th segment is defined as

$$x_i[m]=x[m+i(B-L_h+1)], 0\le m\le B-1,$$

(21)

the first segment $x_0[m]$ is zero-padded at the beginning by $L_h-1$ points. The filter $h[m]$ is similarly zero-padded to match the segment length $B$:

$$h^{\prime }[m]=\left\{\begin{array}{ll}h[m],& 0\le m\le L_h-1\\ 0,& L_h\le m\le B-1\end{array}\right..$$

(22)

The circular convolution between each zero-padded segment and the filter is computed in the frequency domain as

$$Y_i[k]=\mathrm{FFT}\left\{x_i^{\prime }\right\}[k]\cdot \mathrm{FFT}\left\{h^{\prime }\right\}[k],$$

(23)

$${\tilde{y}}_i[m]==\mathrm{IFFT}\left\{Y_i[k]\right\}[m].$$

(24)

Due to the nature of circular convolution, the first $L_h-1$ samples of ${\tilde{y}}_i[m]$ are corrupted by aliasing and must be discarded. The remaining $B-L_h+1$ samples constitute the valid linear convolution output:

$$y_i[m]={\tilde{y}}_i[m+L_h-1], 0\le m\le B-L_h.$$

(25)

The final output sequence $y[n]$ is synthesized by concatenating the valid outputs $y_i[m]$ from all segments:

$$y[n]=OLS(x,h)[n]=\sum _{i=0}^{⌈{\displaystyle \frac{L_x}{B-L_h+1}}⌉}{y}_i[n-iB].$$

(26)

3.3. LOSF

The implementation of large FIR filters is often constrained by the computational limits of the FFT and the structural requirements inherent to the OLS method. To address these constraints, the VOSF method strategically divides large filters into multiple shorter, more manageable sub-filters. This segmentation ensures each sub-filter remains within permissible FFT size constraints, enabling efficient frequency-domain processing.

The partitioning of the original dispersion compensation filter is accomplished by dividing it into $K$ contiguous sub-filters, each of length $M$. The number of segments $K$ is determined by the ratio of the original filter length $L_h$ to the segment length $M$, computed via the ceiling function:

$$K=⌈{\displaystyle \frac{L_h}{M}}⌉,$$

(27)

the $b$-th sub-filter, where $b=0,1,\dots ,K-1$, is defined by extracting the corresponding coefficients from the original filter $h[n]$:

$$h_b[m]=h[m+bM], m=0,1,\dots ,M-1.$$

(28)

Based on this segmentation scheme, the original filter $h[n]$ can be mathematically reconstructed as the sum of all sub-filters, each delayed by $bM$ samples:

$$h[n]=\sum _{b=0}^{K-1}{h}_b[n-bM],$$

(29)

when the full-length filter $h[n]$ is applied to the input signal $x[n]$, and the decomposition expression is substituted, the resulting output $y[n]$ is obtained by convolving the input with each sub-filter independently, followed by an appropriate time-domain shift:

$$y[n]=(x\times h)[n]=\sum _{b=0}^{K-1}(x\times h_b)[n-bM],$$

(30)

this formulation reveals two key operational characteristics of the decomposed filtering approach: (1) Independent Convolution Operations: Each sub-filter $h_b$ performs a linear convolution with the input signal $x[n]$ independently, enabling parallel processing and significantly improving computational efficiency; (2) Time-Domain Shifting of Partial Results: The result of each independent convolution $x\times h_b$ must be delayed by $b\cdot M$ samples to account for the relative position of each sub-filter within the original filter structure.

To further elucidate the combination process, the time-shifted output of the $k$-th branch is defined as

$$q_b[n]=OLS(x,h_b)[n-b\cdot M].$$

(31)

Accordingly, the overall filter output $y[n]$ is expressed as the summation of these branch outputs:

$$y[n]=\sum _{b=0}^{K-1}{q}_b[n].$$

(32)

The architecture of VOSF is illustrated in Figure 3a. The processing workflow comprises two parallel segmentation stages: the input signal $x[n]$ is divided into overlapping blocks of length $N$, suitable for OLS-based convolution, while the original filter $h[n]$ is partitioned into $K$ sub-filters of length $M$. Each sub-filter is convolved with the incoming signal blocks within its dedicated OLS processing unit. Finally, the outputs from all OLS units are temporally aligned via the prescribed time-domain shifts and summed to produce the final output signal $y[n]$, thereby realizing the operation described in Equation (32). This architecture offers a scalable and computationally efficient solution for implementing long FIR filters, effectively overcoming the practical limitations associated with FFT-based convolution techniques.

In conventional implementations, each filter bank within the VOSF architecture requires dedicated FFT and IFFT modules to perform frequency-domain convolution. This multi-branch configuration, however, substantially increases system complexity and constrains efficiency, particularly in high-frequency or real-time signal processing scenarios. To alleviate these limitations, LOSF introduces a structurally optimized framework in which all filter segments share a single unified FFT/IFFT module. This design significantly reduces the computational burden while preserving the accuracy and performance of the filtering operation.

Expanding the OLS processing unit c associated with the $i$-th sub-filter, the output corresponding to the $i$-th signal block can be expressed as

$$q_{i,b}[n]=(IFFT\left\{X_i[k]\cdot H_b[k]\right\}\}\cdot u)[n-b\cdot M],$$

(33)

$$u[n]=\left\{\begin{array}{l}0, n=0,1,\dots ,M-1\hfill \\ 1, else\hfill \end{array}\right.,$$

(34)

where the time-domain shift associated with the $b$-th filter segment is denoted by $b\cdot M$.

Although the core IFFT operation is independent of the filter segment index $k$, the required time-domain shifts vary across segments. This variation previously precluded the direct reuse of a single IFFT module across all branches, as implied by Equation (33). To overcome this challenge, we introduce a frequency-domain phase compensation strategy that effectively transfers the time-shift operation into the frequency domain. Specifically, by embedding the shift condition into the frequency-domain expression prior to the IFFT, we obtain

$$\left.q_{i,b}[n]=\left(IFFT\left\{X_i[k]\cdot H_b[k]\cdot {\mathsf{\Phi }}_b[k]\right\}\right\}\cdot u\right)[n].$$

(35)

The phase rotation factor ${\mathsf{\Phi }}_b[k]$ is defined as

$${\mathsf{\Phi }}_b[k]=\exp \left(-j{\displaystyle \frac{2\pi kbM}{B}}\right).$$

(36)

Let $\Delta B$ denote the time interval between the starting points of two consecutive input blocks. By applying the shift property of the Fourier Transform to the blocked input signal, we derive the condition for achieving correct alignment:

$$X_{i-b}[k]=X_i[k]\cdot e^{-j{\displaystyle \frac{2\pi kb\cdot \Delta B}{B}}}.$$

(37)

The $\Delta B$ is defined as the difference between the block length $B$ and the overlap length $OL$: $\Delta B=B-OL$. To achieve $\Delta B=M$, the necessary overlap length $OL=M$. This leads to

$$M={\displaystyle \frac{B}{2}},$$

(38)

this implies that the overlap length must be 50% of the total block length $B$, ensuring that the time-domain displacement is fully captured within the frequency-domain representation. Based on this configuration, the output for the sub-filter output can be refined as

$$q_{i,b}[n]=(\mathrm{IFFT}\left\{X_{i-b}[k]\cdot H_b[k]\right\}\}\cdot u)[n],$$

(39)

this formulation demonstrates that the required time-domain shift for each sub-filter can be equivalently implemented via frequency-domain phase rotation, thereby enabling all filter segments to share a single IFFT module.

By summing the outputs from all filter segments as prescribed in Equation (39), the total output for the $i$-th block is obtained:

$$\begin{array}{cc}\hfill y_i[n]& =\sum _{i=0}^{K-1}{q}_{i,b}[n]\hfill \\ \hfill & =\left(IFFT\left\{\sum _{i=0}^{P-1}{X}_{i-b}[k]\cdot H_b[k]\right\}\cdot u\right)[n]\hfill \end{array}.$$

(40)

Finally, the complete output signal $y[n]$ is synthesized by concatenating the valid, non-overlapping sections of all output blocks $y_i[n]$:

$$y[n]=\sum _{i=0}^{⌈{\displaystyle \frac{2L_x}{B}}⌉}{y}_i[n-iB].$$

(41)

As illustrated in Figure 3b, the LOSF effectively alleviates the computational bottleneck associated with traditional multi-branch OLS implementations. By combining filter segmentation with a 50% block-overlap strategy, LOSF departs from the independent processing paradigm of the VOSF and significantly enhances resource sharing. For each incoming input block, its frequency-domain representation is phase-shifted and multiplied by each filter sub-segment. The resulting products are aggregated in the frequency domain and transformed back to the time domain via a single IFFT. This approach replaces the multiple IFFT computations and explicit time-domain shifts required in VOSF with a unified and efficient frequency-domain operation, thereby achieving substantial reductions in computational complexity. Consequently, LOSF offers a scalable and high-performance solution for large-scale FIR filtering tasks.

3.4. TS-OLF

The TS-OLF method constitutes an advanced frequency-domain block filtering framework, specifically designed to overcome the inherent limitations of conventional overlap-based filtering strategies in hardware-constrained environments. The core principle of TS-OLF lies in eliminating overlapping regions between signal blocks and representing the filter in segmented time-domain form. This formulation decomposes large-scale filtering tasks into manageable subproblems, thereby improving computational tractability. In addition, TS-OLF incorporates an error compensation mechanism and performs time-shifting and accumulation operations directly on the frequency-domain outputs of individual filter bands. Such a design enables filtering across arbitrary FFT sizes, ensuring both high flexibility and implementation efficiency. A schematic illustration of the proposed approach is presented in Figure 4.

First, the input signal $x[n]$ is partitioned into consecutive non-overlapping blocks of length $N$:

$$x[n]=\sum _{i=0}^{\infty }{x}_i[n-iN], where x_i[n]=\left\{\begin{array}{l}x[iN+n], 0\le n<N\hfill \\ 0, \mathrm{else}\hfill \end{array}\right..$$

(42)

Simultaneously, the filter $h[n]$ of length $L_h$ is partitioned into $M$ segments ($M=⌈{\displaystyle \frac{L_h}{N}}⌉$), each of length $N$. If the final segment does not fully occupy the designated length, it is zero-padded to ensure dimensional consistency:

$$h[n]=\sum _{m=0}^{M-1}{h}_m[n-mN], where h_m[n]=\left\{\begin{array}{l}h[mN+n], 0\le n<N\hfill \\ 0, \mathrm{else}\hfill \end{array}\right..$$

(43)

For each pair consisting of signal block $x_i[n]$ and filter segment $h_m[n]$, the desired result is their linear convolution $p_{im}[n]$. However, direct multiplication of their frequency-domain representations followed by an inverse transform yields the circular convolution, denoted as $p_{im}^{\prime }[n]$. The discrepancy between this circular convolution and the desired linear convolution is defined as the error signal $e_{im}[n]$:

$$e_{im}[n]=p_{im}^{\prime }[n]-p_{im}[n],$$

(44)

this error arises from boundary discontinuities between the current signal block $x_i[n]$ and the previous signal block $x_{i-1}[n]$. Its explicit formulation in the time domain can be derived accordingly:

$$e_{im}[n]=\sum _{j=n+1}^{N-1}{h}_m[j]\cdot (x_i[N-n+j]-x_{i-1}[N-n+j]).$$

(45)

To enable efficient computation in the frequency domain, the error signal is processed via an even–odd frequency decomposition. By defining the signal difference $\Delta x_i=x_i-x_{i-1}$, the frequency-domain representation of the error signal $E_{im}^{\prime }$, can be decomposed into its even component $E_{im,e}^{\prime }$ and odd component $E_{im,o}^{\prime }$:

$$\begin{gathered} E_{im,e}^{\prime }=P_{im}^{\prime }-P_{(i-1)m}^{\prime }, \\ {E}_{im,o}^{\prime }=H_{m,o}\cdot (F_N\times (\widehat{W}\times \Delta x_i)), \end{gathered}$$

(46)

here, $H_{m,o}=F_N\times (\widehat{W}\times h_m),$ and $\widehat{W}$ is a diagonal matrix formed by the twiddle factors $W={[w_{2N}^n]}_{n=0}^{N-1}$.

Subsequently, the required time-domain error $e_{im}[n]$ is obtained by extracting the latter half $e_{im,o}[n]$ from the frequency-domain error signal:

$$e_{im}={\displaystyle \frac{1}{2}}\left(P_{im}^{\prime }-P_{(i-1)m}^{\prime }-{\widehat{W}}^{\prime }(F_N^{-1}(E_{im,o}^{\prime }))\right).$$

(47)

After obtaining the error-corrected linear convolution block $p_{im}[n]$ for each segment, conventional approaches would necessitate a separate IFFT for every segment. In contrast, a key advantage of TS-OLF lies in the non-overlapping structure of the signal blocks: the output $p_m[n]$ of each filter segment requires only a time-shift by $mN$ samples in the final aggregated output, which corresponds to a phase shift in the frequency domain.

Consequently, the final $i$-th output block $y_i[n]$ can be synthesized by aggregating the contributions from all filter segments in the frequency domain, requiring only a single IFFT operation for the combined result:

$$y_i[n]={\displaystyle \frac{1}{2}}\left(I_i[n]+I_{i-1}[n]+e_{i,o}[n]\right),$$

(48)

where $I_i[n]={\mathrm{IFFT}}_N\left(\sum _{m=0}^{M-1}{H}_m\cdot X_{i-m}\right)$ aggregates the frequency-domain products of the current and previous signal blocks with all filter segments, requiring only one IFFT; $I_{i-1}[n]$ denotes the processing result from the previous signal block, which has already been computed and stored; $e_{i,o}[n]={\widehat{W}}^{\prime }\cdot {\mathrm{IFFT}}_N\left(\sum _{m=0}^{M-1}{E}_{(i-m)m,o}^{\prime }\right)$ accumulates the odd-frequency error components from all filter segments, also requiring only one IFFT operation.

3.5. Complexity Analysis

To assess the computational requirements of the considered filtering schemes, we adopt the standard analytical model in which an FFT (or IFFT) of length $B$ incurs $B\log B$ complex multiplications, as dictated by the radix-2 Cooley–Tukey framework. Given that overlap–add processing generally entails substantially greater arithmetic cost, the subsequent analysis concentrates on the more computationally efficient overlap–save–based approaches.

Although the classical OLS framework is not directly deployable for large-scale dispersion compensation due to system-level constraints, its complexity serves as a reference for evaluating the segmented variants. Each OLS block requires one FFT, one IFFT, and a frequency-domain multiplications. For a signal of length $L_x$, block size $N$, filter length $L_h$, and overlap length $L_h-1$, leading to a normalized per-sample complexity:

$$C_{\mathrm{OLS}}={\displaystyle \frac{N}{N-L_h+1}}\cdot (2\log N+1).$$

(49)

In the VOSF scheme, the filter is decomposed into ${\displaystyle \frac{L_h}{M}}$ subfilters of length $M$, each handled by a separate OLS unit. The resulting normalized complexity is

$$C_{\mathrm{VOSF}}={\displaystyle \frac{L_h}{M}}\cdot C_{OLS}={\displaystyle \frac{L_h\cdot N}{M(N-M+1)}}(2\log N+1).$$

(50)

The LOSF method integrates shared frequency-domain operations across filter segments, thereby reducing FFT/IFFT requirements. With a 50% block overlap ($M={\displaystyle \frac{N}{2}}$), the normalized complexity becomes

$$C_{\mathrm{LOSF}}={\displaystyle \frac{2}{N}}\cdot \left(2N\log N+2{\displaystyle \frac{L_h}{N}}\cdot N\right)=4\left(\log N+{\displaystyle \frac{L_h}{N}}\right).$$

(51)

The summary of computational complexity analysis indicates that the LOSF method shares the same complexity as the TS-OLF scheme. Furthermore, for clarity, Figure 5 compares the computational complexities of OLS, VOSF, and LOSF.

Figure 5a quantifies the advantage of sharing IFFTs in the LOSF architecture by plotting the complexity improvement of LOSF relative to VOSF across different filter lengths for two FFT sizes. The improvement rate increases with the number of filter taps and gradually converges. Figure 5b provides a comparative assessment of the computational complexity of the evaluated methods as a function of FFT size for two filter configurations (512 and 1024 taps). When the FFT size is smaller than the filter length, the conventional OLS method becomes inapplicable, whereas the LOSF framework remains fully operational due to its segmented processing structure. In contrast, when the FFT size exceeds the filter length, the OLS algorithm demonstrates a relative advantage in terms of computational cost.

4. Compensation Algorithm with Penalty

In modern optical coherent systems, power consumption has become a bottleneck where DSP occupies half of the total transceiver power. Among them, CDC is one of the most energy-hungry DSP blocks [78]. Conventional complexity-reduction techniques for CDC, such as chirp filtering or uniform quantization of FIR taps, can lower the arithmetic cost, but they typically lack hardware implementations that rigorously validate gains in energy efficiency and chip area [61,79]. Other alternative implementations, for example finite-field-based CDC, have only been demonstrated for short-reach links and rely on scaling arguments [80].

4.1. Cluster-Based TDCE

Based on the Time-Domain Clustered Equalizer (TDCE) introduced in [81] and observation that CDC filters exhibit strong redundancy and tap overlap in the complex plane, a scheme that exploits geometric clustering of taps to redesign CDC as a hardware friendly structure is proposed [82]. It is validated that in FPGA implementation when memory organization and parallelization are co-optimized, the proposed algorithm can achieve significantly lower energy per recovered bit and multiplier usage than FDE, highlighting that theoretical multiplication complexity alone is an insufficient predictor of CDC hardware efficiency.

A formal explanation of tap redundancy in CDC filters and its geometric structure in the complex plane is explained. In impulse response of the CDC filter as below:

$$g[m]=\sqrt{{\displaystyle \frac{jc{T}^2}{D{\lambda }^{2}z}}}\exp \left(-j{\displaystyle \frac{\pi c{T}^2}{D{\lambda }^{2}z}}{m}^2\right)$$

(52)

Each tap has constant magnitude while its phase evolves quadratically with the tap index, so that the taps lie on a circle in the complex plane and can be interpreted as rotating phasors that apply pure phase shifts to the input samples. Because the phase term is proportional to $m^2$ and is taken modulo $2\pi$, many taps share identical or very similar phases. Different indices correspond to points that overlap or cluster at similar angles on the unit circle, leading to strong redundancy among taps. To quantify this phenomenon, the maximum number of taps $N$ required to avoid aliasing is derived as

$$-⌊{\displaystyle \frac{N}{2}}⌋\le m\le ⌊{\displaystyle \frac{N}{2}}⌋, N=2⌊{\displaystyle \frac{\left|D\right|{\lambda }^{2}z}{2c{T}^2}}⌋+1$$

(53)

Also, the angular distribution of a truncated subset of taps (about 60% of $N$) is introduced. Figure 6 demonstrates the angular distribution in short-reach and long-haul transmissions using circular histograms. In Figure 6 signals are transmitted takes place over standard single-mode fiber with chromatic dispersion $D=16.8 \mathrm{p}\mathrm{s}/(\mathrm{n}\mathrm{m}\cdot \mathrm{k}\mathrm{m})$ at the wavelength of 1550 nm. Fiber lengths are 80 km and 8000 km, respectively. Signals are sampled by 64 GSa/s. There are 30 bins in total, each occupies 12°.

Based on the angular distribution characteristics, a simple non-uniformity metric $\rho =(N_M-N_L)/\mu$ is introduced, where $N_M$ and $N_L$ denote the most and least populated angle bins, and $\mu$ is the average tap count per bin. This parameter shows how strongly taps concentrate into clusters as dispersion increases. The analysis reveals that short distances exhibit highly localized clusters, whereas for larger dispersion the phase distribution becomes more uniform but still retains at least one non-negligible cluster with $\rho >0$, implying persistent geometric redundancy.

The characteristic helps optimize the FIR filtering operation by first summing input samples associated with taps in the same angular cluster and then multiplying each sum, helping the FIR filter translate into a reduced-complexity time-domain equalizer. Given the standard FIR convolution

$$y[n]={\displaystyle \sum _{k=0}^{M-1}x}[k]g[n-k],$$

(54)

where $M$ is the filter length, $x[k]$ is the input sample, and $g[\cdot ]$ is the CDC filter. In this form, each output $y[n]$ requires $M$ complex multiplications as every tap $g[n-k]$ is directly multiplied by an input sample $x[k]$. The key idea is that taps close to each other in the complex plane are grouped into clusters, then all taps in a cluster can be approximated by a single representative tap denoted $g_C[k]$. Using the distributive property of multiplication over addition, the convolution can be re-expressed by first summing all input samples in the same tap cluster, and then multiplying this sum by the corresponding clustered tap. This leads to the approximated form

$$y[n]\approx {\displaystyle \sum _{j=0}^{N_C-1}{x}_S}[j]g_C[j], g[n-k]\approx g_C[j],\forall k\in Q_j,$$

(55)

where $N_C$ is the total number of tap clusters, $g_C[k]$ denotes the complex clustered taps, and $x_S[k]$ represents the summation of all input samples whose original taps belong to cluster $k$. It shows that each output sample is computed as a dot product between a cluster-summed input vector $x_S$ and a clustered coefficient vector $g_C$. The computational benefit is that instead of $M$ multiplications per output symbol in the original FIR, the clustered scheme needs only $N_C$ multiplications, at the cost of additional additions to form $x_S\left[k\right]$. Since additions are much cheaper in hardware than multipliers, this factorization enables a substantial reduction in multiplier usage and power while preserving most of the CDC performance.

TDCE based on two cluster schemes are also proposed. First is called TDCE-KMNS, which is a purely clustering-based design where the angular distribution is grouped by an unsupervised K-Means (KMNS) algorithm, and the cluster centroids are directly used as the clustered filter taps. The other is called TDCE-GD. This method is a refined design in which the KMNS clusters provide initial centroids, and a supervised gradient-descent (GD) optimization further adjusts the clustered taps to reduce approximation error and improve BER performance.

4.2. LS-FIR CDC

With the advent of coherent detection, high-order constellations and fast ADCs, CD is increasingly compensated digitally by applying an FIR filter with frequency response $H(e^{j\omega T})$ that approximates the ideal inverse CD transfer function [80]. The FIR CD compensation filter is derived with a complex impulse response given by

$$h(n)=\sqrt{{\displaystyle \frac{j}{4K\pi }}}{e}^{-j{\displaystyle \frac{n^2}{4K}}}, -⌊{\displaystyle \frac{N}{2}}⌋\le n\le ⌊{\displaystyle \frac{N}{2}}⌋$$

(56)

where the length of the filter is odd and given as

$$N=2⌊2K\pi ⌋+1.$$

(57)

It is designed over the full Nyquist band and hence ignore the fact that pulse-shaping filters already limit the effective signal bandwidth. Also, increasing the number of taps does not necessarily improve CDC quality, and the performance becomes clearly suboptimal and exhibits BER floors. Against this background, optimal least-squares (LS) FIR CD-compensation filters is proposed [81].

The LS-FIR filter seeks to optimally approximate the ideal dispersion inverse transfer function $H_{Des}(e^{j\omega T})$ with a finite-length complex FIR filter. The filter of odd length $N_C$ is defined as

$$H(e^{j\omega T})={\displaystyle \sum _{n=-\frac{N_c-1}{2}}^{\frac{N_c-1}{2}}h}(n)e^{-jn\omega T}.$$

(58)

To symbolize the accuracy of approximation, the complex error energy between $H(e^{j\omega T})$ and $H_{Des}(e^{j\omega T})$ within a frequency band $[{\mathrm{\Omega }}_1,{\mathrm{\Omega }}_2]$ is minimized:

$$E={\displaystyle \frac{1}{2\pi }}{\displaystyle {\int }_{{\mathrm{\Omega }}_1}^{{\mathrm{\Omega }}_2}{\left|H(e^{j\omega T})-H_{\mathrm{Des}}(e^{j\omega T})\right|}^2}d(\omega T).$$

(59)

The optimization problem is then formulated as

$$\widehat{h}=\arg {\min }_{h}E.$$

(60)

A closed-form of LS solution is given by

$$\widehat{h}=Q^{-1}D$$

(61)

where $Q$ is an $N_C\times N_C$ Hermitian Toeplitz matrix defined as

$$Q(n,m)=\left\{\begin{array}{ll}{\displaystyle \frac{{\mathrm{\Omega }}_2-{\mathrm{\Omega }}_1}{2\pi }},\hfill & n=m,\hfill \\ {\displaystyle \frac{e^{-j(n-m){\mathrm{\Omega }}_1}-e^{-j(n-m){\mathrm{\Omega }}_2}}{2j\pi (n-m)}},\hfill & n\ne m.\hfill \end{array}\right.$$

(62)

The vector $D$ collects the cross-correlation terms between the desired CD-compensation response and the FIR basis functions:

$$\begin{array}{ll}D(n)& ={\displaystyle \frac{1}{2\pi }}{\displaystyle {\int }_{{\mathrm{\Omega }}_1}^{{\mathrm{\Omega }}_2}{H}_{\mathrm{Des}}}(e^{j\omega T})e^{jn\omega T}d(\omega T)\\ & ={\displaystyle \frac{1}{2\pi }}{\displaystyle {\int }_{{\mathrm{\Omega }}_1}^{{\mathrm{\Omega }}_2}{e}^{j\left(K{(\omega T)}^2+n\omega T\right)}}d(\omega T).\end{array}$$

(63)

For full-band CDC, ${\mathrm{\Omega }}_2=-{\mathrm{\Omega }}_1=\pi$. However, if the properties of pulse shaping filters are utilized, the values of ${\mathrm{\Omega }}_1$ and ${\mathrm{\Omega }}_2$ can be reduced, hence the filter length can be reduced simultaneously. In a typical coherent transmission system, interpolation and decimation structure, as well as the pulse-shaping filters constrain the effective signal bandwidth, and this can be exploited to shorten the CDC FIR filter. To be more specific, interpolation at the transmitter and decimation at the receiver by an integer oversampling factor $L>1$ are implemented using an up-sampler or down-sampler together with anti-imaging and anti-aliasing filters $G_{TX}(e^{j\omega T})$ and $G_{RX}(e^{j\omega T})$. These are low-pass “Lth-band” filters with roll-off factor of $0<\rho <1$, passband/stopband edges at $\pi \left(1-\rho \right)/L$ and $\pi \left(1+\rho \right)/L$, and are designed so that the cascade of interpolation and decimation has an approximately flat in-band response

$${\displaystyle \sum _{l=0}^{L-1}{G}_{\mathrm{TX}}}(e^{j({\omega }_T-2\pi l/L)})G_{\mathrm{RX}}(e^{j({\omega }_T-2\pi l/L)})\approx 1.$$

(64)

A common choice is a square-root raised-cosine (SRRC) pulse shaper with impulse response $g_{TX}(t)=g_{RX}(t)$. Because these filters restrict the spectrum of the interpolated signal to $\left|{\omega }_T\right|\le {\omega }_s^T=\pi (1+\rho )/L<\pi$, the CD equalizer no longer needs to approximate the ideal inverse response over the full Nyquist band ${\omega }_T\in [-\pi ,\pi ]$. Instead, the least-squares design can be restricted to the reduced band ${\omega }_T\in [-{\omega }_s^T,{\omega }_s^T]$ by setting ${\mathrm{\Omega }}_2=-{\mathrm{\Omega }}_1={\omega }_s^T$ in the LS formulation, which directly lowers the required filter length $N_C$ and hence implementation complexity and delay. However, narrowing the design band can make the LS matrix $Q$ ill-conditioned. To stabilize the optimization and avoid pathological out-of-band behavior, the paper introduces a small regularization term and replaces the solution $\widehat{h}=Q^{-1}D$ with

$$\widehat{h}={(Q+ϵI_{N_c})}^{-1}D,$$

(65)

where $\epsilon >0$ controls the trade-off between numerical robustness and approximation accuracy in the reduced band.

The simulation setup is as follows: a 32 GBaud, 16-QAM dual-polarization transmission at optimal launch power across standard single-mode fiber (SSMF). The fiber parameters are dispersion $D=16.8\mathrm{ps}/(\mathrm{nm}\cdot \mathrm{km})$, nonlinearity coefficient $\gamma =1.2 {(\mathrm{W}\cdot \mathrm{km})}^{-1}$, and attenuation $\alpha =0.21\mathrm{dB}/\mathrm{km}$. Erbium-doped fiber amplifiers (noise coefficient 4.5 dB) were included after each 80 km span, with the number of spans ranging from 1 to 8. All filters operated at 2 samples per symbol. The propagation dynamics were simulated using the Manakov equation and the split-step Fourier method.

Numerical results show that these LS FIR filters yield lower BER with fewer taps than both the traditional analytical CD-compensation filter and frequency-sampling-method (FSM) filters, with the performance advantage becoming more pronounced for high-order QAM and other high-spectral-efficiency formats.

Table 1. Simulation Parameters in Example 1 and 2.

Example	N	z km	F Hz	K
1	251	4000	$21.4\times {10}^9$	19.9227
2	875	1000	$80\times {10}^9$	69.605

demonstrates two transmission scenarios. Figure 7 shows the BER of 16 QAM signals with different values of $N = N_C$ in (56) and (61). As can be seen, if we increase $N_C$ in (61), the value of BER decreases. This means that we can obtain fine CDC filters by increasing $N_C$. However, this does not apply to classical CDC and the value of BER may even increase when increasing N. Figure 8 demonstrates the simulated BER of Examples 1 and 2 over an AWGN channel where $\epsilon ={10}^{-14}$. For each curve, the authors report the shortest LS filter length $N_C$ whose BER is no worse than that of the conventional analytical CD filter with length N. For instance, with Example 1, one can choose $N_C=119$ for the LS filter and still obtain a lower BER than with the traditional filter of length N = 251. Similarly, for Example 2 with 16-QAM and 32-QAM, an LS filter with only $N_C=477$ taps already matches or surpasses the BER performance of the 875-tap classical filter, demonstrating that the proposed LS approach together with band-limited case can achieve equal or better performance with substantially shorter filters.

In terms of multiplier count, a conventional FIR chromatic dispersion compensation filter of length (N) requires (N) multiplications per output symbol, whereas the TDCE clustering approach groups taps into (C) clusters, reducing the multiplier count to (C), typically about 40–60% of the original taps. For example, in a 640 km transmission scenario, FPGA implementation showed that TDCE achieved up to a 71.4% reduction in multiplier usage compared to the frequency-domain equalizer (FDE). Regarding memory, clustering requires storing only cluster centroids and pre-accumulated input samples, with about (2C) memory positions, corresponding to ~40% savings in coefficient storage.

In terms of energy and latency, FPGA prototypes demonstrated that although TDCE has higher theoretical multiplication complexity than FDE, optimized hardware implementations with parallelization and memory organization achieved up to 70.7% energy savings. Latency overhead is negligible, since clustering only introduces an intra-cluster addition step that can be pipelined; in FPGA implementations this adds less than one clock cycle. Control complexity is not significantly increased, because cluster membership is fixed during design and does not require runtime reconfiguration. The datapath remains a regular structure of adders and multipliers, mapping efficiently to FPGA DSP slices or ASIC MAC arrays.

ASIC studies also confirm the hardware friendliness of clustering, reporting ~25% chip area reduction and ~20% power savings, with BER performance within 0.2 dB of full-length FIR design. The LS-FIR approach, through least-squares optimization and band-limited design, achieves lower BER with significantly fewer taps.

Table 2. Summary of Dispersion Compensation Algorithm.

Algorithm Variants	Key Principle/Structure	Main Parameters	Computational/Hardware Metrics
OLS	Block convolution with overlapping segments; discards aliasing samples	Block length, overlap size, filter length	Requires per-block FFT + IFFT
VOSF	Splits long FIR into multiple shorter subfilters; each processed by independent OLS units	FFT length, total filter length	Multiple FFT/IFFT modules; parallel processing; higher hardware cost due to replicated FFT engines;
LOSF	All subfilters share a single FFT/IFFT; time shifts handled via frequency-domain phase rotation	FFT length, total filter length	Only 2 FFT/IFFT per block; reduced complexity vs. VOSF;
TS-OLF	Non-overlapping blocks; error compensation via odd/even frequency decomposition	FFT length, total filter length	4 FFT/IFFT per block; additional diagonal matrix multiplications;
TDCE	Groups redundant FIR taps into clusters based on phase geometry	Number of clusters	Reduces multiplications (≈60% taps discarded); extra additions; FPGA-friendly
LS-FIR CDC	FIR designed via least-squares approximation of inverse CD transfer function	Filter length, regularization parameter	Fewer taps than analytical FIR; matrix inversion cost; improved BER with shorter filters

is a summary table that integrates all algorithm variants, their key parameters, and basic computing or hardware metrics.

5. Conclusions

This paper provides a systematic review of dispersion compensation algorithms in long-distance fiber optic transmission. Based on the balance between performance and implementation cost during the compensation process, a classification framework of two major technical paradigms is proposed: compensation without penalty and compensation with penalty.

For compensation algorithms without penalty, we detail block processing strategies such as OLA and OLS, as well as various enhancement strategies combining intelligent filter segmentation and optimized frequency domain workflows. For compensation algorithms wit penalty, we focus on the scheme of redesigning the CDC into a hardware-friendly structure using tapped geometric clustering, and the FIR filter based on the approximate ideal inverse CD transfer function of the frequency response. We conduct simulations of each algorithm and analyze its core principles, computational complexity, compensation performance, and applicable scenarios in depth.

The results show that while compensation algorithms without penalty, represented by LOSF, can achieve complete dispersion compensation with a fixed short FFT size, their high computational complexity and processing latency become bottlenecks for their application in real-time high-speed systems. Compensation algorithms with penalty, through controllable performance loss, achieve a significant improvement in computational efficiency and hardware feasibility, reflecting the art of balancing accuracy and feasibility in engineering practice.

Looking ahead, the development of this research field will inevitably be guided by low complexity and small deployment scale, with a focus on breakthroughs in key technologies such as lightweight dispersion compensation models, providing a solid algorithmic foundation and theoretical support for building the next generation of high-capacity, high-energy-efficiency long-distance optical fiber communication systems.