A Novel Hybrid Whale Optimization Algorithm-Based SLM (HWOA-SLM) for PAPR Reduction in Optical IM/DD OFDM Systems

Abstract

This paper presents a comprehensive analysis and simulation of a cost-effective optical Intensity-Modulation/Direct-Detection (IM/DD) Orthogonal Frequency Division Multiplexing (OFDM) system. Implemented via a MATLABR2024a and OptiSystem 23 co-simulation environment, the study evaluates a 4-QAM modulated link over a 120 km transmission distance, providing detailed investigations into signal spectral properties and constellation characteristics. To address the critical performance limitation posed by high Peak-to-Average Power Ratio (PAPR), a novel Hybrid Whale Optimization Algorithm with Selective Mapping (HWOA-SLM) is proposed. Simulation results demonstrate that the proposed scheme significantly outperforms conventional reduction techniques; specifically, at a Complementary Cumulative Distribution Function (CCDF) of 10⁻² and a fixed computational budget of 256 evaluations, the HWOA-SLM achieves a PAPR reduction gain of 3.9 dB relative to the original OFDM signal. Furthermore, in terms of algorithmic efficiency, it outperforms standard Genetic Algorithm (GA) and WOA-based SLM techniques by approximately 0.4 dB under identical computational budgets. Parametric analysis further confirms that increasing population size and iteration numbers consistently improves convergence, thereby minimizing non-linear distortions and enhancing signal integrity. Moreover, the technique exhibits superior Bit Error Rate (BER) performance, delivering Optical Signal-to-Noise Ratio (OSNR) gains of 0.63 dB, 1.31 dB, and 2.0 dB over standard WOA-SLM, GA-SLM, and conventional SLM, respectively. Conclusively, the HWOA-SLM offers a favorable trade-off between computational complexity and reduction efficiency, validating its potential for reliable, high-speed optical communication networks.

1. Introduction

Optical communication systems are continuously evolving to meet the growing demand for high data rate transmission and increased network capacity. Contemporary wired and wireless communication systems face significant limitations, including susceptibility to signal distortion, scattering, security vulnerabilities, and restricted bandwidth. In contrast, optical communication systems offer a robust alternative capable of overcoming these challenges. To further enhance performance, Orthogonal Frequency Division Multiplexing (OFDM) has emerged as a prominent modulation technique, favored for its ability to support high-speed data transmission by effectively mitigating inter-symbol interference and robustness in noisy environments [1]. Functionally, OFDM divides a high-speed serial information stream into multiple lower-rate parallel streams, which are transmitted via orthogonal sub-carriers. This approach effectively minimizes Inter-Symbol Interference (ISI) while maintaining high spectral efficiency.

However, a major drawback of Optical OFDM (O-OFDM) systems is their inherently high Peak-to-Average Power Ratio (PAPR). High PAPR peaks can exceed the linear dynamic range of key optical components, such as the Light Emitting Diode (LED) or laser diode source. This leads to signal clipping and significant non-linear distortion, which degrades overall system performance [2]. To address this issue, various PAPR reduction methods have been proposed, including Partial Transmit Sequence (PTS), clipping and filtering, Selective Mapping (SLM), and coding schemes [3].

Each technique presents a distinct trade-off. While simple to implement, clipping methods introduce in-band distortion and out-of-band noise by limiting signal amplitude to a specific threshold. The SLM method, while avoiding signal distortion, compromises bandwidth efficiency due to the necessity of transmitting side information to the receiver. In comparison, coding techniques offer the potential advantage of reducing PAPR without inherently degrading Bit Error Rate (BER) performance, though they may impose constraints on data rate or system complexity [4].

1.1. Importance of Work

The primary drawback of the conventional SLM technique is its reliance on an exhaustive search to identify the optimal phase sequence, which results in prohibitive computational complexity. To mitigate this, we propose a novel scheme: the Hybrid Whale Optimization Algorithm with Selective Mapping (HWOA-SLM). By leveraging the adaptive search capabilities of the HWOA, our method intelligently optimizes the selection of phase factors, moving beyond inefficient random exploration. This approach achieves an effective balance between PAPR reduction performance and computational efficiency, thereby rendering the system viable for practical, high-speed optical OFDM deployments.

1.2. Related Works

Recently, meta-heuristic algorithms have attracted considerable interest due to their efficiency in addressing complex and computationally intensive challenges. Consequently, a growing body of research has been devoted to applying these optimization strategies to the SLM technique, aiming to replace its exhaustive search mechanism with intelligent selection processes that drastically lower system complexity.

The authors first developed an optical intensity modulation direct detection (IM/DD) OFDM communication system employing various modulation techniques [5,6]. This was followed by the design of a 100 Gbps long-haul IM/DD OFDM system to support high-speed downstream transmission [7]. Subsequently, they proposed a bidirectional hybrid long-haul optical IM/DD OFDM WDM-PON based on OOK-RSOA remodulation [8]. The work was further extended in [9] with a bidirectional WDM-PON architecture capable of delivering 1.6 Tbps. To address the high PAPR challenge, the researchers applied swarm intelligence algorithms to the PTS method in coherent detection OFDM systems [10]. They also employed a Discrete Forest Optimization algorithm to improve the SLM technique for optical IM/DD OFDM applications [11].

Extensive research has been conducted to identify effective PAPR reduction strategies for optical OFDM signals [12]. Early contributions focused on signal distortion techniques; for instance, ref. [13] presented a minimization strategy based on clipping, though it notably degraded BER performance. Similarly, the authors of [14] employed recurrent clipping and filtering to mitigate high peaks. broadening the scope, several studies have comparatively analyzed distortion-less techniques alongside clipping. Milon et al. [15] investigated amplitude clipping, SLM, and PTS, while ref. [16] extended this evaluation to include Tone Reservation (TR) and Phase Insertion (PI). More recently, the focus has shifted toward comparative assessments in diverse waveform topologies, such as FBMC versus OFDM [17], and the integration of intelligent algorithms. Innovative approaches include the application of Machine Learning (ML) strategies [18] and modern swarm intelligence.

Sanjana et al. [19] developed a Hanowa matrix-based approach that enhances the performance of the C-SLM method for PAPR reduction. Building on optimization techniques, the researchers in [20] proposed an improved PTS technique that reduces BER and power spectral density (PSD) using moth flame optimization. Similarly, a PTS method based on m-arrangement was introduced by the authors in [21], while ref. [22] presented a two-step (TR) method for PAPR reduction, further diversifying the approaches available.

In 2022, the application of fuzzy neural networks led to the proposal of an adaptive PTS strategy aimed at decreasing PAPR in OFDM systems [23]. Complementing these advancements, the ICF method [24] was evaluated through three approaches to minimize PAPR in OFDM communication systems. Additionally, Namitha S. [25] analyzed PAPR reduction using the Hadamard-SLM approach in OFDM systems, highlighting alternative matrix-based techniques. The authors in [26] introduced and optimized novel hybrid algorithms combining PTS and SLM with PSO to effectively reduce the Peak-PAPR in NOMA systems. The improved PTS technique combined with Mu-Law companding proposed in [27] further enhances performance during division and phase rotation stages for PAPR reduction in OFDM systems. Extending the use of machine learning, Mayakannan et al. [28] introduced a neural network and linear regression method achieving PAPR reductions of 84.1% for 16QAM and 82.9% for 64QAM-OFDM without degrading BER.

Addressing computational efficiency, an improved PAPR reduction technique combining quantum GA and PTS was proposed in 2021, achieving a 64% reduction in computational complexity [29]. Feng Zou et al. [30] reported a neural network-based PAPR reduction method founded on simplified clipping and filtering (SCF) techniques, while ref. [31] presented a hybrid approach combining mixed companding and clipping techniques for PAPR reduction. The comprehensive overview in [32] covers PAPR reduction methods for three key modulation schemes in optical and wireless communication systems O-OFDM, O-FBMC, and O-NOMA highlighting their role in enhancing system efficiency. Moreover, the study in [33] focuses on integrating OFDM with 5G radio waveforms for visible light communication (VLC) systems, addressing the adverse effects of high PAPR by evaluating algorithms such as clipping, PTS, SLM, and companding to maintain efficiency and signal integrity.

To further mitigate nonlinear distortion in high PAPR systems, ref. [34] proposes a scheme combining DCO-OFDM with index modulation and convex optimization algorithms. This method employs partially activated subcarriers to transmit constellation-modulated symbols alongside a dither signal, illustrating a novel approach to balancing complexity and performance.

Expanding on modulation techniques, the work in [35] explores O-OFDM methods in VLC systems, examining parameters such as constellation size, data arrangement, power efficiency, computational complexity, BER, and PAPR. The study introduces the Vandermonde-Like Matrix (VLM) precoding technique, a non-distorting method implemented across various O-OFDM schemes including DCO-OFDM, ADO-OFDM, ACO-OFDM, FLIP-OFDM, ASCO-OFDM, and LACO-OFDM, specifically aimed at reducing PAPR.

Among the popular PAPR reduction algorithms, the SLM technique is widely favored due to its straightforward structure and effective performance without signal distortion. In this method, multiple phase sequences are randomly generated and multiplied by the original QAM-modulated data. The phase sequence yielding the lowest PAPR in the transmitted signal is selected [36,37]. However, since the SLM algorithm requires an IFFT operation for each generated phase sequence, its computational complexity increases significantly with the number of sequences. Therefore, to prevent excessive computational load, it is essential to limit the number of searches while still achieving acceptable PAPR reduction.

1.3. Motivation

Metaheuristic optimization strategies have become increasingly pivotal in the advancement of long-haul Optical OFDM frameworks. The primary impetus for this adoption is the necessity for efficient phase factor selection within SLM protocols. In long-range transmission scenarios, such optimization is indispensable; elevated PAPR triggers deleterious fiber non-linearities—specifically Self-Phase Modulation (SPM) and the Kerr effect—which accumulate over distance and severely compromise signal fidelity. Nevertheless, conventional algorithms often struggle under the rigorous latency constraints of real-time optical networks, frequently stagnating at local optima rather than achieving global convergence. While standard metaheuristics such as Particle Swarm Optimization (PSO), Genetic Algorithms (GA), and WOA have demonstrated potential, they often suffer from inconsistent convergence rates and excessive computational overhead. Furthermore, certain existing methods mitigate PAPR only at the expense of increased BER. To surmount these limitations, this study introduces a Hybrid WOA-SLM (HWOA-SLM) framework. Inspired by the cooperative “bubble-net” hunting tactic of humpback whales, this method incorporates a modified mutualism phase to effectively balance search space exploration with exploitation. This enhancement ensures robust PAPR suppression and system stability. It should be emphasized that the objective of this study is not to outperform all existing enhanced WOA variants, but to demonstrate the effectiveness of a discrete, low-complexity hybrid adaptation specifically designed for the SLM problem.

1.4. Contributions and Novelties

This research presents a distinct SLM approach tailored to resolve the inefficiencies of prior optimization tools through the integration of a HWOA algorithm. The principal innovations and contributions are summarized as follows:

• A new SLM scheme is proposed that utilizes the HWOA algorithm to optimize phase factors dynamically, replacing inefficient random searches.
• The method significantly reduces the computational burden compared to exhaustive search techniques, while improving PAPR reduction, power efficiency, and signal fidelity (BER/PSD).
• Simulation results validate that the HWOA-SLM offers the fastest convergence rate and the most effective global solution for PAPR reduction when compared to leading benchmarks.
• This work demonstrates, for the first time relative to [5,6,7,8,9,10,11], a 40 Gbps optical link over 120 km of SMF achieving a BER of 10⁻⁵ through the application of the proposed hybrid optimization.

1.5. Paper Organization

The remainder of this paper is organized as follows: Section 2 outlines the Optical OFDM system model and provides the mathematical definition of PAPR. Section 3 reviews the principles of the conventional SLM technique and formulates the optimization problem. Section 4 details the proposed HWOA-SLM framework, elaborating on the algorithmic integration. Section 5 presents a comprehensive analysis of the simulation results and performance metrics. Finally, Section 6 concludes the study.

2. Optical OFDM System Model and PAPR

Figure 1 illustrates the block diagram of optical IM/DD OFDM system. The IFFT/FFT block plays a critical role in processing signals for both the transmitter and the receiver, serving as the central component of the system. Traditional FT transmits continuous signals in the time/frequency domain, simplifying signal processing by sampling them in the traditional transformation.

Fast algorithms FFT and IFFT are widely used in digital signal processing applications to get the DFT and IDFT. Optical OFDM systems utilize IFFT for modulation at a transmitter and FFT for demodulation at a receiver. Let $X=\left[X_0,X_1,\dots ,X_{N-1}\right]$ be the frequency-domain input data vector of length $N$, where $X_k$ represents the QAM modulated symbol QAM on the $k$-th subcarrier. The discrete-time OFDM signal $x\left[n\right]$ is generated via the Inverse Fast Fourier Transform (IFFT):

$$x\left[n\right]={\displaystyle \frac{1}{\sqrt{N}}}{\sum }_{k=0}^{N-1}{X}_k\cdot e^{j2\mathrm{\pi }{\displaystyle \frac{nk}{N}}},\hspace{1em}0\le n<N$$

(1)

The PAPR of the transmitted signal is defined as the ratio of the maximum instantaneous power to the average power:

$$PAPR\left(x\right)={\displaystyle \frac{{\underset{0\le n<\mathit{LN}}{\max }\left|x\left[n\right]\right|}^2}{E\left[{\left|x\left[n\right]\right|}^2\right]}}$$

(2)

where $L$ is the oversampling factor (typically $L=4$) required to capture the true analog peaks [34] and $E\left[\cdot \right]$ expresses the expectation operator. CCDF is used to assess the PAPR through a probabilistic estimation, which is provided as follows:

$$CCDF=1-{\left(1-e^{-PAP{R}_0}\right)}^{NL}$$

(3)

where $PAP{R}_0$ represents a specified threshold.

3. SLM Technique and Problem Formulation

Figure 2 provides the principal structure of the SLM scheme. In SLM, $U$ different phase sequences are generated.

Let $b^{\left(u\right)}$ be the $u$-th phase vector, where elements are drawn from a discrete set (e.g., $\{\pm 1, \pm j\}$):

$$b^{\left(u\right)}=\left[b_0^{\left(u\right)},b_1^{\left(u\right)},\dots ,b_{N-1}^{\left(u\right)}\right],\hspace{1em}{b}_k^{\left(u\right)}\in \{e^{j0},e^{j{\displaystyle \frac{\pi }{2}}},e^{j\pi },e^{j{\displaystyle \frac{3\pi }{2}}}\}$$

(4)

The modified frequency vector becomes $X^{\left(u\right)}=X\odot b^{\left(u\right)}$ (element-wise multiplication). The candidate time-domain signal is:

$$x^{\left(u\right)}=IFFT\{X\odot b^{\left(u\right)}\}$$

(5)

Consequently, the optimization process seeks to identify the specific phase sequence $b_{opt}$ that yields the lowest possible PAPR among all candidate signals.

$$b_{opt}=\mathit{arg}\underset{b^{\left(u\right)}}{\mathit{min}}\left(PAPR\left(IFFT\{X\odot b^{\left(u\right)}\}\right)\right)$$

(6)

Then, the candidate OFDM signal with the minimized PAPR is selected to be sent.

$$x^{*}\left(n\right)=\underset{1\le u\le U}{\mathit{min}}\left\{x^{\left(u\right)}\left(n\right)\right\}$$

(7)

The receiver should find the data with the appropriate optimized phase rotation sequences $b_{opt}$ to recover the transmitted OFDM signals. The simple method involves transmitting the appropriate side information (SI) to the receiver for signal recovery.

4. The Proposed HWOA-SLM Technique

The WOA is a meta-heuristic technique rooted in swarm intelligence, originally introduced by Mirjalili and Lewis in 2016 [38]. It was subsequently refined by Chakraborty et al. in 2021 [39] to improve its efficacy in continuous optimization tasks, specifically targeting faster and more accurate convergence. The core inspiration for WOA is the distinctive ‘bubble-net’ feeding performance of humpback whales. In this strategy, the whales herd prey—typically krill or small fish by generating spiral bubbles along a ‘9’-shaped path as shown in Figure 3, effectively trapping them before attacking.

In this work, instead of testing random phase vectors (Classical SLM), we search for the optimal $b_{opt}$ using HWOA. This consists of two phases: Seeding and Optimization.

4.1. Phase 1: Seeding (Initial Exploration)

To prevent the algorithm from starting at a poor local minimum, we perform a random search for a limited budget $M_{seed}$.

• Let $f\left(b\right)$ be the objective function (PAPR value).
• Generate $M_{seed}$ random phase vectors.
• Select the best vector $b_{best}$ such that:

  $$b_{best}=\mathit{arg}\underset{i=1\dots M_{\mathit{seed}}}{\mathit{min}}f\left(b_i\right)$$

  (8)

This $b_{best}$ becomes the “Leader Whale” ${(X}^{\mathrm{*}})$ for the optimization phase.

4.2. Phase 2: Whale Optimization Loop

We initialize a population of search agents (whales) matrix $X$ of size $N_{pop}\times N$. The leader $X^{*}\left(t\right)$ is the position of the whale with the lowest PAPR at iteration $t$. The algorithm iterates through three behaviors:

• Shrinking Encircling Prey (Exploitation)

When probability $p<0.5$ and $\left|A\right|<1$, whales update their position towards the current best solution $X^{\mathrm{*}}$.

$$D=\left|C\cdot X^{\mathrm{*}}\left(t\right)-X\left(t\right)\right|$$

(9)

$$X(t+1)=X^{\mathrm{*}}\left(t\right)-A\cdot D$$

(10)

where coefficients $A$ and $C$ are calculated as:

$$A=2a\cdot r_1-a$$

(11)

$$C=2\cdot r_2$$

(12)

Here, $r_1,r_2$ are random vectors in $\left[\mathrm{0,1}\right]$, and $a$ decreases linearly from 2 to 0 over iterations.

B.

Spiral Updating (Bubble-Net Attack)

When probability $p\ge 0.5$, whales move in a spiral path towards the best solution:

$$D^{\prime }=\left|X^{*}(t)-X(t)\right|$$

(13)

$$X(t+1)=D^{\prime }\cdot e^{bl}\cdot \mathit{cos}\left(2\pi l\right)+X^{*}(t)$$

(14)

where $\mathit{b}$ is a constant for spiral shape and $\mathit{l}\in \left[-\mathrm{1,1}\right]$.

C.

Search for Prey (Exploration)

When $p<0.5$ and $\left|A\right|\ge 1$, whales move towards a random whale $X_{rand}$ to avoid local optima:

$$D=\left|C\cdot X_{rand}-X\left(t\right)\right|$$

(15)

$$X(t+1)=X_{rand}-A\cdot D$$

(16)

4.3. Hybrid Modifications (Discrete and Greedy)

“Hybrid Innovation Strategy: Standard WOA algorithms are inherently designed for continuous search spaces, rendering them inefficient for the discrete phase-factor selection required by SLM. The ‘Hybrid’ nature of our proposed HWOA-SLM specifically bridges this gap through three distinct algorithmic innovations:

• Smart Seeding (Initialization): Unlike standard WOA which initializes blindly, we employ a preliminary random search (Seeding Phase) to inject high-quality ‘Leader’ candidates into the initial population. This prevents early stagnation in local minima common to random initialization.
• Discretization-Aware Mapping: We introduce a specialized Discrete Mapping operator (Equation (17)) that projects continuous whale positions onto the nearest valid phase constellation point {±1, ±j} after every position update, ensuring strictly valid SLM candidates.
• Greedy-Mutualism Mechanism: To counter the stochastic volatility of standard WOA, we implement a Greedy Update Rule (Equation (18)). This ensures monotonicity in convergence; a whale’s position is only updated if the new discrete phase sequence yields a strictly lower PAPR, effectively combining exploration with aggressive exploitation.”

4.3.1. Discrete Mapping

After calculating the new continuous position $X(t+1)$, the values are mapped to the nearest valid phase index $k \in \{\mathrm{1,2},\mathrm{3,4}\}$:

$$b_{new}=\Phi \left(X(t+1)\right)$$

(17)

where $\Phi \left(\cdot \right)$ maps real numbers to the discrete phase set $\{1, j, -1, -j\}$.

4.3.2. Greedy Update Rule

To ensure strict convergence (PAPR never increases), the new position is only accepted if it improves the PAPR:

$$X_i\left(t+1\right)=\left\{\begin{array}{c}{X}_{new} \mathrm{i}\mathrm{f} f\left(X_{new}\right)<f\left(X_i\left(t\right)\right)\\ X_i\left(t\right) Otherwise \end{array}\right.$$

(18)

4.3.3. Mutation Operator

To add diversity, a mutation mask $M$ is applied with low probability $P_m$: $\mathrm{If} rand\left(\right)<P_m,\hspace{1em}{b}_k=\mathrm{random}\left(\{\pm 1,\pm j\}\right)$

4.3.4. Final Selection

After $T$ iterations, the transmitter selects the final global best phase vector:

$$b_{final}=X(t+1)$$

(19)

The transmitted signal is: $x_{TX}=IFFT\{X\odot b_{final}\}$.

This optimized signal has the minimum PAPR found within the computational budget, effectively reducing non-linear distortion in the fiber optic link. The pseudo-code representing the process of the HWOA-SLM technique is presented in Algorithm 1.

Algorithm 1. Pseudocode for HWOA-SLM

Figure 4 shows the flowchart of the proposed algorithm, a two-stage optimization process designed to minimize PAPR. The process begins with Phase 1: Seeding, where the algorithm initializes parameters and generates random phase vectors to identify a starting “Global Best” vector with the lowest PAPR. The workflow then transitions to Phase 2: Whale Optimization, an iterative loop where a population of search agents (whales) updates their positions based on mathematical coefficients ($p$ and $\left|A\right|$); this logic dictates whether the agents “encircle” the current best solution or explore the search space randomly. For every iteration, the new positions are discretized and evaluated using a greedy selection strategy—accepting changes only if they improve the local fitness—while simultaneously updating the Global Best if a new minimum PAPR is achieved, continuing until the maximum number of iterations is reached.

5. Simulation Results and Discussion

Figure 5 presents the schematic of the proposed IM/DD optical OFDM architecture, which was simulated using OptiSystem software. The system comprises three primary modules: the RTO transmitter, the optical fiber channel, and the OTR receiver. The global simulation parameters utilized are detailed in

Table 1. Simulation parameters.

Component Name	Variable Name	Value
Global System	Sequence Length	262,144
	Samples per Bit	4
	Bit Rate	40 Gbps
	Number of samples	1,048,576
	Symbol Rate	20 × 109 Symbols/s (4-QAM)
OFDM modulator	FFT points	128
	Number of used subcarriers	80
	Cyclic prefix	5
CW laser	Operating wavelength	1552.5 nm
	Power	1 dBm
	Line width	0.016 nm (0.01 MHz)
Mach-Zehnder modulator	Extinction ratio	30 dB
	Switching bias voltage	4 V
	Switching RF voltage	4 V
	Insertion loss	1 dB
PIN photodetector	Dark current	10 nA
	Responsivity type	InGaAs
	Shot noise distribution	Gaussian

.

To evaluate system performance, a BER test component operating at a reference bit rate of 40 Gb/s is employed to generate and analyze the digital bit stream. The modulation scheme utilizes a 128-point FFT with 80 active sub-carriers. As detailed in Table 1, a cyclic prefix of 5 is appended to each OFDM symbol immediately following the IFFT stage to mitigate Inter-Symbol Interference (ISI).

The transmission process begins with a QAM sequence generator that maps the input bits into QAM symbols. In this study, 4-QAM is adopted as the modulation format to validate the fundamental convergence behavior of the proposed HWOA-SLM algorithm. It is important to note that the PAPR reduction mechanism in SLM is algorithmic and mathematically independent of the underlying constellation order; therefore, the optimization gains demonstrated here are universally applicable to higher-order formats (e.g., 16-QAM, 64-QAM) [11].

These symbols are subsequently modulated onto orthogonal subcarriers within the OFDM modulator, configured with 128 FFT points and 80 active subcarriers. Following modulation, the I-Q OFDM signals pass through a low-pass filter with a 15 GHz cutoff frequency. Quadrature modulators then up-convert the signals to a transmission frequency of 12 GHz. Finally, the RF electrical signal is modulated onto the optical domain using a Mach-Zehnder Modulator (MZM). Detailed parameters for the CW laser, MZM, and PIN photodetector are provided in Table 1.

Subsequent to the MZM, the optical signal propagates through a span consisting of 20 km of Single Mode Fiber (SMF) and 10 km of Dispersion Compensated Fiber (DCF). The DCF is utilized to mitigate chromatic dispersion introduced by the primary fiber, thereby enhancing overall transmission performance. To offset propagation losses, the signal is boosted by an in-loop optical amplifier. At the receiver, a photodetector performs optical-to-electrical conversion, generating a bandpass signal that is subsequently amplified. The signal is then recovered using an electrical quadrature demodulator; ensuring synchronization of parameters between the OFDM modulator and demodulator is critical for accurate QAM symbol retrieval. Finally, the QAM sequence detector decodes the symbols into bits, allowing the BER test set to compute the error rates.

Figure 6 shows the constellation diagrams for the transmitted and received 4-QAM symbols following transmission over a total distance of 120 km (achieved via four loops of 30 km). The transmission distance of 120 km was selected to represent a typical extended metro-access link, stressing the system with chromatic dispersion to observe the interaction between PAPR reduction and fiber propagation effects. As demonstrated in Figure 6b, the received symbols are accurately recovered, verifying the system’s signal integrity even over this extended propagation range.

Figure 7 illustrates the electrical OFDM spectrum of the optical system at 12 GHz after the quadrature modulator and photodetector, respectively. A quadrature modulator comprises three main parameters to be set up: gain, frequency and bias. The bias is increased to 2 to remove the negative values of the transmitted OFDM signal. In the optical OFDM systems, the transmitted OFDM signal should be positive and real due to the increasing bias of the quadrature modulator.

Figure 8a shows the optical OFDM spectrum at the output of the LiNb MZ Modulator. The modulator is configured with a 30 dB extinction ratio, and both the switching RF and bias voltages are set to 4 V. By modulating a CW laser source at an operating wavelength of 1552.5 nm, the MZM performs the necessary electrical-to-optical conversion. The signal is subsequently transmitted through an amplified optical link consisting of two fiber segments. Figure 8b presents the resulting optical spectrum at the receiver end, immediately prior to the PIN photodetector.

5.1. PAPR Reduction Performance

Figure 9 illustrates the CCDF of the PAPR for different reduction techniques. The CCDF denotes the probability ($Pr$) that the PAPR of an OFDM symbol exceeds a specific threshold level (${PAPR}_0$). The plot compares the performance of the original unprocessed signal against three reduction schemes: Classical SLM, GA-SLM, classical WOA-SLM, and the proposed HWOA-SLM. To ensure a fair comparison regarding computational complexity, the number of searches is standardized to 256 for all evaluated PAPR reduction methods. As evidenced by the curves, the original signal exhibits the highest PAPR characteristics. All implemented SLM variants successfully shift the CCDF curves to the left, indicating a reduction in the probability of high PAPR occurrence. It is observed that the proposed HWOA-SLM method outperforms other PAPR reduction techniques. For quantitative comparison, at a CCDF probability of ${10}^{-2}$, the original signal exhibits a PAPR of approximately $9.9$ dB. Classical SLM, GA-SLM, and WOA-SLM reduce this to approximately $6.4$ dB, $6.2$ dB, and $6.1$ dB, respectively. The proposed HWOA-SLM achieves the best performance, further reducing the PAPR to approximately $6$ dB at the same probability level, demonstrating its superior capability in mitigating high signal peaks. In terms of algorithmic benchmarking, this study prioritizes a direct comparison between the proposed HWOA-SLM and standard meta-heuristic baselines (Standard WOA and GA). While various ‘improved’ WOA variants exist for general continuous functions, our HWOA-SLM is specifically tailored for the discrete combinatorial nature of the SLM problem. Therefore, we demonstrate the efficacy of our specific hybrid modifications (Seeding and Discrete Greedy Mapping) against the fundamental WOA, showing a clear performance uplift without the need for the excessive computational overhead often associated with other generalized complex WOA variants.

Table 2. PAPR values and reduction gains for different schemes with fixed computational complexity.

Methods	Number of Searches	PAPR at $\mathbf{CCDF} = {10}^{-2}$ (dB)	PAPR Reduction Gain (dB)
Original signal	0	9.9	-
Classical SLM	U = 256	6.4	3.5
GA-SLM	${Pop}_{GA}$$\times {Gen}_{GA}$ = 16 × 16 = 256	6.2	3.7
WOA-SLM	${Pop}_{WOA}$$\times {Maxiter}_{WOA}$ = 16 × 16 = 256	6.1	3.8
The proposed HWOA-SLM	$M_{seed}$$\mathbf{+} {(Pop}_{HWOA}$$\mathbf{\times } {Maxiter}_{HWOA})$ = 56 + 20 × 10 = 256	6	3.9

provides a comparative analysis of the PAPR reduction performance and computational complexity for the Original Signal, Classical SLM, GA-SLM, WOA-SLM, and the proposed HWOA-SLM [11]. For the Classical SLM, this complexity corresponds to $U=256$ independent random phase sequences. In the GA-SLM stage, the genetic evolution is driven by specific crossover and mutation probabilities to balance exploration and exploitation. A crossover rate of 100% is utilized, ensuring that single-point crossover is applied in every generation to generate offspring from the top-performing parents. Furthermore, a mutation rate of 0.15 is employed, meaning that each gene (phase factor) within a candidate sequence has a 15% probability of being randomly reset to a new discrete value between 1 and 4. In the case of the GA-SLM, the searches are defined by the product of the population size and the number of generations ($16\times 16=256$). In the case of the WOA-SLM, the searches are defined by the product of the population size and the number of iterations ($16\times 16=256$). A unique computational strategy is employed for the proposed HWOA-SLM. As detailed in the table, the total search count is calculated as the sum of the initial seeds ($M_{seed}$) and the optimization process ($Po{p}_{HWOA}\times Maxite{r}_{HWOA}$). Specifically, the search budget is partitioned to balance exploration and exploitation: approximately 20% of the searches are allocated to the seeding phase ($M_{seed}=56$) to establish a diverse initial population, while the remaining 80% are dedicated to the Whale Optimization Algorithm ($20\times 10=200$) to iteratively refine the phase factors. Regarding PAPR reduction gain relative to the original signal, the Classical SLM, GA-SLM, and WOA-SLM achieve improvements of 3.5 dB, 3.7 dB, and 3.8 dB, respectively. The proposed HWOA-SLM outperforms both, securing the maximum gain of 3.9 dB.

5.2. Power Saving Efficiency

In comparing methods for PAPR reduction, it is essential to consider power-saving performance. This performance is defined by:

$$Power saving \left(\%\right)={\displaystyle \frac{{PAPR}_{linear}^{Orig}-{PAPR}_{linear}^{reduced}}{{PAPR}_{linear}^{Orig}}} \times 100$$

(20)

Table 3. Comparative analysis of PAPR reduction performance and Power Saving capabilities (at CCDF = 10⁻²).

Algorithm	PAPR (dB)	PAPR Reduction (dB)	Power Saving (%)
Original Signal	9.9	-	-
SLM	6.4	3.5	55.3%
GA-SLM	6.2	3.7	57.3%
WOA-SLM	6.1	3.8	58.3%
HWOA-SLM	6.0	3.9	59.3%

presents the power saving efficiency of the investigated PAPR reduction techniques relative to the original OFDM signal, which exhibits a high peak power of 9.9 dB. The power saving metric quantifies the percentage reduction in linear scale achieved by each method; for instance, the conventional Classical SLM yields a 55.3% saving, effectively lowering the PAPR by roughly 3.5 dB. The integration of evolutionary computation enhances this performance, with GA-SLM and WOA-SLM achieving savings of 57.3% and 58.3%, respectively, demonstrating the superiority of heuristic searches over random phase selection. Most notably, the proposed HWOA-SLM delivers the highest performance with a 59.3% power saving, corresponding to a substantial 4.0 dB reduction from the baseline, thereby maximizing the power amplifier’s efficiency and ensuring the most robust protection against non-linear distortion.

5.3. Parameter Sensitivity Analysis of the HWOA-SLM Algorithm

Figure 10 illustrates the impact of varying the maximum number of iterations on the PAPR reduction performance of the proposed HWOA-SLM scheme, with the population size held constant at 10. The CCDF curves demonstrate a monotonic improvement in performance as the iteration count increases from 5 to 60, evidenced by the progressive shift of the curves to the left. This trend indicates that a higher number of iterations allows the optimization algorithm sufficient time to exploit the search space more effectively, thereby identifying superior phase rotation vectors that minimize the peak signal power. However, it is observed that the rate of improvement diminishes at higher iteration counts; while there is a significant reduction in PAPR when increasing iterations from 5 to 40, the performance gap narrows between 40 and 60 iterations, suggesting that the algorithm approaches convergence and that further computational investment yields diminishing returns.

Figure 11 depicts the sensitivity of the proposed scheme to variations in population size ($Pop$), ranging from 5 to 50, while maintaining a fixed iteration count of $Maxiter=10$. The results indicate that a larger population size significantly enhances the algorithm’s exploration capability, leading to a lower probability of high PAPR events. As the population increases from 5 to 50, the diversity of the candidate solutions improves, reducing the likelihood of the optimizer getting trapped in local minimum. Notably, the improvement from $Pop=5$ to $Pop=30$ is substantial, highlighting the necessity of a sufficient population size for effective global search. While $Pop=50$ yields the best performance, the incremental gain over $Pop=30$ suggests a trade-off where the additional computational overhead of larger populations must be balanced against the required PAPR threshold.

5.4. Power Spectral Density (PSD) Analysis

Figure 12 presents the PSD comparison of the original OFDM signal against the signals processed by Classical SLM, GA-SLM, WOA-SLM, and the proposed HWOA-SLM technique. The PSD curves for all five scenarios are virtually identical, overlapping significantly across the entire frequency band (from −80 GHz to +80 GHz). This overlap confirms that the phase rotation vectors applied by the SLM-based optimization algorithms do not alter the fundamental power distribution or the spectral mask of the OFDM signal. Specifically, the side lobes and out-of-band radiation levels remain unchanged compared to the original signal. This is a critical validation, demonstrating that the proposed HWOA-SLM method successfully reduces the PAPR without introducing spectral regrowth, non-linear distortion, or violating spectral containment requirements.

5.5. BER Performance Analysis

Figure 13 illustrates the system’s performance in terms of BER against the Optical Signal-to-Noise Ratio (OSNR). The plot compares the proposed HWOA-SLM technique against classical SLM, GA-SLM, WOA-SLM, and the Original OFDM signal. The results demonstrate that the proposed HWOA-SLM scheme achieves superior performance across the entire OSNR range compared to all other benchmarked methods. As the OSNR increases, the BER decreases monotonically for all scenarios; however, the rate of improvement is most significant for the HWOA-SLM. In optical OFDM systems, high PAPR signals are susceptible to non-linear impairments (such as saturation in electrical amplifiers or non-linear fiber effects). By effectively reducing the PAPR, the HWOA-SLM method minimizes these non-linear distortions, thereby improving the effective SNR at the receiver. This is evidenced by the distinct gap between the “Original” curve (blue) and the “HWOA-SLM” curve (black). Quantitatively, to achieve a standard Forward Error Correction (FEC) threshold of $BER={10}^{-5}$ (indicated by the red dashed line), the original OFDM signal requires an OSNR of approximately 43.6 dB. In contrast, the proposed HWOA-SLM achieves the same error rate at an OSNR of approximately 40.8 dB. This represents a significant coding gain of roughly 4 dB.

5.6. Comparative Analysis of Computational Complexity

The computational complexity of the considered PAPR reduction schemes is primarily determined by the number of Inverse Fast Fourier Transform (IFFT) operations required, as this is the most computationally intensive step in the OFDM transmission chain. Let $N$ denote the number of subcarriers.

The baseline Original OFDM system involves a single IFFT operation without any optimization iterations. Consequently, its computational complexity is given by:

$$C_{Original}=O\left(N {\mathit{log}}_{2}N\right)$$

(21)

For the Classical SLM scheme, the complexity is directly proportional to the number of candidate phase vectors generated, denoted by $U$. Since each candidate requires an IFFT operation and a phase multiplication step (represented by $+N$), the total complexity is:

$$C_{SLM}=U\times \left[O\left(N {\mathit{log}}_{2}N\right)+N\right]\approx O\left(U\cdot N {\mathit{log}}_{2}N\right)$$

(22)

The complexity of the GA-SLM depends on the population size, $Po{p}_{GA}$, and the number of generations, $Ge{n}_{GA}$. The algorithm performs an IFFT for every individual in the population across all generations. Thus, the complexity is expressed as:

$$C_{GA}=O\left({(Pop}_{GA}\times {Gen}_{GA}) \times N {\mathit{log}}_{2}N\right)$$

(23)

Similarly, the WOA-SLM is governed by its population size, $Po{p}_{WOA}$, and the number of iterations, $Maxite{r}_{WOA}$. The computational cost is defined as:

$$C_{WOA}=O\left(\left(Po{p}_{WOA}\times Maxite{r}_{WOA}\right)\times N {\mathit{log}}_{2}N\right)$$

(24)

Finally, the proposed HWOA-SLM scheme divides the computational budget into a random seeding phase and an evolutionary WOA phase. Letting $M_{seed}$ represent the number of random searches in the seeding phase, $Po{p}_{HWOA}$ the population size, and $Maxite{r}_{HWOA}$ the number of iterations in the optimization phase, the total complexity is calculated as:

$$C_{Hybrid}=\left[O\left(M_{seed}+\left(Po{p}_{HWOA}\times Maxite{r}_{HWOA}\right)\times N {\mathit{log}}_{2}N\right)\right]$$

(25)

6. Conclusions

This research presents the analysis and simulation of a cost-effective optical IM/DD OFDM communication system. The system was implemented using MATLAB2024a and OptiSystem 23 software, deploying 4-QAM modulation scheme over a propagation distance of 120 km. Detailed investigations were conducted on the signal characteristics, including the constellation diagrams of 4-QAM and the spectral properties of both the optical and electrical OFDM signals. To address the critical issue of high PAPR in OFDM systems, a novel HWOA-SLM was proposed. Simulation results confirm that the HWOA-SLM effectively mitigates high peak power, achieving a total PAPR reduction of 3.9 dB compared to the original unoptimized signal at a CCDF of ${10}^{-2}$. Furthermore, regarding algorithmic efficiency, the proposed hybrid strategy demonstrates consistent superiority, outperforming standard WOA and GA-based SLM techniques by approximately 0.1 to 0.4 dB under identical computational budgets, while ensuring robust convergence and lower bit error rates. A parametric analysis of the HWOA-SLM algorithm indicates that performance is sensitive to optimization constraints. Increasing the population size (from 5 to 50) and the maximum number of iterations (from 5 to 60) resulted in a consistent improvement in the algorithm’s convergence, leading to lower PAPR values. This reduction in PAPR enhances signal integrity and minimizes non-linear distortions, thereby improving the overall reliability and spectral efficiency of the optical communication system. Furthermore, the proposed technique exhibits superior BER performance compared to existing methods. In terms of OSNR requirements, the HWOA-SLM provides a coding gain of 0.63 dB over standard WOA-SLM, 1.31 dB over GA-SLM, and 2 dB over conventional SLM. While various improved WOA variants exist in the literature for continuous optimization problems, this study specifically compares HWOA-SLM against standard WOA and GA benchmarks to isolate the gains provided by our specific discrete mapping and greedy update mechanisms. Future work may extend this comparison to other specific improved WOA variants. Finally, the proposed method demonstrates a favorable trade-off between performance and computational cost, offering higher reduction efficiency than comparable schemes at equivalent search complexities.