Generation of Soliton Molecules in an All-Normal Dispersion Yb-Doped Fiber Laser with a Lyot Filter

Abstract

Soliton molecules offer practical advantages in high-speed optical communication, precision spectroscopy, and micromachining. In all-normal dispersion fiber lasers, group velocity dispersion broadens the pulse duration, hindering the attainment of the nonlinearity dispersion balance essential for soliton molecule formation. Consequently, the generation of soliton molecules in such lasers is a technically demanding task. Here, we report an all-normal dispersion fiber laser, mode-locked via nonlinear polarization evolution (NPE) and Lyot filtering. By adjusting the intracavity polarization, this setup allows direct control over pulse interactions, enabling the generation of stable soliton molecules, soliton bound states, and multipulse states. A spectral modulation period of up to 0.95 nm is achieved. In addition, different types of solitons, such as soliton singlets and soliton molecules in tightly and loosely bound states, are observed.

1. Introduction

In recent years, soliton molecules have attracted attention because of their unique properties. It has been demonstrated that soliton molecules have significant application potential in high-speed optical communications, precision spectroscopy, and micromachining owing to their unique interaction mechanisms in temporal and spectral domains [1]. Recent comprehensive reviews have summarized the current state of the art in soliton molecule research, highlighting advances in soliton molecule dynamics, including inter-pulse coupling mechanisms and environmental stability, as well as emerging regulatory strategies. These reviews also emphasize the critical role of soliton molecules in cutting-edge applications such as reconfigurable all-optical signal processing and ultrahigh density data transmission [2]. The rich evolution dynamics from individual solitons to soliton molecules, and even soliton crystals, further reveals the collective nonlinear behavior of optical pulses, enriching the fundamental understanding of soliton many-body physics [3]. Moreover, they identify the controllable generation of soliton molecules in all-normal dispersion fiber lasers as a remaining key challenge, attributed to the difficulty of balancing dispersion and nonlinearity, further underscoring the relevance of and necessity for the present work. Within mode-locked soliton lasers, the generation of soliton molecules is closely related to the dynamic energy balance within the cavity [4]. The temporal separation between these pulses, which often exhibits an uneven distribution, arises from complex nonlinear interactions among solitons [5]. These interactions include direct soliton collisions [6], acousto-optic effects [7], and modulation induced by gain medium relaxation oscillations [8]. Beyond conventional free-propagating solitons, recent studies have demonstrated the existence of accelerating solitary wavepackets, whose unique trajectory dynamics provide new insight into the manipulation of localized wavepackets under external potentials [9]. Among the various routes to soliton molecule formation, the broadband nature of NPE mode locking serves as a distinctive advantage. Specifically, this inherent bandwidth advantage enables exceptional tunability and precise control capability over the generation and inter-pulse binding of soliton molecules [10].

The NPE technique is widely used in passive mode locking, which serves as a convenient approach for soliton manipulation [11,12]. In an NPE mode-locked laser, the synergistic effect of intracavity peak power clamping and intense pump power injection constitutes a critical factor that induces the multipulse phenomenon [13]. In the case of anomalous dispersion in the cavity, conventional solitons emerge when a balance between the dispersion and nonlinear effects is achieved. In contrast, the formation mechanism of dissipative solitons is distinct, as they are typically generated in all-normal dispersion fiber lasers [14]. The generation of dissipative solitons requires a dynamic balance among four factors: dispersion, nonlinearity, gain, and loss [15,16]. Simultaneously, amplitude modulation induced by spectral filtering plays an indispensable role in their formation [15]. Despite different generation mechanisms, dissipative soliton fiber lasers produce bound states and harmonic mode locking like conventional soliton lasers [17,18,19].

Bound states comprising dual dissipative solitons have also been observed in an all-normal dispersion fiber laser mode-locked via NPE. A. Zavyalov et al. have utilized NPE mode locking to generate dissipative soliton molecules with independently evolving or flipping phases in an all-fiber laser cavity, with the dynamics characterized via spectral interferometry [20]. Bound dissipative-pulse evolution in an all-normal dispersion fiber laser through numerical and experimental investigations was reported by X. Liu et al., who used a 45° tilted fiber grating as a saturable absorber [21]. X. He et al. demonstrated bound states of dissipative solitons in the single-mode Yb-doped fiber laser that employed NPE mode locking [22]. However, in the case of all-normal dispersion fiber lasers, the formation of soliton molecules is generally challenging to observe directly. This difficulty arises because the dispersive effect leads to significant pulse broadening, thereby hindering the attainment of the necessary nonlinearity–dispersion balance required for soliton molecule formation.

As a typical polarization filter based on the birefringence effect, the Lyot filter is widely used in mode-locked fiber lasers for flexible spectral selection and polarization regulation. It can suppress spectral sidebands, tune operating wavelengths, and form periodic low loss temporal windows in the cavity, which is conducive to optimizing soliton nonlinear interaction and promoting the formation of stable soliton bound states [18,22]. Its simple structure and adjustable transmission spectrum via polarization control make it an ideal component for soliton molecule regulation in all-normal dispersion fiber lasers.

In this work, an all-normal dispersion fiber laser is constructed to generate bound states of solitons, with a Lyot filter employed to stimulate the formation of soliton molecules. The Lyot filter exploits the birefringence effect of light to achieve wavelength-selective optical filtering [23,24]. The polarization controller is designed to act as both a polarization-selective element for NPE mode locking and a critical component of the Lyot filter. The Lyot filter facilitates the formation of soliton molecules by providing periodic spectral selection and temporal modulation. This approach mitigates issues such as pulse energy dispersion and nonlinear competition, improves the synchronization of pulse intervals, and enables stable coexistence of multiple pulses through polarization control, thereby facilitating the formation of robust soliton bound states. In this manner, the laser delivers a stable mode-locked pulse train with a repetition rate of 75.8 MHz and a spectral modulation period in the range of 0.3 nm to 0.95 nm.

2. Experimental Setup and Principle

In Figure 1, the configuration of the solid-state fiber laser incorporates a fiber component and a free-space component. The fiber component consists of a 55 cm-long Yb-doped fiber as the gain fiber (Coractive-Yb-550, CorActive High-Tech, Québec City, Canada), a 13.2 cm-long polarization-maintaining fiber (PMF), and two Wavelength Division Multiplexing (WDM) collimators. The gain fiber has a core diameter of 4 μm with an absorption coefficient of ~1750 dB/m at 976 nm. Both ends of the gain fiber are integrated into the WDM collimators to minimize the length of the laser cavity. The WDM collimators have a coupling loss of ~1 dB. Two 900 mW 976 nm pump lasers are coupled into both ends of the gain fiber through WDM collimators. Bidirectional pumping is adopted to increase the intracavity energy. An isolator is employed after each pump laser, thereby preventing damage to the pump lasers from back reflections. The transmission grating (IBSEN, Farum, Denmark, 1250 lines/mm, polarization-dependent) employed in the setup exhibits a diffraction efficiency exceeding 90% across the 1000–1100 nm operating band.

Unidirectional operation is guaranteed by a Faraday rotator combined with a half-wave plate and two PBSs. The orientation of the wave plates is adjusted to maintain high-energy pulses circulating inside the cavity while low-energy pulses are reflected by the PBSs and serve as the laser output. The gain fiber exhibits a group velocity dispersion of 23 fs²/mm, and the single-mode fiber at 1030 nm contributes 9 fs²/mm. Thus, the fibers yield a net positive dispersion of 0.02795 ps² in this cavity. With no additional dispersion-compensating elements present, the total net cavity dispersion remains positive. NPE mode locking is achieved by rotating the optical axis angles of QWP1, QWP2, HWP1, and HWP2 to introduce appropriate losses and mimic the effect of a saturable absorber.

In Figure 1, the Lyot filter consists of a PMF, QWP1, QWP2, HWP1, and HWP2. As illustrated in Figure 2, the operating scheme of the Lyot filter is as follows: the angle θ1 between the fast axis of the PMF and the polarizer is adjusted by a polarization controller (PC1), while the angle θ2 between the fast axis and the analyzer is modified by a polarization controller (PC2). If the polarization direction of the input light is aligned along with the y-axis, it can be expressed as the Jones vector $\left[\begin{array}{c}1\\ 0\end{array}\right]$. PC1 transforms linearly polarized light into elliptically polarized light, with its transmission matrix expressed as follows:

$${\mathrm{R}}_1=\left[\begin{array}{cc}\cos {\mathsf{\theta }}_1\mathrm{e}\mathrm{x}\mathrm{p}\left(\mathrm{i}∆{\mathsf{\varphi }}_1/2\right)& \sin {\mathsf{\theta }}_1\mathrm{e}\mathrm{x}\mathrm{p}\left(\mathrm{i}∆{\mathsf{\varphi }}_1/2\right)\\ -\sin {\mathsf{\theta }}_1\mathrm{e}\mathrm{x}\mathrm{p}\left(\mathrm{i}∆{\mathsf{\varphi }}_1/2\right)& \cos {\mathsf{\theta }}_1\mathrm{e}\mathrm{x}\mathrm{p}\left(-\mathrm{i}∆{\mathsf{\varphi }}_1/2\right)\end{array}\right]$$

(1)

where ${\mathsf{\theta }}_1$ is the angle between the fast axis and the slow axis of the PMF.

$∆{\mathsf{\varphi }}_1$ is the phase delay introduced by PC1. During light propagation through a birefringent polarization-maintaining fiber, in the absence of gain, the transmission matrix, $\mathrm{P}$, between the fast and slow axes of the polarization-maintaining fiber is given by:

$$\mathrm{P}=\left[\begin{array}{cc}\mathrm{e}\mathrm{x}\mathrm{p}\left(\mathrm{i}∆{\varphi }^{\prime }/2\right)& 0\\ 0& \mathrm{e}\mathrm{x}\mathrm{p}\left(\mathrm{i}∆{\varphi }^{\prime }/2\right)\end{array}\right]$$

(2)

$∆{\mathsf{\varphi }}^{\prime }=2\mathsf{\pi }\mathrm{L}{\mathrm{B}}_{\mathrm{m}}/\mathsf{\lambda }$ represents the phase delay induced by the birefringent polarization-maintaining fiber, where $\mathrm{L}$ is the fiber length, ${\mathrm{B}}_{\mathrm{m}}$ is the birefringence coefficient, and $\mathsf{\lambda }$ is the wavelength.

The analyzer ultimately determines the polarization state of the output light, with its transmission matrix expressed as:

$${\mathrm{R}}_2=\left[\begin{array}{cc}\cos {\mathsf{\theta }}_2\mathrm{e}\mathrm{x}\mathrm{p}\left(\mathrm{i}∆{\mathsf{\varphi }}_2/2\right)& \sin {\mathsf{\theta }}_2\mathrm{e}\mathrm{x}\mathrm{p}\left(\mathrm{i}∆{\mathsf{\varphi }}_2/2\right)\\ -\sin {\mathsf{\theta }}_2\mathrm{e}\mathrm{x}\mathrm{p}\left(\mathrm{i}∆{\mathsf{\varphi }}_2/2\right)& \cos {\mathsf{\theta }}_2\mathrm{e}\mathrm{x}\mathrm{p}\left(-\mathrm{i}∆{\mathsf{\varphi }}_2/2\right)\end{array}\right]$$

(3)

where ${\mathsf{\theta }}_2$ is the angle between analyzer and the fast axis, and $∆{\varphi }_2$ is the phase delay introduced by PC2. The overall transmission coefficient is given as follows:

$$\mathrm{T}={\mathrm{R}}_2\mathrm{P}{\mathrm{R}}_1\left[\begin{array}{c}1\\ 0\end{array}\right]$$

(4)

The overall transmission is defined as:

$$\begin{gathered} {\left|\mathrm{T}\right|}^2={\mathrm{c}\mathrm{o}\mathrm{s}}^2{\mathsf{\theta }}_1{\mathrm{c}\mathrm{o}\mathrm{s}}^2{\mathsf{\theta }}_2+{\mathrm{s}\mathrm{i}\mathrm{n}}^2{\mathsf{\theta }}_1{\mathrm{s}\mathrm{i}\mathrm{n}}^2{\mathsf{\theta }}_2 \\ +{\displaystyle \frac{1}{2}}\sin 2{\mathsf{\theta }}_1\sin 2{\mathsf{\theta }}_2\cos \left(∆{\mathsf{\varphi }}_1+∆{\mathsf{\varphi }}_2+∆{\mathsf{\varphi }}^{\prime }\right) \end{gathered}$$

(5)

According to Equation (5), the transmission coefficient, T, depends on the phase delay, $∆{\mathsf{\varphi }}_1$, $∆{\mathsf{\varphi }}_2$, and $∆{\mathsf{\varphi }}^{\prime }$ introduced by PC1, PC2, and the PMF. Phase delay, $∆{\mathsf{\varphi }}^{\prime }$, is determined by $\mathrm{L}$, ${\mathrm{B}}_{\mathrm{m}}$, and $\mathsf{\lambda }$. When $\mathrm{L}$ and $\mathsf{\lambda }$ remain constant, the transmission coefficient, $\mathrm{T}$, depends on the value of the angle between the fast axis and the slow axis of the PMF, the angle between the analyzer and the fast axis, and the birefringence coefficient. Consequently, transmittance tuning of the Lyot filter is achieved by adjusting the PC rotation angles to alter the intracavity polarization state or by applying controlled stress to the PMF to modulate ${\mathrm{B}}_{\mathrm{m}}$.

To address insufficient theoretical rigor and quantitative verification, numerical simulations based on the split-step Fourier method were conducted with the Lyot filter integrated. Pulses propagated 500 roundtrips in the simulated all-normal dispersion Yb fiber laser cavity to ensure dissipative soliton stability. The Lyot filter’s Jones matrix model was optimized to quantitatively link its PMF-derived parameters (length and birefringence) to experimental spectral fringe spacings and pulse separations. Simulations with modulation periods of 0.3 nm, 0.6 nm, and 0.9 nm showed pulse splitting with separations of 0.741 ns, 0.889 ns, and 1.141 ns, consistent with the experiments, confirming a causal relationship between the filter’s periodic transmission windows and soliton locking (Figure 3a–f). The pulse width extraction (FWHM via linear interpolation at half-maximum intensity) is explicitly defined, and references were rechecked to avoid overstated applications, strengthening the work’s theoretical and quantitative support.

3. Results and Discussion

In Figure 4a, the pulse train of the output soliton molecules was recorded by a 2.5 GHz photodetector and a 4 GHz oscilloscope, with a time interval of 13.2 ns. Mode locking was achieved at 0.35 W pump power. The fundamental repetition rate of the soliton molecules was measured to be 75.8 MHz in Figure 4b, with a signal-to-background ratio of 80 dB, which matched the value calculated from cavity length. There were no other frequencies generated before or after the fundamental frequency spectral line in Figure 3b, which indicated that the laser operated in a stable mode-locked state. Both the pulse train and radio frequency (RF) spectrum exhibited high stability without satellite pulses or spurious RF spectral lines, indicating stable fundamental mode locking.

The pulse characteristics of the dissipative soliton bound state were measured at a pump power of 0.7 W. The optical signal from the laser output was characterized using a spectrum analyzer (Agilent, N9000A, Agilent Technologies, Inc., Santa Clara, CA, USA) connected to a photodetector. Figure 5a shows the output spectrum with a 0.3 nm spectral modulation period. Figure 5b displays the corresponding autocorrelation trace, featuring three widely spaced peaks with an intensity ratio of approximately 0.8:2:0.8, deviating from the ideal 1:2:1 ratio. This discrepancy arises from spectral interference fringes and relative phase variations within the soliton bound state, which modify the intensity distribution of the soliton. The pulse separation of 11.6 ps in Figure 5b is consistent with the 0.3 nm spectral modulation period. Furthermore, this 11.6 ps separation is approximately 8.8 times the pulse duration (1.32 ps), indicating a loosely bound soliton state. Loosely bound solitons exhibit weak inter-soliton interactions, distributed energy profiles, and large separations. Their weak stability makes them susceptible to minor environmental perturbations. Unlike tightly bound states with well-defined discrete separation distances, loosely bound solitons demonstrate variable spacing [5].

When the pump power was increased to 1.7 W, the corresponding pulse characteristics of the dissipative soliton bound state were shown in Figure 4. A spectral modulation period of 0.95 nm for the bound state is indicated in Figure 5c, corresponding to a pulse separation of 3.7 ps in the autocorrelation trace. This 3.7 ps separation is approximately 2.3 times the pulse duration (1.64 ps), indicating a tightly bound soliton state. Such solitons typically form under stronger interactions, leading to reduced inter-soliton distances and enhanced stability during soliton evolution. Unlike loosely bound solitons, tightly bound states maintain fixed discrete pulse separations that remain robust against external perturbations [5]. The observed transition from loosely to tightly bound states with increasing pump power arises from enhanced nonlinear effects in the laser field. This intensifies inter-soliton interactions, progressively reducing soliton separation until the tightly bound configuration is achieved. The nonlinear effects dominating the soliton molecule formation and their binding state transition in this all-normal dispersion Yb-doped fiber laser are primarily the fiber Kerr nonlinearity, including self-phase modulation (SPM) and cross-phase modulation (XPM). SPM induces nonlinear phase shift in individual intracavity pulses, shaping the spectral modulation characteristics of soliton molecules and determining the basic interaction between soliton pulses; XPM further modulates the mutual attractive/repulsive forces between adjacent soliton pulses when multiple pulses circulate in the cavity. The increase in pump power elevates the intracavity pulse intensity, which directly enhances the strength of Kerr nonlinearity, thus amplifying the inter-soliton attractive interaction and driving the soliton molecules to transform from loosely bound state to tightly bound state.

The increase in pump power promotes the intracavity energy. The Lyot filter mitigates pulse energy dispersion and nonlinear competition in NPE mode locking through periodic spectral selection and temporal modulation [25,26]. It enhances pulse-to-pulse interval synchronization and establishes fixed temporal relationships among multiple pulses via polarization control, leading to the formation of robust bound state solitons. The wavelength-dependent transmission of the Lyot filter, arising from its birefringence, effectively suppresses optical spectrum sidebands while preserving the central spectral portion [27].

In the spectral domain, self-phase modulation produces a characteristic modulation of the pulse spectrum. In the time domain, in accordance with the time–frequency duality, this spectral modulation gives rise to oscillating waveforms at both the leading and trailing edges of the pulse [28]. When multiple pulses circulate within the cavity, they interact via cross-phase modulation (XPM) and direct field coupling. A balance between attractive and repulsive forces typically exists between them. This equilibrium is maintained via the Lyot filter. Its periodic transmission spectrum generates low loss temporal windows. When soliton separations coincide with these slots, cavity loss is minimized, thereby passively locking the pulses into a synchronous, stable configuration [20]. The period of its transmission spectrum corresponds to a fixed temporal pulse interval. When the pulse spacing matches the inherent period of the Lyot filter, the overall transmission loss of the pulse sequence is minimized. This phenomenon results from precise spectral and temporal matching between the pulse train and the filter characteristics: all pulse spectra coincide with the filter’s maximum transmittance band, while their temporal arrangement minimizes transmission losses [29]. The overall transmission loss is minimized when the pulse interval matches this optimal value, which facilitates the formation of stable bound state pulses.

When the pump power was beyond 1.3 W, the spectral fringe spacing stabilized at 0.95 nm because the nonlinear effects saturated, establishing a sustained balance between dispersion and nonlinearity, as shown in Figure 6a. In this regime, further increases in pump power left the stable spectral fringe spacing unchanged, indicating that the repulsive and attractive forces between solitons had reached a dynamic balance. The good stability of the bound state solitons was demonstrated by a 6 h output power monitoring period in Figure 6b, which revealed a root mean square jitter of 0.297%, confirming good power stability.

Comparing the key parameters of soliton molecules/bound dissipative solitons in all-normal dispersion Yb-doped fiber lasers reported in recent years with our work was shown in

Table 1. Key parameter comparison of soliton-related bound states in all-normal dispersion Yb-doped fiber lasers.

Reference	Spectral Modulation Period	Soliton Binding State	Repetition Rate (MHz)	Power Stability (RMS Jitter)
He et al., 2016 [22]	Spectral interference fringes	Tightly/loosely bound	34.3	0.012% (1.5 h)
Pu et al., 2024 [25]	Single pulse	Loosely bound	29.7	Not reported
Zhou et al., 2025 [10]	0.2–0.6 nm	Loosely bound	52.3	0.58%
This work	0.3–0.95 nm	Tightly/loosely bound	75.8	0.297% (6 h)

. The existing studies, mainly based on NPE mode locking, have achieved a spectral modulation period of 0.2–0.6 nm, repetition rate of 29.7–52.3 MHz, and power stability RMS jitter of 0.012–0.58% [10,22,25]. Our work achieves a larger spectral modulation period (0.3–0.95 nm), a higher repetition rate (75.8 MHz), and good power stability (0.297% RMS jitter) by combining NPE with a Lyot filter. More importantly, we realize the continuous and controllable transition from loosely to tightly bound soliton molecules by adjusting pump power, a feature not reported in the aforementioned studies, providing a more effective regulation method for soliton molecule dynamic control.

4. Conclusions

In summary, we demonstrate a Yb-doped mode-locked fiber laser, which generates bound states of solitons with a repetition rate of 75.8 MHz. This is achieved through the cavity design that incorporates a Lyot filter for precise spectral control and leverages NPE to facilitate robust mode locking. Tightly and loosely bound solitons are observed as the pumping power and the orientation of the waveplates are precisely controlled. The findings not only advance the fundamental understanding of multi-soliton dynamics but also highlight the potential applications in high-density optical communications and ultrafast micromachining. The controllable generation of soliton molecules demonstrated in this work provides a fundamental building block for reconfigurable all-optical signal processing units, enabling practical implementations of optical logic gates, buffers, and delay lines in integrated photonic circuits.