Hybrid THz/FSO Transmission System with a Shared Photonic Transmitter Enabled by PMMA-Based Beam Combining

Abstract

Hybrid terahertz (THz)/free-space optical (FSO) systems offer a promising paradigm for high-capacity, all-weather wireless communication, yet their deployment is often hindered by the bulky size and high complexity of discrete transceivers. This paper experimentally demonstrates a low-complexity hybrid THz/FSO transmission architecture featuring a unified photonic transmitter. By leveraging a polymethyl methacrylate (PMMA) plate serving as a dichroic beam combiner—which reflects the 1550 nm optical signal while transmitting the 300 GHz THz signal—we realize simultaneous signal propagation over a shared aperture and link. Photonics-aided techniques are employed to generate both carriers, ensuring system integration and coherence. The experimental results verify that both the THz and FSO links independently support 30-GBaud quadrature phase-shift keying (QPSK) transmission over a 10-m wireless distance, achieving a net data rate of 60 Gbps per link while satisfying the 7% hard-decision forward error correction (HD-FEC) threshold of 3.8 × 10⁻³. This work validates the feasibility of shared-transmitter designs and provides a compact, cost-effective solution for future high-speed fronthaul/backhaul networks.

1. Introduction

Terahertz communication has emerged as a pivotal technology for next-generation wireless systems. Its ultra-broadband characteristics enable ultra-high-speed data transmission, while offering key advantages such as high resolution, low latency, and strong penetration capabilities [1,2]. These attributes position THz communication as a promising solution for a wide range of applications, including 6G networks, satellite communication systems, and the Internet of Things (IoT). Recent reports have demonstrated the successful transmission of 2 Gbit/s over a distance of 4.6 km within the terahertz frequency band [3,4]. In recent years, photonics-aided methods have significantly advanced wireless communication by enabling flexible and high-capacity signal transmissions across diverse frequency bands (from microwave to terahertz) and modulation schemes, with photonic generation of THz signals emerging as the dominant approach due to its inherent bandwidth scalability and compatibility with fiber-optic infrastructures [5,6,7]. However, despite its immense potential, THz communication faces critical challenges. Atmospheric absorption, particularly due to water vapor, leads to significant free-space path loss (FSPL) and frequency-dependent attenuation, limiting achievable transmission distances and data rates. Furthermore, THz signals are prone to multipath interference and fading effects, which can severely degrade performance in complex propagation environments.

Free-space optical communication plays a crucial role in applications requiring high-speed, high-bandwidth, secure, and reliable data transmission because of its abundant spectrum resources, freedom from spectrum licensing, high spectral efficiency, and narrow beam angles [8]. However, FSO systems are extremely sensitive to atmospheric turbulence and adverse weather conditions (e.g., fog and snow), which can severely limit their transmission range and degrade communication performance [9]. In addition, FSO transmissions are highly sensitive to pointing errors, requiring precise alignment between the transmitter and receiver, and even slight perturbations in the environment can significantly affect the stability and quality of the transmission. Considering the advantages and disadvantages of THz and FSO transmission, THz is subject to huge path loss and molecular absorption loss while FSO is affected by atmospheric turbulence and fog, if the two are fused for transmission, it could overcome the problem of the low reliability of a single channel in complex environments.

In recent years, such hybrid THz/FSO transmission technology, which integrates the complementary advantages of optical communication and radio frequency (RF) systems, has emerged as a prominent research focus worldwide [10]. Hybrid THz/FSO systems are pivotal in enabling seamless satellite-terrestrial integrated networks. They bridge the “last-mile” connectivity gap between satellites, high-altitude platforms (HAPs), and ground stations through dual-mode synergy. This architecture supports global 6G coverage, particularly for maritime and aviation connectivity, where traditional fiber is inaccessible, achieving high availability through THz/FSO diversity in rain and turbulence. In early hybrid wireless systems, channel diversity was commonly exploited through conventional selection combining (SC), maximum-ratio combining (MRC), or threshold-based link switching, in which the link with the better instantaneous channel quality was selected or weighted according to predefined criteria [11,12,13]. These approaches provide a relatively simple means of improving link availability by exploiting the complementary propagation characteristics of heterogeneous channels. However, their performance can be limited when the channel state varies rapidly, since hard switching relies on instantaneous or threshold-based channel decisions and does not fully exploit the information contained in the received optical fields.

Recent advances in FSO communication have therefore increasingly focused on channel-aware sensing and adaptive signal processing. Structured optical fields provide additional degrees of freedom for probing and characterizing atmospheric disturbances, enabling the propagation environment to be sensed rather than treated solely as an unknown channel impairment [14,15]. In particular, recent multi-link FSO systems have demonstrated closed-loop turbulence sensing and mitigation, in which channel conditions are actively sensed and the transmission paths are adaptively controlled. More recently, attention-based neural combining has been introduced for FSO diversity reception, enabling information from multiple turbulence-impaired links to be adaptively fused rather than simply selecting one link according to a fixed threshold [16]. These developments indicate a transition from conventional hard selection toward sensing-assisted, adaptive, and soft-combining strategies for robust FSO diversity reception.

Nevertheless, most of these advanced adaptive FSO diversity schemes focus on multiple optical links and require dedicated sensing, beam-control, or receiver-side processing architectures, and existing research has concentrated mainly on the performance analysis of hybrid networks [17,18,19,20]. Their integration with heterogeneous THz/FSO systems, particularly architectures in which the THz and FSO signals share a common photonic transmitter and physical aperture, remains relatively underexplored. From a hardware-integration perspective, several studies have explored shared-aperture or dual-purpose antenna architectures. However, these approaches involve relatively large antenna structures and have demonstrated limited transmission rates [21,22]. To further reduce transmitter complexity, one current approach leverages the incomplete photoelectric conversion characteristics of unitraveling-carrier photodiodes (UTC-PDs). Through this methodology, researchers have achieved integrated signal transmission at a 1 Tb/s bit rate over a 50-m wireless distance [23]. Additionally, we previously proposed a shared-transmitter architecture for RF/FSO signal transmission. We use the properties of light and electricity to share the transmitter and the transmitter link, with the insertion of a media board [24,25,26,27]. However, comprehensive performance analysis of this design in THz/FSO integrated systems remains unexplored. Therefore, this paper can be considered as an extension of our prior series of work on this research trajectory. In addition, the present work focuses on establishing the feasibility of a compact shared-transmitter THz/FSO architecture and therefore adopts selection combining as a baseline analytical strategy to characterize the complementary reliability of the two heterogeneous links. More sophisticated channel-sensing and adaptive/soft-combining techniques can be incorporated into the proposed architecture in future work to further improve its performance under dynamically varying atmospheric conditions.

In this paper, we propose a hybrid THz/FSO transmission system that enables THz and FSO links to share a transmitter simultaneously, greatly reducing the complexity and weight of the fusion system. We have analyzed the system and built a system for experiments to achieve proof-of-principle for this architecture. The THz band is set at 300 GHz and the experiments demonstrate that each link in this hybrid system can support 60 Gbps signal transmission and is under the HD-FEC threshold of 3.8 × 10⁻³. This work presents an experimentally demonstrated shared-transmitter architecture for hybrid THz/FSO transmission, which lays the foundation for future high-speed broadband communication in all environments.

2. Principle

2.1. THz/FSO Shared Transmitter

To minimize the complexity of the transmitter and facilitate hybrid THz and FSO transmission, we propose an innovative architecture, as illustrated in Figure 1a. In this design, THz signals are generated using photonics-aided technology, while the optical signal produced at the transmitter is split into two separate paths. In the first path, the optical signal is converted into an electrical signal by a photodetector, which is then focused into a THz beam using a lens before passing through a dielectric plate. In the second path, the optical signal propagates through an optical fiber and is shaped into a parallel beam by a fiber collimator. This beam is then reflected off a material plate to produce the FSO signal. By carefully adjusting the angle of the dielectric plate, the architecture enables the simultaneous transmission of both FSO and THz signals over the same physical link. At the receiving end, the optical receiver is required to be precisely aligned with the transmitter, whereas the electrical receiver can be positioned adjacent to it.

Figure 1. (a) Design of the hybrid THz/FSO structure with a shared transmitter based on photonics-aided technology. (b) Absorption coefficient of PMMA.

A critical component of the proposed architecture is the dielectric plate, which needs to provide sufficient transmission for the 300 GHz THz signal while effectively reflecting the 1550 nm optical signal. Polymethyl methacrylate (PMMA) is selected as the beam-combining medium owing to its favorable frequency-dependent electromagnetic properties and low fabrication complexity. Experimentally, under an optical input power of 14 dBm at 1550 nm, the measured transmitted and reflected optical powers are approximately 12 dBm and 2 dBm, respectively. As shown in Figure 1b, the THz absorption coefficient of PMMA is frequency dependent, reaching approximately 27 cm⁻¹ around 0.7 THz, while decreasing toward lower frequencies below 500 GHz, which is favorable for the transmission of the 300 GHz THz signal [28]. These complementary optical and THz characteristics enable PMMA to serve as a simple beam-combining element for the proposed shared-aperture architecture. Although the optical reflection efficiency of the bare PMMA plate is not optimized, wavelength-selective high-reflection or composite coatings could be introduced in future work to improve the 1550 nm reflection efficiency while preserving low attenuation at 300 GHz.

2.2. Hybrid THz/FSO Link Model

For the THz link in the proposed shared-aperture transmission architecture, the received complex baseband signal before digital signal processing (DSP) can be expressed as

$$y_{\mathrm{THz}}=\sqrt{P_T}{h}_l{h}_{T}s+n_T,$$

(1)

where $y_{\mathrm{THz}}$ denotes the equivalent complex-valued received THz baseband signal after free-space propagation and channel impairments but before receiver-side DSP. s is the transmitted symbol with unit energy, $P_T$ is the transmit power, $h_l$ is the path loss, and $h_T=h_P{h}_f$ is the THz channel-fading coefficient. $h_P$ is the coefficient for pointing errors, and $h_f$ is the coefficient for channel fading. $n_T$ represents the additive white Gaussian noise (AWGN), which has 0 mean and variance ${\sigma }_T^2$. This model describes the principal factors determining the received THz signal quality and provides the basis for the subsequent signal-to-noise ratio (SNR) and link-performance analysis.

The instantaneous received SNR ${\gamma }_T$ at the THz receiver is given as

$${\gamma }_T={\displaystyle \frac{P_T{h}_l^2{∥h_T∥}^2}{{\sigma }_T^2}}.$$

(2)

Equation (2) relates the received THz SNR to the effective transmit power, propagation loss, channel fading, and receiver noise. In the present experiment, the THz carrier is generated using a photonics-aided UTC-PD, and the recovered signal is down-converted to a 15.68 GHz intermediate frequency before DSP and bit-error-rate (BER) evaluation. Therefore, the measured BER reflects the combined influence of the effective THz received power, propagation loss, receiver noise, and residual channel impairments described by Equation (2). It should be noted that the experimental input power corresponds to the optical power launched into the UTC-PD, rather than the generated RF power at 300 GHz. Accordingly, $P_T$ in Equation (2) represents the effective THz transmit-power term in the analytical model and should not be directly equated with the optical input power used as the experimental control variable.

At terahertz frequencies, the path loss $h_l$ can be given as [18]

$$h_l={\displaystyle \frac{c\sqrt{G_{t,T}{G}_{r,T}}}{4\pi f_T{L}_T}}exp[-{\displaystyle \frac{L_T}{2}}({\sum }_i{F}_i(f_T,v)+G(f_T,v))],$$

(3)

where $G_{t,T}$ and $G_{r,T}$ are the antenna gains at the transmitting and receiving ends, respectively, $f_T$ is the operating frequency, c is the speed of light in vacuum, and $L_T$ is the transmission distance for the THz link. The atmospheric absorption characteristics are frequency dependent and can be given by the polynomials F and G, and v is defined according to its role in the atmospheric absorption model [29]. Due to the influence of the pointing error, the THz channel-fading coefficient $h_T$ will experience $\alpha -\mu$ decay, where $\alpha$ represents the non-linearity parameter of the propagation environment and $\mu$ is related to the number of multipath clusters. Both $\alpha$ and $\mu$ are positive real numbers, serving as the channel-fading parameters. The probability density function (PDF) of $h_T$ can be expressed as [19]

$$f_{h_T}\left(x\right)={\displaystyle \frac{{\xi }_T^2{\mu }^{\frac{{\xi }_T^2}{\alpha }}{x}^{{\xi }_T^2-1}}{A_{0,T}^{{\xi }_T^2}{\mathsf{\Omega }}^{{\xi }_T^2}\mathsf{\Gamma }\left(\mu \right)}}\mathsf{\Gamma }\left({\displaystyle \frac{\alpha \mu -{\xi }_T^2}{\alpha }},{\displaystyle \frac{\mu x^{\alpha }}{A_{0,T}^{\alpha }{\mathsf{\Omega }}^{\alpha }}}\right),$$

(4)

where $\mathsf{\Gamma }\left(·,·\right)$ represents the upper incomplete gamma function. ${\xi }_T$ is the pointing-error parameter, ${\xi }_T={\omega }_{L_{eq}}/2{\sigma }_{s_T}$, ${\omega }_{L_{eq}}$ is the equivalent beam waist and ${\sigma }_{s_T}$ is the jitter standard deviation, and $A_{0,T}$ is the fraction of the collected power without any pointing errors [30]. $\alpha$ and $\mu$ are the channel-fading parameters, and $\mathsf{\Omega }$ is the $\alpha$-root mean of the fading channel envelope. Since ${\gamma }_T=\overline{{\gamma }_T}\Vert h_t{\Vert }^2$, $\overline{{\gamma }_T}$ is the average received SNR, the PDF of the THz link’s SNR can be expressed as

$$f_{{\gamma }_{\mathrm{THz}}}\left({\gamma }_T\right)={\displaystyle \frac{{\xi }_T^2{\mu }^{\frac{{\xi }_T^2}{\alpha }}{\gamma }_T^{\frac{{\xi }_{\mathrm{T}}^2}{2}-1}}{2A_{0,T}^{{\xi }_T^2}{\mathsf{\Omega }}^{{\xi }_T^2}\mathsf{\Gamma }\left(\mu \right){\overline{\gamma }}_T^{\frac{{\xi }_T^2}{2}}}}\mathsf{\Gamma }\left({\displaystyle \frac{\alpha \mu -{\xi }_T^2}{\alpha }},{\displaystyle \frac{\mu {\gamma }_T^{\frac{\alpha }{2}}}{A_{0,T}^{\alpha }{\mathsf{\Omega }}^{\alpha }{\overline{\gamma }}_T^{\frac{\alpha }{2}}}}\right).$$

(5)

Finally, the cumulative distribution function (CDF) of ${\gamma }_T$ is

$$F_{{\gamma }_{\mathrm{THz}}}\left({\gamma }_T\right)={\displaystyle \frac{{\xi }_T^2{\mu }^{\frac{{\xi }_T^2}{\alpha }}{\gamma }_T^{\frac{{\xi }_T^2}{2}}}{\alpha A_{0,T}^{{\xi }_T^2}{\mathsf{\Omega }}^{{\xi }_T^2}\mathsf{\Gamma }\left(\mu \right){\overline{\gamma }}_T^{\frac{{\xi }_T^2}{2}}}}{G}_{2,3}^{2,1}\left({\left.{\displaystyle \frac{\mu x^{\frac{\alpha }{2}}}{A_{0,T}^{\alpha }{\mathsf{\Omega }}^{\alpha }{\overline{\gamma }}_T^{\frac{\alpha }{2}}}}\right|}_{0,{\displaystyle \frac{\alpha \mu -{\xi }_T^2}{\alpha }},-{\displaystyle \frac{{\xi }_T^2}{\alpha }}}^{1-{\displaystyle \frac{{\xi }_T^2}{\alpha }},1}\right),$$

(6)

where $G_{p,q}^{m,n}\left[·\right]$ represents the Meijer-G function [31].

Equations (4)–(6) provide the statistical characterization of the THz channel under pointing errors and fading and are used to establish a general theoretical framework for evaluating the SNR distribution. Since the present experiment is a 10 m controlled laboratory demonstration without statistically varying atmospheric turbulence or pointing-error conditions, these equations are not used for parameter fitting to the measured BER curves. Instead, they provide a theoretical basis for interpreting how propagation loss and channel fluctuations would affect the system performance under more general channel conditions.

In addition, for the FSO link, the received signal $y_{\mathrm{FSO}}$ can be represented as

$$y_{\mathrm{FSO}}=\sqrt{P_F\eta I_{l}I}s+n_F,$$

(7)

Similar to the THz link, Equation (8) relates the FSO SNR to the optical transmit power, atmospheric attenuation, channel fading, optical-to-electrical conversion efficiency, and receiver noise. $P_F$ is the transmit power, and $\eta$ is the coefficient of the optical-to-electrical conversion. $I_l$ is the atmospheric attenuation associated with the FSO link, $I=I_a{I}_p$ is the fading coefficient, $I_a$ is the fading due to atmospheric turbulence, and $I_p$ represents the pointing-error fading coefficient. $n_{\mathrm{F}}$ is the AWGN with mean 0 and variance ${\sigma }_F^2$. The instantaneous received SNR ${\gamma }_F$ at the FSO receiver is given as

$${\gamma }_F={\displaystyle \frac{P_F\eta I_{l}I}{{\sigma }_F^2}}.$$

(8)

According to the Beer–Lambert law, the atmospheric attenuation is given as

$$I_l=exp(-{\sigma }_l{L}_F),$$

(9)

where ${\sigma }_l$ is the attenuation coefficient and $L_F$ is the transmission distance of the FSO link.

The atmospheric turbulence $I_a$ follows a gamma–gamma distribution, and the PDF is given as [18]

$$f_{I_{\mathrm{a}}}\left(x\right)={\displaystyle \frac{2{\left({\alpha }_F{\beta }_F\right)}^{\frac{{\alpha }_F+{\beta }_F}{2}}}{\mathsf{\Gamma }\left({\alpha }_F\right)\mathsf{\Gamma }\left({\beta }_F\right)}}{x}^{{\displaystyle \frac{{\alpha }_F+{\beta }_F}{2}}-1}{K}_{{\alpha }_F-{\beta }_F}\left(2\sqrt{{\alpha }_F{\beta }_{F}x}\right),$$

(10)

where ${\alpha }_F$ and ${\beta }_F$ are the fading coefficients of the atmospheric turbulence, and $K_v(·)$ represents the $v^{th}$-order modified Bessel function of the second kind.

As for the pointing-error fading coefficient $I_p$, similar to THz links [18], the PDF of $I_p$ can be expressed as

$$f_{I_p}\left(x\right)={\displaystyle \frac{{\xi }_F^2}{A_{0,F}^{{\xi }_2^2}}}{x}^{{\xi }_F^2-1},0\le x\le A_{0,F},$$

(11)

where ${\xi }_F$ is the pointing-error parameter and $A_{0,F}$ is the fraction of the collected power without any pointing error. Therefore, taking the above effects into account, $\delta$ is the average received power of the FSO link; the PDF of the FSO link’s SNR can be expressed as

$$f_{{\gamma }_{\mathrm{FSO}}}\left({\gamma }_F\right)={\displaystyle \frac{{\xi }_F^2}{\mathsf{\Gamma }\left({\alpha }_F\right)\mathsf{\Gamma }\left({\beta }_F\right){\gamma }_F}}{\mathrm{G}}_{1,3}^{3,0}\left({\left.{\displaystyle \frac{{\alpha }_F{\beta }_F{\gamma }_F}{A_{0,F}\delta }}\right|}_{{\xi }_F^2,{\alpha }_F,{\beta }_F}^{{\xi }_F^2+1}\right).$$

(12)

Finally, the CDF of ${\gamma }_F$ is given as

$$F_{{\gamma }_{\mathrm{FSO}}}\left({\gamma }_F\right)={\displaystyle \frac{{\xi }_F^2}{\mathsf{\Gamma }\left({\alpha }_F\right)\mathsf{\Gamma }\left({\beta }_F\right)}}{\mathrm{G}}_{2,4}^{3,1}\left({\left.{\displaystyle \frac{{\alpha }_F{\beta }_F{\gamma }_F}{A_{0,F}\delta }}\right|}_{{\xi }_F^2,{\alpha }_F,{\beta }_F,0}^{1,{\xi }_F^2+1}\right).$$

(13)

For the system-level analysis, SC is considered as a baseline strategy in which the link with the higher instantaneous SNR is selected. Under the assumption of statistically independent THz and FSO channel states, the corresponding hybrid SNR can be expressed as ${\gamma }_{\mathrm{hybrid}}=max\left({\gamma }_F,{\gamma }_T\right)$. The resulting CDF describes the theoretical outage behavior of an idealized SC receiver [24]. This formulation is used to illustrate the potential benefit of exploiting the complementary channel characteristics of the two links rather than to represent the dynamic switching mechanism implemented in the present experiment. It can be expressed as

$$\begin{array}{cc}\hfill F_{{\gamma }_{\mathrm{hybrid}}}\left(\gamma \right)=P(max({\gamma }_F,{\gamma }_T)\le \gamma )=F_{{\gamma }_F}\left(\gamma \right)·F_{{\gamma }_T}\left(\gamma \right)& \le \min (F_{{\gamma }_F}\left(\gamma \right),F_{{\gamma }_{\mathrm{T}}}\left(\gamma \right)).\hfill \end{array}$$

(14)

In wireless communication systems, the statistical interpretation of $F\left(\gamma \right)$ corresponds to a quantitative measure assessing the risk of link quality degradation when the SNR deteriorates below a specified threshold $\gamma$.

Finally, the hybrid BER can be expressed as [24]

$$\begin{array}{cc}\hfill P_{BER}& =P({\gamma }_F\ge {\gamma }_T)·P_{BER|F}\left({\gamma }_F\right)+P({\gamma }_T>{\gamma }_F)·P_{BER|T}\left({\gamma }_T\right)\hfill \\ \hfill & ={\int }_0^{\infty }\left[P_{BER|F}\left(x\right)·f_{{\gamma }_F}\left(x\right)·F_{{\gamma }_T}\left(x\right)+P_{BER|T}\left(x\right)·f_{{\gamma }_T}\left(x\right)·F_{{\gamma }_F}\left(x\right)\right]dx\hfill \\ \hfill & \le {\int }_0^{\infty }min\left(P_{BER|F}\left(x\right),P_{BER|T}\left(x\right)\right)·\left[f_{{\gamma }_F}\left(x\right)+f_{{\gamma }_T}\left(x\right)\right]dx\hfill \\ \hfill & =min\left(E\left[P_{BER|F}\right],E\left[P_{BER|T}\right]\right).\hfill \end{array}$$

(15)

It should be noted that the present experimental demonstration independently characterizes the THz and FSO links over the same 10 m propagation distance, rather than implementing real-time channel-state-based switching. Therefore, Equations (14) and (15) are interpreted as theoretical system-level analysis. In practical time-varying environments, hard-threshold SC may introduce frequent switching when the channel qualities fluctuate around the decision threshold. Hysteresis-based switching, predictive channel sensing, adaptive weighting, or soft combining could be employed to mitigate such effects. The analytical results indicate that idealized SC can improve the statistical outage performance by exploiting the complementary channel conditions of the THz and FSO links. The present experiment provides a proof-of-concept validation of the individual THz and FSO transmission capabilities, while real-time adaptive link selection and soft-combining strategies will be investigated in future work.

3. Experimental Setup

As shown in Figure 2, the experimental setup consists of three major modules: an optical transmitter, a THz/FSO link fusion module, and THz/FSO receivers. The physical configuration of the hybrid THz/FSO transmission system is illustrated in Figure 3, demonstrating the integration of FSO and THz links within a single shared transmission path. In the transmitter module, as shown in Figure 4, a pseudorandom binary sequence (PRBS) is generated and undergoes quadrature amplitude modulation (QAM) mapping, root-raised cosine (RRC) filtering, and resampling to match the desired transmission rate. These processes are followed by digital-to-analog conversion (DAC). Once the DSP on the transmitter side is complete, the processed data is loaded onto a 120 GSa/s arbitrary waveform generator (AWG), with the resulting signal amplified by electronic amplifiers (EAs). The optical carrier, centered at 1552.40 nm and generated by external cavity laser-1 (ECL1), is modulated using an in-phase and quadrature (I/Q) modulator. After modulation, the optical signal is amplified by an erbium-doped fiber amplifier (EDFA) and subsequently coupled with an optical signal generated by external cavity laser-2 (ECL2), operating at 1550.00 nm. As shown in Figure 5a, the frequency spacing between the two optical carriers is configured to approximately 300 GHz.

Figure 2. The experimental setup of the hybrid THz/FSO transmission system with shared transmitter and communication link. AWG: arbitrary waveform generator; EA: electrical amplifier; I/Q MOD: in-phase/quadrature modulator; ECL: external cavity laser; EDFA: erbium-doped fiber amplifier; UTC-PD: uni-traveling-carrier photodiode; HA: horn antenna; OC: optical coupler; LO: local oscillator; LNA: low-noise amplifier; OSC: oscilloscope; ICR: integrated coherent receiver.

Figure 3. The physical diagram of the hybrid THz/FSO transmission structure.

Figure 4. The DSP flowchart of the system. PRBS: pseudorandom binary sequence; QAM: quadrature amplitude modulation; RC: raised cosine; DAC: digital-to-analog converter; GSOP: Gram–Schmidt orthogonalization process; CMA: constant modulus algorithm; FOE: frequency offset estimation; CPE: carrier-phase estimation; DD-LMS: decision-directed least mean squares; BER: bit error rate.

Figure 5. (a) The optical spectrum at the transmitter. (b) The electrical spectrum at the receiver.

The generated optical signal is split into two paths: one for FSO transmission and the other for THz transmission. In the FSO link, the optical signal is amplified using an erbium-doped fiber amplifier (EDFA). A Thorlabs F810FC-1550 fiber collimator form USA is used to collimate the 1550 nm optical signal, which provides a theoretical $1/e^2$ beam diameter of 7.0 mm and a full-angle divergence of 0.016° at the 1550 nm alignment wavelength according to the manufacturer’s specifications. The collimated beam is incident on a 5 mm-thick PMMA plate, where it is reflected and subsequently transmitted into free space for wireless propagation. In the THz link, the optical signal is amplified by an EDFA and fed into a uni-traveling-carrier photodiode (UTC-PD), which generates the corresponding THz electrical signal. This THz signal passes directly through the PMMA plate and is transmitted into free space via a horn antenna (HA). Lens-1 (model TPX 100 from Thorlabs) is used to focus the THz signal for efficient transmission.

After propagating over a 10-m shared wireless link, the THz and FSO signals are received separately by their respective receivers. The FSO receiver requires precise alignment with the FSO transmitter to ensure optimal performance. The optical signal captured by the FSO receiver is initially pre-amplified using an EDFA and then processed by an integrated coherent receiver (ICR). Homodyne detection is performed with the assistance of a local oscillator (LO). Subsequently, the baseband signal is recorded using an 80 GSa/s digital oscilloscope (OSC) for offline DSP. As shown in Figure 4, the DSP workflow for THz signals includes resampling, the Gram–Schmidt orthogonalization procedure (GSOP), the constant modulus algorithm (CMA), frequency offset estimation (FOE), common phase error (CPE) correction, decision-directed least mean squares (DD-LMS) adaptation, and BER analysis. These steps ensure accurate signal recovery and performance evaluation.

The THz receiver can be flexibly positioned within 0.2 m to the left or right of the optical receiver, ensuring adaptability in system configuration. The received electrical signal is initially focused by lens-2 and captured by the HA. Down-conversion is performed using an integrated mixer/amplifier/multiplier chain, comprising a ×24 frequency multiplier and a mixer, with the LO frequency set to 13.12 GHz. This setup generates an intermediate frequency (IF) of 15.68 GHz, calculated as $13.12\times 24-299.2=15.68$ GHz, as shown in Figure 5b. To enhance the sensitivity of the receiver, a low-noise amplifier (LNA) with a gain of 35 dB is employed. The resulting 15.68 GHz IF signal is then captured by a 100 GSa/s digital OSC for further processing. As shown in Figure 4, the DSP for the THz receiver includes digital down-conversion, resampling, GSOP, CMA, FOE, CPE, DD-LMS, and BER counting, ensuring robust signal recovery and performance evaluation.

4. Results and Discussion

The theoretical model described in Section 2.2 provides a unified framework for interpreting the experimentally measured transmission performance of the two heterogeneous links. For the FSO link, Equation (8) indicates that the received SNR increases with the optical transmit power under fixed channel conditions. Consistent with this relationship, Figure 6 shows a monotonic improvement in BER as the optical power launched into the fiber collimator increases from 6 to 11 dBm. Experimentally, when the optical input power to the UTC-PD is 10 dBm, the corresponding RF output power at 300 GHz is approximately −28 dBm. The 30-GBaud QPSK FSO signal satisfies the 7% HD-FEC threshold of $3.8\times {10}^{-3}$. For the THz link, Equations (1)–(3) indicate that the received signal quality is jointly determined by the effective THz transmit power, propagation loss, channel fading, and receiver noise. Experimentally, Figure 7 shows the BER performance of the 30-GBaud QPSK THz signal as the optical power launched into the UTC-PD is varied. The THz link also satisfies the $3.8\times {10}^{-3}$ HD-FEC threshold. These results experimentally demonstrate the power-dependent link behavior described by the received-signal and SNR models, while the statistical channel models in Equations (4)–(6) and (10)–(13) provide a general framework for extending the analysis to channels with pointing errors, atmospheric turbulence, and fading. Therefore, each link of the hybrid THz/FSO system supports 30-GBaud QPSK transmission, corresponding to 60 Gbps per link.

Figure 6. The BER curve of the 30-GBaud QPSK signal in the FSO link.

Figure 7. The BER curve of the 30-GBaud QPSK signal in the THz link.

The experimental measurements indicate that the FSO link requires 6 dB lower signal power than the THz channel to achieve the same BER. Spectral evaluations further demonstrate superior a SNR for FSO compared to THz under controlled laboratory conditions. The performance advantage of FSO stems from its operation in a stabilized experimental environment, where atmospheric turbulence, fog, rain, and mechanical vibrations were systematically eliminated. In contrast, THz transmission exhibited measurable SNR degradation due to humidity-dependent attenuation, even in indoor settings. These results demonstrate the performance advantage of the FSO link under the controlled line-of-sight conditions considered in this experiment.

However, practical deployment reverses this dynamic. While FSO systems face severe reliability challenges in outdoor environments with turbulence and adverse weather, THz links can provide complementary transmission availability under environmental conditions that strongly degrade FSO performance, as suggested by their different propagation characteristics. This complementary behavior underpins the significance of our hybrid THz/FSO architecture, which strategically combines both technologies: FSO provides high-throughput connectivity during clear atmospheric conditions, while the THz sub-system ensures continuity during fog, rain, or turbulence-induced FSO interruptions.

The hybrid system’s environmental adaptability addresses a critical gap in next-generation communication infrastructures, where conventional single-technology solutions struggle to ensure seamless connectivity across dynamic operational landscapes. In a future adaptive implementation, intelligent switching between the FSO and THz links could exploit their complementary propagation characteristics to improve link availability under dynamically varying environmental conditions. This hybrid THz/FSO architecture transcends traditional terrestrial communication boundaries, offering a pivotal solution for integrated space–air–ground data transmission.

Compared with previously reported THz, FSO, and hybrid THz/FSO transmission systems, the proposed system demonstrates a competitive combination of transmission capacity and hardware integration. In this work, 30-GBaud QPSK transmission is experimentally demonstrated over a 10 m wireless link for both the THz and FSO channels. While many existing THz or FSO systems focus primarily on extending the transmission distance or maximizing the achievable data rate, the main advantage of the proposed architecture lies in its transmitter integration. Specifically, the THz and FSO signals are generated using a shared photonic transmitter and combined through a PMMA-based wavelength-selective structure, eliminating the need for two independent transmit apertures. Therefore, although the demonstrated transmission distance is limited to 10 m in the present proof-of-concept experiment, the proposed architecture provides a compact and low-complexity solution for hybrid THz/FSO transmission. Further optimization of the PMMA optical reflection efficiency, transmitter power, and beam-coupling efficiency could enable the proposed architecture to be extended to longer-distance applications.

5. Conclusions

We experimentally demonstrate a hybrid THz/FSO transmission system that employs a photonics-assisted technique to generate both FSO and THz signals, substantially reducing system complexity and weight through a shared transmitter design. This work provides a theoretical analysis and experimental validation of the hybrid architecture. Measurement results confirm that each link of the hybrid THz/FSO system supports 30-GBaud QPSK transmission, satisfying the 7% HD-FEC threshold of 3.8 × 10⁻³. The experimental results validate the feasibility of the proposed shared-transmitter architecture and demonstrate the complementary transmission characteristics of the THz and FSO links. These advancements establish critical technological foundations for future stable, high-speed broadband communication systems.