Global–Local Cooperative Optimization in Photonic Inverse Design Algorithms

Abstract

We developed the Global–Local Integrated Topology inverse design algorithm (denoted as the GLINT algorithm), which employs a trajectory-based optimization strategy with waveguide–substrate material-flipping structural modifications, enabling the direct optimization of discrete waveguide–substrate binary structures. Compared to the conventional Direct Binary Search (DBS), the GLINT algorithm not only significantly enhances computational efficiency through its global search–local refinement framework but also achieves a superior 20 nm × 20 nm optimization resolution while maintaining its optimization speed—substantially advancing the design capability. Utilizing this algorithm, we designed and experimentally demonstrated a 3.5 µm × 3.5 µm dual-port wavelength division multiplexer (WDM), achieving a minimum crosstalk of −11.3 dB and a 2 µm × 2 µm 90-degree bending waveguide exhibiting a 0.31–0.52 dB insertion loss over the 1528–1600 nm wavelength range, both fabricated on silicon-on-insulator (SOI) wafers. Additionally, a 4.5 µm × 4.5 µm three-port WDM structure was also designed and simulated, demonstrating crosstalk as low as −36.5 dB.

1. Introduction

Integrated photonic devices are crucial for advancing modern optical technologies, facilitating breakthroughs in high-speed communication [1], quantum information processing [2], and biosensing [3]. Traditional design strategies often focus on optimizing a limited set of geometric parameters, such as waveguide widths or cavity radii, which are typically confined to predefined regular shapes. Although this approach is straightforward, it significantly restricts design freedom by confining solutions to low-dimensional design spaces that struggle to meet the increasing demand for compact, high-performance devices capable of exploiting complex light–matter interactions. To address these limitations, inverse design methodologies have emerged as a transformative paradigm [4], utilizing computational optimization to explore high-dimensional design spaces and unlock photonic devices with unprecedented functionality, enabling innovations from the three-channel wavelength demultiplexer [5] to meta-dispersion microresonators [6] and non-reciprocal pulse routers [7]. These advancements highlight the paradigm’s capability to surpass traditional design constraints, establishing it as an essential tool for next-generation integrated photonics [8,9,10,11,12,13].

Despite these achievements, current mainstream adjoint-based inverse design approaches utilizing density topology optimization (DTO) [14,15,16] face a critical challenge that substantially constrains the optimization efficiency. The inherent assumption of a spatially continuous refractive index distribution within this framework fundamentally conflicts with practical fabrication requirements. The mandatory binarization process for discretizing continuous refractive indices may completely negate the carefully optimized optical characteristics, thereby necessitating optimization reinitialization, which significantly compromises the design efficiency. While level-set optimization (LST) [17,18] algorithms are commonly combined with DTO [5,18,19] to mitigate this issue through the direct boundary optimization of binary structures, their standalone application suffers from critical limitations. Specifically, LST methods can only modify existing boundaries or internal cavities but cannot spontaneously generate new void structures within solid regions, thus requiring complementary optimization strategies for a practical implementation.

Alternative approaches such as direct binary search (DBS) [20,21,22,23,24,25,26] algorithms and rotatable DBS [27] variants circumvent the continuous refractive index assumption by discretizing the design space into rectangular or circular pixels. However, such pixel-based methods suffer from two critical drawbacks. First, these methods sacrifice substantial design freedom compared to DTO approaches, with typical pixel dimensions (~100 nm × 100 nm) being excessively large, severely constraining their optimization flexibility. Second, prohibitive computational demands stemming from exhaustive pixel-by-pixel searching in DBS algorithms render these methods impractical for large-scale or functionally complex photonic devices.

In this work, we introduce a Global–Local Integrated Topology inverse design algorithm, denoted throughout this work as the GLINT algorithm or the GLINT inverse design algorithm, that addresses these challenges through an iterative optimization framework. Our approach strategically combines a global search for a rapid performance improvement and local refinement for precision tuning. The global search phase rapidly identifies promising design regions where structural modifications yield significant performance gains, while the local refinement phase progressively refines geometries within these regions through meticulous parameter adjustments. When the local refinement stagnates, the algorithm automatically reactivates the global search to locate new optimization zones, cycling between these modes until target specifications are achieved. Crucially, our method not only eliminates the continuous refractive index assumption inherent in DTO algorithms but also, compared to the DBS algorithm, significantly enhances the design freedom and computational efficiency.

To validate our approach, we experimentally demonstrate its efficacy through the realization of two compact devices: a two-port wavelength division multiplexer (WDM) and a 90-degree bending waveguide. Additionally, we numerically design a three-port WDM, showcasing the algorithm’s capability to handle increasingly complex design challenges. Remarkably, the 90-degree bending waveguide occupies a footprint of just 2 µm × 2 µm, while the two-port and three-port WDMs measure 12.25 µm² and 20.25 µm², respectively. These results highlight the algorithm’s ability to achieve ultra-compact device geometries, significantly reducing spatial requirements for chip layouts and paving the way for large-scale, densely integrated photonic circuits.

2. The Principle of the GLINT Inverse Design Algorithm

The underlying principle of the GLINT inverse design algorithm can be illustrated through a WDM design example. In this example, all simulations employed a 3D finite-difference time-domain (3D-FDTD) methodology with a spatial resolution of 20 nm × 20 nm, uniform perfectly matched layer (PML) boundary conditions, and transverse-electric (TE) polarized excitation sources. The optimization domain is defined with dimensions of 3.5 µm × 3.5 µm × 220 nm. The substrate material is silicon dioxide (represented in white), and the waveguide material is silicon (represented in blue). To facilitate global search and local refinement operations, the optimization domain is subdivided into a uniform grid with a cell size of 20 nm × 20 nm. Notably, the grid size (20 nm × 20 nm in this work) defines the minimum feature size of the optimized structures. While this parameter is not fixed and can be scaled according to fabrication capabilities, all designs in this study adopt 20 nm × 20 nm features to rigorously validate the GLINT algorithm’s performance under near-lithographic-resolution constraints. Figure 1a illustrates the initial structure, where the silicon (white) and silica (blue) regions are clearly defined. Two input signals at λ₁ = 1607.5 nm and λ₂ = 1528.8 nm enter from the left port, with the design objective of routing λ₁ to output port 1 and λ₂ to port 2. Notably, GLINT allows the free selection of initial structures—including fully waveguide- or substrate-filled configurations. However, better initial structures could reduce the optimization time. To assess the device’s performance, the fundamental principle governing the Figure of Merit (FOM) formulation dictates that as the performance of the optimized structure asymptotically approaches the target performance, the FOM converges toward 0; conversely, the deviation from the target performance drives the FOM toward 1. For this work, we define the FOM as

$$FOM=1-{\displaystyle \frac{P_1({\lambda }_1)}{P_{in}({\lambda }_1)}}{\displaystyle \frac{P_2({\lambda }_2)}{P_{in}({\lambda }_2)}}$$

(1)

where P_in(λ_i) is the input power at wavelength λ_i, and P_i(λ_i) is the output power at λ_i from port i (i = 1,2). The optimization target aims to minimize this FOM (ideally to zero) through iterative structural modifications. After each iteration, the updated FOM value is denoted as FOM₀.

As shown in Figure 1b, the algorithm initiates the global search by randomly selecting the circular region B within the design area (in this case, the radius of region B is 80 nm). The temporary material inversion (silicon ↔ silica) within B generates a trial structure, followed by the trial FOM calculation (FOM_t). After restoring the original configuration, this process repeats with new random regions until it identifies a candidate satisfying

$${\displaystyle \frac{FO{M}_0-FO{M}_t}{FO{M}_0}}>G_{th}$$

(2)

where G_th is the global searching threshold. The processes illustrated in Figure 1b are termed the global search phase, and the qualified region B is referred to as the global optimization area.

As demonstrated in Figure 2a, once a circular region B meets the specified condition (2), it is condensed to a smaller concentric circular region S (in this case, the radius of region S is 40 nm), referred to as the local optimization area. Within this region, the materials are permanently inverted, and FOM₀ is updated to reflect the current FOM value. Thereafter, the algorithm randomly selects a small adjacent region X next to S (Figure 2b), inverts the materials in X, calculates the trial value FOM_t, and then restores the materials in X. Repeat the above procedures until the following condition is met:

$${\displaystyle \frac{FO{M}_0-FO{M}_t}{FO{M}_0}}>L_{th}$$

(3)

where L_th is the local refinement threshold. When the region X satisfies the aforementioned condition (3), it is integrated into S (i.e., S = S∪X) (Figure 2c), subsequent to a permanent inversion of the materials in X and an update of FOM₀. The above procedures occurring in the vicinity of S are referred to as the local refinement phase, and the local optimization area S has the potential to expand through successive local searches until no adjacent region X meets the specified condition (Figure 2d). Once this occurs, the local refinement terminates, and a new cycle of the global search initiates (Figure 3a). The algorithm alternates between the global search and local refinement (Figure 3a–c) until FOM₀ achieves a value below a predefined performance threshold (FOM_s) or the cumulative number of search steps executed during either the global search or local refinement phases exceeds the preset maximum iteration limit (Figure 3d).

It is critical to emphasize that within the GLINT algorithm, the dimensions of regions B and S are user-configurable parameters that can either remain fixed or dynamically adapt during optimization, depending on specific optimization requirements. Let R_B and R_S denote the radii of regions B and S, respectively. Upon the completion of each local refinement phase and before recommencing the global search procedure, both parameters (R_B and R_S) can be adaptively adjusted based on the optimization progress. While a reduction in the size of region B inherently increases the global search duration, this strategy becomes essential when larger initial B regions fail to identify domains satisfying the global search threshold G_th. Under such conditions, strategically downsizing region B enables an accelerated convergence toward G_th-compliant regions, thereby significantly enhancing the algorithm’s search efficiency in complex optimization scenarios. Similarly, G_th and L_th serve as adjustable parameters in the GLINT framework, and when tuning R_B and R_S in the algorithm, G_th and L_th can be simultaneously adjusted. In the presented example, these thresholds were empirically initialized at G_th = 0.01 and L_th = 0.001. The precise adjustment of G_th, L_th, and the dimensions of regions B and S is critical for identifying higher-quality local optima throughout the optimization process. To prevent the algorithm from stagnating due to a premature convergence to suboptimal local optima, we recommend employing larger initial parameters (1 > G_th ≥ 0.01; G_th ≥ L_th ≥ 0.001; R_B ≥ 120 nm; R_B ≥ R_S ≥ 60 nm) and progressively scaling them down throughout iterations. This adaptive strategy ensures balanced exploration–exploitation trade-offs, coordinating the rates of the search efficiency and the FOM reduction across the optimization trajectory. Notably, the selection of G_th and L_th retains full flexibility. Practitioners may alternatively initialize smaller optimization thresholds (e.g., G_th = 1 × 10⁻⁵, L_th = 1 × 10⁻⁷) along with reduced B and S dimensions (e.g., R_B = 20 nm, R_S = 10 nm) to prioritize the rapid acquisition of functionally viable devices, albeit with an increased risk of premature convergence to suboptimal local optima. This configurability empowers users to tailor the optimization workflow to application-specific priorities.

The GLINT inverse design algorithm introduces two fundamental advancements over the conventional DBS algorithm: First, it employs a dual-phase strategy integrating the global search with the local refinement, achieving a minimum feature size comparable to DTO (20 nm × 20 nm). This high-resolution capability enables the full exploitation of the optimization potential within the design space, directly resolving the limited flexibility issue inherent in the pixel-based DBS algorithm. Second, the global–local optimization architecture significantly enhances the computational efficiency through the rapid identification of critical optimization regions during the global phase, thereby addressing the computational inefficiency inherent in conventional DBS algorithms. Collectively, these innovations expand the applicability of the GLINT inverse design to complex photonic devices while simultaneously improving the computational efficiency and design flexibility. In the subsequent section, the practicability of our algorithm will be demonstrated by several examples.

3. Simulation and Experimental Results

3.1. Dual-Port WDM Structure

The complete simulation flow of the 1528.8/1607.5 nm WDM structure’s crosstalk–iteration relationship (Section 2 case study) is presented in Figure 4a. Notably, this case maintains fixed R_B and R_S dimensions, with G_th and L_th adjusted only when algorithmic stagnation occurs due to undiscoverable regions satisfying conditions (2) and (3)—a deliberate configuration to benchmark the worst-case optimization speed. Utilizing a single i9-10900K CPU, the total optimization required 240 h across 1360 iterations. The slower optimization speed of this WDM structure is primarily attributed to the progressively diminishing probability of discovering region B configurations that satisfy the FOM reduction threshold during later optimization stages, where global search operations demanded excessive computational resources. Although reducing R_B/R_S would accelerate the search velocity in regions B/S and facilitate the escape from suboptimal local optima, these parameters were intentionally kept fixed in this case to validate the worst-case optimization performance.

Within Figure 4a, intermediate structures at iterations 1110 and 1215 exemplify GLINT’s progressive optimization deceleration. At iteration 1110 (62 h), both ports’ crosstalk reached −11.7 dB. By iteration 1215 (110 h), the crosstalk improved to −13.5 dB (upper port) and −11.7 dB (lower port). The markedly slower progression between iterations 1110 and 1215 indicates approaching optimization bottlenecks near local optima. Despite this, we continued reducing G_th/L_th thresholds to evaluate the performance characteristics of this local optimum. Upon reaching 240 h, the iteration rate degraded to approximately 1 step/3 h without the R_B/R_S adjustment, confirming the proximity to a local optimum. Final crosstalk values of −14.4 dB (upper port) and −12.5 dB (lower port) represent marginal improvements (<1 dB for both ports) over the 1215-iteration solution despite 130 additional hours. We emphasize that this configuration represents a near-optimal local solution (not global optimum) obtained within 110 h without parameter precise tuning. The precise co-optimization of R_B, R_S, G_th, and L_th would yield faster searching and superior local optima approaching the global optimum performance.

The schematic of the final optimized structure is shown in Figure 4b. The electric field distributions for input signals at wavelengths of 1607.5 nm and 1528.8 nm are depicted in Figure 4d and Figure 4e, respectively, clearly demonstrating that the energy at different wavelengths is directed to different output ports. The upper port achieves a −14.4 dB crosstalk at 1607.5 nm, while the lower port provides a −12.5 dB crosstalk at 1528.8 nm, as shown in Figure 4f. To enable the systematic performance evaluation of the WDM structure, we define the normalized transmission ratio, denoted as T. More specifically, T_i designates this metric for the i-th output port, expressed as $T_i=P_i/{\sum }_k{P}_k$, where i, k ∈ (up, down) and P_i are the output power at port i.

All nanostructures presented herein were fabricated via industry-standard electron beam lithography (EBL) with subsequent inductively coupled plasma (ICP) dry etching and wet chemical stripping, achieving minimum feature widths of 20 nm (±10 nm tolerance). The designed structure was fabricated on an SOI wafer. In the experimental setup, the output from a tunable laser source is directed through a polarization controller and into a lensed fiber, which subsequently couples the light to the face of the test chip. The output is similarly collected using a lensed fiber and measured by a power meter. To enhance the optical coupling efficiency at the chip’s end face, inverse taper structures were introduced. Figure 4c shows an SEM image of the fabricated structure. It can be seen that several fine features present in Figure 4b are missing in Figure 4c. These deviations result in experimental normalized transmission ratio measurements (indicated by the red and blue dashed lines in Figure 4f) that do not closely match those simulation results, leading to an increased crosstalk of −9.7 dB at the upper port and −11.3 dB at the lower port, respectively. Similarly, the insertion loss (black dashed lines in Figure 4f) has also increased compared to the simulation results. These fine structural features originate from interstices between discrete circular refinement zones during the local optimization phase. When present in excessive quantities, such non-manufacturable features (<40 nm) significantly enlarge the discrepancy between simulated and experimental results. In our future work, these fine structural features will be effectively removed through spatial filtering, a heuristic learning-driven structural simplification, or an integrated LST algorithm to significantly enhance the device manufacturability.

3.2. The 90-Degree Bending Waveguides

The radii of conventional 90-degree bending waveguides typically range from the tens of micrometers, primarily due to the constraints imposed by radiation loss, which significantly hampers the development of ultra-compact optical integration. By employing the GLINT inverse design algorithm, the size of the 90-degree bending waveguide can be reduced by more than an order of magnitude. We validate the feasibility of the GLINT framework through the design, fabrication, and experimental characterization of a subwavelength-scale 90-degree bending waveguide, with a primary focus on demonstrating extreme dimensional miniaturization.

During the design process, a Euler bend with a radius of 1.4 µm is used as the initial structure, as illustrated in Figure 5a. The white regions represent silicon, while the blue regions denote silicon dioxide. The dimensions of the entire optimization region are 2 µm × 2 µm × 220 nm. After 11 h of iterative optimization, the final optimized structure is given in Figure 5b, with a 1.4 µm radius. Assuming an optical signal enters from the left side and exits from the top, the electric field distributions for the optimized structures are shown in Figure 5c.

The optimized structures were fabricated on a silicon-on-insulator (SOI) wafer. Figure 5d shows the scanning electron microscope (SEM) image of the test chip. The complete test structure contains twelve cascaded optimized 90-degree bending waveguides, only eight are visible here due to the limited imaging field of view. To mitigate the impact of fabrication errors, the average insertion loss of the individual waveguide structure is calculated and used as the experimental result.

Figure 5e presents a comparison between the simulation and experimental results of the optimized structure. The solid line represents the simulation results, while the dashed line corresponds to the experimental data. The optimized structure achieves insertion losses as low as 0.22–0.26 dB across the 1528–1600 nm wavelength range in the simulation. Experimental measurements reveal insertion losses of 0.31–0.52 dB for the structure over the identical wavelength range. A comparative analysis of experimental and simulated results reveals an insertion loss increase of 0.09–0.26 dB in measured devices. This discrepancy is attributed to fabrication-induced losses; fabrication errors cause the structural deformation in the optimized configuration, deviating from the local optimum and consequently increasing the insertion loss.

3.3. Three-Port WDM Structure

To further evaluate the capability of our algorithm, we designed a more complex three-port WDM capable of separating input signals at wavelengths of 1450 nm, 1550 nm, and 1650 nm. As shown in Figure 6a, the optimized design measures 4.5 µm × 4.5 µm on a 220 nm thick SOI wafer. The input port is positioned on the left side, while the three output ports are located on the right side, arranged top to bottom.

The optimized structure exhibits insertion losses of 1.36 dB, 1.15 dB, and 1.08 dB at wavelengths of 1450 nm, 1550 nm, and 1650 nm, respectively, as shown in the lower portion of Figure 6b. Assuming the output powers for the upper, middle, and lower output ports are P_up, P_middle, and P_down, respectively, the normalized transmission ratio T_i is defined as $T_i=P_i/{\sum }_k{P}_k$, where i, k ∈ (up, middle, down). T_up, T_middle, and T_down are also plotted in the upper portion of Figure 6b, represented by the red, green, and blue curves, respectively. Based on the electric field distribution simulation results shown in Figure 6c–e, at wavelengths of 1650 nm, 1550 nm, and 1450 nm, the input energy is effectively routed to the corresponding upper, middle, and lower ports, respectively. The crosstalk for each output port at three wavelengths is detailed in

Table 1. Crosstalk simulation results for the three-port WDM at up, middle, and down ports.

Crosstalk (dB)		Input Wavelength (µm)
		1.45	1.55	1.65
Output ports	up	−22.9	−18.1	—
	middle	−36.5	—	−18.0
	down	—	−20.1	−22.3

, with values ranging from −18.0 dB to −36.5 dB.

Throughout the optimization process, we dynamically scaled down R_B, R_S, G_th, and L_th parameters to prevent algorithmic stagnation. The parameter co-regulation strategy enabled the completion of a functionally more complex three-port WDM design within 190 h over 1842 iterations using a single CPU. Notably, despite doubling the simulation domain area compared to the dual-port WDM in Section 3.1, this three-port WDM design achieved both a faster optimization time and lower crosstalk—convincingly demonstrating that precise parameter co-regulation enhances GLINT’s ability to escape suboptimal local optima and the optimization speed.

The compact size and excellent performance of this complex WDM structure demonstrate the robustness of our algorithm in addressing intricate design challenges. In future work, we will expand the simulation bandwidth of this WDM structure to enhance its tolerance to fabrication errors [28]. By simply modifying the optimization objective function, the design can be readily extended to accommodate more output ports. These compact WDMs have the potential to serve as key components in optical communication systems [29], optical neural networks [30], and optical frequency synthesizers [31,32].

4. Conclusions

In summary, we proposed the GLINT inverse design algorithm for integrated photonics, leveraging global search and local refinement strategies. The proposed algorithm addresses two critical limitations of conventional DBS algorithms through its unique global–local refinement framework. The high-precision local refinement phase resolves the constrained design freedom issue, while the efficient global search mechanism significantly enhances the computational efficiency. Compared to traditional DBS algorithms, this framework demonstrates a superior computational efficiency and design flexibility.

To demonstrate the effectiveness of the proposed algorithm, we have designed and experimentally validated a 90-degree bending waveguide structure and multiple WDM structures. The experimentally measured insertion loss of the optimized 90-degree bending waveguide ranges from 0.31 dB to 0.52 dB across the 1528–1600 nm wavelength. Additionally, the fabricated dual-port WDM structure achieves crosstalk values of −9.7 dB and −11.3 dB at the two output ports for the targeted wavelengths. The three-port WDM structure further demonstrates the algorithm’s capability to optimize more complex design tasks.

Currently, we are actively extending our algorithm to further demonstrate its superior capabilities compared to conventional DBS algorithms. We are also developing several acceleration strategies to improve the algorithm’s search speed and efficiency. In addition, we are exploring various heuristic methods that can regenerate the initial structure based on phased optimization results, thus simplifying the final optimized structure and improving its manufacturability. With ongoing advancements in computational power and fabrication precision, we anticipate that the GLINT inverse design algorithm will play an increasingly pivotal role in integrated photonics in the near future.