Nonlocal Interactions in Metasurfaces Harnessed by Neural Networks

Abstract

Optical metasurfaces enable compact, lightweight and planar optical devices. Their performances, however, are still limited by design approximations imposed by their macroscopic dimensions. To address this problem, we propose a neural network-based multi-stage gradient optimization method to efficiently modulate nonlocal interactions between meta-atoms, which is one of the major effects neglected by current design methods. Our strategy allows for the use of these interactions as an additional design dimension to enhance the performance of metasurfaces and can be used to optimize large-scale metasurfaces with multiple parameters. As an example of application, we design a meta-hologram with a zero-order energy suppressed to 26% (theoretically) and 57% (experimentally) of its original value. Our results suggest that neural networks can be used as a powerful design tool for the next generation of high-performance metasurfaces with complex functionalities.

1. Introduction

Metasurfaces offer versatile control of light beams in multiple dimensions, enabling advanced optical functionalities such as metalenses [1,2,3,4], meta-holograms [5,6,7] and other unique applications [8,9]. Even though there has been impressive progress in fundamental research and applications, metasurface designs are still largely based on selecting meta-atoms from a predefined library to modulate the phase at specific positions [10,11]. This method provides a fast approach to design metasurfaces of macroscopic dimensions by assuming that the optical properties of each meta-atom are the same in the metasurfaces as they are in a periodic array (the so-called local periodic approximation). The downside of the standard method is that it overlooks nonlocal effects [11], including interactions between meta-atoms and leaky modes, thus limiting optical performance [12,13]. Furthermore, nonlocal effects can enable unique functionalities, such as wavelength selection [14,15], wavefront selection [16] and image processing [17,18]. Thus, there is a pressing need to develop practical design strategies to control nonlocal effects.

Recently, inverse design methods such as adjoint-based optimization [19,20] have been proposed to address these issues by incorporating nonlocal effects to adjust the complex amplitude of the meta-atoms, thus improving the design of phase-matching metasurfaces. Despite their benefits, these approaches have important limitations. For example, adjoint-based optimization requires full-structure simulations, making it computationally expensive and unsuitable for large-scale devices [21,22,23]. The supercell method [24,25] simplifies the design of specific devices, but it is cumbersome if complex functionalities are to be accommodated. Deep learning methods have emerged as an alternative, enabling rapid predictions of wavefront outputs by learning meta-atom responses under nonlocal interactions [26,27,28,29]. Moreover, the direct generation capability of generative neural networks for structural design also provides a new perspective [30]. However, due to a strong dependence on the dataset, this method is not easy to extend to other types of metasurfaces, resulting in poor scalability. Additionally, the gradient optimization techniques based on differentiable calculations utilize the prediction capability of neural network models to design large-scale metasurfaces with different functions [31,32,33]. However, challenges such as limited optimization parameters and gradient calculation errors from discrete datasets still constrain device performance and functionality.

Here, we propose a new neural network-based multi-stage gradient optimization strategy to modulate nonlocal interactions between meta-atoms. Currently, there are two main approaches to optimize metasurface structures and mitigate nonlocal effects. One approach, which is adopted here, takes the ideal performance of the metasurfaces as the optimization target [23,31]. The other approach uses the actual phase response of the metasurfaces as the optimization target [28]. Our strategy is based on a decomposition of a multi-parameter optimization into sequential stages, with each stage employing a neural network trained on continuous parameter variations. This approach reduces data requirements and minimizes errors caused by discrete datasets, enabling optimization of more parameters. Additionally, discretization methods are incorporated to ensure the optimized structure meets fabrication requirements, significantly enhancing the practicality of gradient optimization. As an example, we apply this method to optimize meta-holograms for zero-order suppression, achieving a reduction to 26% (theoretically) and 57% (experimentally) of the original zero-order energy. Compared to conventional solutions such as off-axis design strategies [5], polarization detection methods [34,35], or supercell approaches [36], our approach avoids limitations like reduced imaging range, lower efficiency, or increased system complexity. We believe that this method offers a promising route to design large-scale complex metasurfaces and effectively modulate nonlocal effects.

2. Method

Neural network-based gradient optimization methods for metasurface design typically rely on a single network model to optimize parameters. When using a dataset with continuous parameters for training, the increased complexity of the dataset often limits these parameters to the diameters of the meta-atoms’ nanopillars, which restricts the optimization scope and effectiveness. Additionally, the optimized structural parameters are often not compatible with conventional fabrication techniques. To increase the range of optimizable parameters, for example, by varying the position of the nanopillars within the meta-atoms, datasets with only preset discrete values are often used for training. However, the lack of data between these discrete values leads to calculation errors in the continuous gradient, which negatively impacts the optimization outcome.

Here, we use multiple Prediction Neural Network (PNN) models to independently optimize different parameters, with each model learning the continuous dataset for its respective parameters. This strategy improves gradient accuracy and enables optimization of additional parameters. The optimization process is divided into coarse and fine adjustment stages and combined with discretization methods to stay within the range compatible with fabrication requirements.

As an example of application, we optimize a meta-hologram to suppress its zero-order energy. Compared to traditional holograms, meta-holograms offer smaller device sizes, wider Field of View (FoV) and higher efficiency, making them more suitable for applications in information storage [37,38,39] and optical encryption [40,41]. However, their imaging quality is often degraded by the zero-order component [5,34]. Theoretically, the zero-order component vanishes in an ideal phase hologram, but in practice it arises from two main factors: ideal phase deviation caused by nonlocal interactions between meta-atoms and variations in their transmittance. Although neural networks have been applied to the design of meta-holograms [42], previous models are typically trained on meta-atom libraries and neglect the influence of nonlocal interactions. In contrast, our method effectively reduces the impact of nonlocal effects, enabling improved hologram performance.

Our optimization strategy is divided into two stages. In the first stage, the diameters of the nanopillars are optimized with their positions fixed at the center of the meta-atoms. In the second stage, the positions of the nanopillars are optimized within a predefined range, but their diameters are fixed according to the previous optimization. The process for each optimization stage follows the flowchart shown in Figure 1a. Here, the conventional design based on the meta-atoms library (neglecting nonlocal interactions) is used to quickly generate an initial structure. And then, our method optimizes this initial structure by manipulating its actual complex amplitude response under nonlocal interactions. To predict the response of meta-atoms under nonlocal interactions, we use a sliding window to decompose the metasurface structure into macrocells [27,28]. The complex amplitude response of the central meta-atom is obtained by simulating a macrocell containing both the central and neighboring meta-atoms (as shown in Figure 1b, crystalline silicon (c-Si) nanopillars on a sapphire substrate are used as meta-atoms). To balance accuracy and efficiency, the macrocell size is set to 9 × 9 meta-atoms (for more details on meta-atom and macrocell design, see Appendix A). This approach allows any metasurface to be decomposed into a series of macrocells, simplifying prediction and optimization via neural networks.

A major design challenge is that each optimization iteration requires simulating millions of macrocells to obtain the complex amplitude of a macroscopic metasurface. Thus, traditional numerical simulation methods, such as the Finite Difference Time Domain (FDTD) method, become prohibitively time-consuming. Here, we show that PNN is an efficient and practical tool to overcome this challenge. In the first optimization stage, the Prediction Neural Network for Diameter (PNN-D) is used to obtain the complex amplitude of metasurfaces with nanopillars of arbitrary diameters at the center of the meta-atoms. In the second optimization stage, the Prediction Neural Network for Position (PNN-P) is used to obtain the complex amplitude of metasurfaces with nanopillars at arbitrary positions and preset discrete diameters. As shown in Figure 2, for these two networks, 24,000 macrocells were simulated for training and 3000 for testing, by using x-polarized sources. Periodic boundaries were applied, and due to translational symmetry, 81 macrocells were simulated simultaneously in each run, expanding the dataset to 1.94 million instances. Both networks consist of five convolutional layers and four fully connected layers (more details in Appendix B). Implemented in PyTorch (version 1.7.1), the networks used Mean Absolute Error (MAE) as the loss function and the Adaptive Moment Estimation (Adam) optimizer. PNN-D was trained for 30,000 epochs, achieving errors of 0.0887 on the training set and 0.1023 on the test set. PNN-P was trained for 60,000 epochs, with errors of 0.0782 and 0.0791 on the training and test sets, respectively.

The Figure of Merit (FoM) is then calculated using the predicted complex amplitude distribution and the target complex amplitude distribution of the metasurface structure under x- and y-polarized incident light. The corresponding definitions of the target complex amplitude distribution can be found in Appendix C. The FoM for x- or y-polarized incident light quantifies the difference between the predicted and target complex amplitude distributions, as defined by:

$${\mathrm{FoM}}_{\mathrm{x}}={\displaystyle \frac{1}{\mathrm{n}}}{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{n}}\left|{\mathrm{A}}_{\mathrm{x},\mathrm{pred},\mathrm{i}}\exp (\mathrm{j}{\mathrm{\phi }}_{\mathrm{x},\mathrm{pred},\mathrm{i}})-{\mathrm{A}}_{\mathrm{x},\mathrm{tar},\mathrm{i}}\exp (\mathrm{j}{\mathrm{\phi }}_{\mathrm{x},\mathrm{tar},\mathrm{i}})\right|}$$

(1)

$${\mathrm{FoM}}_{\mathrm{y}}={\displaystyle \frac{1}{\mathrm{n}}}{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{n}}\left|{\mathrm{A}}_{\mathrm{y},\mathrm{pred},\mathrm{i}}\exp (\mathrm{j}{\mathrm{\phi }}_{\mathrm{y},\mathrm{pred},\mathrm{i}})-{\mathrm{A}}_{\mathrm{y},\mathrm{tar},\mathrm{i}}\exp (\mathrm{j}{\mathrm{\phi }}_{\mathrm{y},\mathrm{tar},\mathrm{i}})\right|}$$

(2)

Here, n represents the number of the meta-atoms, |·| denotes the magnitude of the difference between complex amplitudes, A_pred and φ_pred correspond to the amplitude and phase of the predicted distribution, and A_tar and φ_tar correspond to the target distribution. The overall FoM is calculated from the FoM_x and FoM_y. The smaller the FoM is, the closer the complex amplitude distribution of the predicted structure is to the target complex amplitude distribution.

$$\mathrm{FoM}={\displaystyle \frac{\sqrt{{\mathrm{FoM}}_{\mathrm{x}}^2{+\mathrm{FoM}}_{\mathrm{y}}^2}}{2}}$$

(3)

Finally, the gradient optimization is either continued to optimize the structural parameters using the backpropagation algorithm or terminated, depending on whether the number of iterations reaches the preset value or the FoM meets the termination condition. In the backpropagation algorithm, the differentiable pipeline from the optimization parameters to the FoM allows us to calculate the gradient of each nanopillar parameter with respect to the FoM. We then use the Adam optimizer, a gradient descent method, to obtain the optimized structural parameters. Discretization functions (see Appendix D for more details) are then applied to adjust the parameters for subsequent optimization iterations. The parameters of the final optimized structure can thereby meet the discretization requirements.

3. Result

We design and optimize the meta-hologram with a 1000 × 1000 pixel resolution and a structural size of 350 μm × 350 μm, at the operating wavelength of 800 nm. The Fourier holographic phase map is calculated from the target image using the Iterative Fourier transform algorithm [43], and the initial meta-hologram is obtained through the conventional meta-atoms library. It features eight fixed values for the diameter of 500 nm high nanopillars, ranging from 126 nm to 224 nm, arranged in a meta-atom with a 350 nm period (see meta-atom parameters in Appendix A). To simplify experimental fabrication, the final optimized structure retains nanopillars with eight fixed discretized diameters and a 9 nm step size in their positions. The diameters are identical to those used in the phase matching method, while ensuring a minimum interval of more than 50 nm between nanopillars. Considering these factors, the optimization of the diameter D ranges from 100 nm to 240 nm, and the position parameters Δx and Δy range from −36 nm to +36 nm.

Due to the difficulty of simulating the large-scale metasurfaces, we use the prediction results from the PNN-P to obtain the complex amplitude of the metasurfaces. Then, we apply the Fast Fourier Transform (FFT) to simulate the hologram intensity distribution in the k-space. In Appendix E, we verify on a smaller meta-hologram with a size of 87.5 μm × 87.5 μm that the prediction results from the PNN-P closely match the simulation results from the FDTD method from Ansys, Inc. Furthermore, as shown in Appendix E, due to nonlocal interactions between meta-atoms and variations in their transmittance, the complex amplitude distribution of the ideal phase hologram without zero-order energy differs significantly from the simulated results of the original metasurfaces. In our method, we directly optimize the structure by fine-tuning the meta-atoms with the explicit goal of suppressing the zero-order energy. The optimization of the structural parameters may significantly affect the internal modes of the nanopillars and the nonlocal interactions between them (more details in Appendix I). As a result, the complex amplitude distribution of the optimized structure more closely resembles that of the original metasurface rather than the ideal hologram, while still achieving effective zero-order suppression.

The simulation of the initial structure reveals a prominent zero-order component in k-space, significantly brighter than the image area, as shown in Figure 3a. Then, the neural network-based multi-stage gradient optimization is performed (Figure 3b). The two optimization stages are terminated upon completing the 600 iterations for optimizing the diameter and position, and the FoM reaches 6.6×10⁻⁴. Comparison between the intensity distribution in the k-space of the optimized structure and that of the original reveals that the zero-order component is significantly reduced, as shown in Figure 3c. The proportion of zero-order energy decreases to 26% of its original value (Figure 3d), effectively improving the imaging quality by minimizing the zero-order energy.

By using this method, many meta-atoms and complex wavefront distributions can be designed with relatively little computational time, even less than the simulation time required for the entire metasurface in FDTD (more details in Appendix F). It is worth noting that, during the optimization process, if the position step size is changed to a stricter requirement of 1 nm for fabrication, then the obtained optimization results are similar, as shown in Appendix G.

SEM (Zeiss Auriga) micrographs of the structure of size 350 μm × 350 μm and fabricated using electron beam lithography are shown in Figure 4a. To minimize the impact of incident light outside the structure, a square aperture diaphragm of 350 μm is used, aligned and fixed with the metasurface.

The imaging optical setup is used to capture the hologram intensity distribution of the meta-holograms with an 800 nm laser light source. The power of the zero-order light is obtained by measuring the intensity within a circle of diameter 3 × FWHM of the zero-order light in the CMOS imaging plane. The proportion of zero-order energy after optimization is reduced to 57% of its initial value in the experiment (more details in Appendix H), compared to 26% in the simulation. This discrepancy is mainly attributed to fabrication errors, which are not accounted for in the optimization process. Specifically, deviations in the diameters of the nanopillars have a significant influence on the zero-order component, as shown in Appendix J. As shown in Figure 4b, the zero-order over-exposure area in the hologram intensity distribution of the original structure is significantly larger than in the optimized structure, thus leading to a more pronounced impact on the image quality.

4. Discussion and Conclusions

In conclusion, we propose a neural network-based multi-stage gradient optimization method that takes the ideal performance of the metasurface as the optimization target. This approach enables efficient modulation of nonlocal interactions, thereby significantly enhancing the overall performance of the metasurface. This multi-stage optimization strategy, using multiple network models, resolves the trade-off between gradient calculation accuracy and the number of optimized parameters. Additionally, discretization ensures that the optimized parameters meet preset fabrication requirements. Together, these features provide a practical and fast approach for optimizing large-scale metasurfaces.

Through simulations and experiments, we applied this method to suppress the zero-order component of meta-holograms by optimizing the complex amplitude of the meta-atoms considering nonlocal interactions. We showed a theoretical zero-order energy reduction to 26% of its original value, and the intensity dropped to the same level as the image area. Experimentally, the zero-order energy was reduced to 57%. To reduce the discrepancy between the experimental and theoretical results, the method’s robustness can be enhanced by incorporating error-aware training or robustness strategies [19]. This approach may reduce the impact of fabrication errors on the optimized structure, making the optimization method more practical for real-world applications. Unlike other methods [5,34,36], our approach adjusts the diameters and positions of nanopillars within the meta-atoms, without additional optical components or restrictions on phase units. This makes ours a versatile method to control nonlocal interactions, enabling efficient applications of metasurfaces with intricate complex-amplitude designs. To further test the generality of our method, we used it to optimize a beam deflector and improve its deflection efficiency (more details in Appendix I). By extending the network to predict the complex amplitudes for multiple wavelengths and different polarization components, our method can be applied to the regulation of complex polarization distributions and broad-spectrum responses in imaging, sensing, and communication, like polarization converters or achromatic metalenses. Moreover, by integrating it with differentiable computational processes, like diffraction calculations [31], more applications with complex light field distributions can be realized, like multi-focus [44] or multi-topological beam focusing [45].