Robust Diffractive Optical Neuromorphic System Created via Sharpness-Aware and Immune Training

Abstract

Diffractive deep neural networks (D²NNs) have garnered significant attention for their ultra-low energy consumption and parallel optical computing capabilities. However, their practical deployment is hindered by the “model–reality” gap caused by fabrication inaccuracy, device fluctuation, assembly misalignment, environmental perturbation, etc. Here, we propose a combined framework that integrates sharpness-aware minimization (SAM) and aberration-immune learning (AIL), enabling joint immunity against both stochastic noise and systematic deviations from theoretical model training. Specifically, we show that under multiple perturbations such as salt-and-pepper noise, Gaussian noise, and wavefront aberration, the SAM–AIL framework achieves significant classification accuracy improvements on MNIST and Fashion-MNIST compared to conventional offline training approaches. D²NN trained with the SAM–AIL scheme exhibited significant accuracy enhancement under moderate salt-and-pepper noise, Gaussian noise, X-axis, and Y-axis tilting perturbations, respectively. Our work provides an efficient solution for offline training and deploying high-robustness D²NNs on realistic physical systems that are resilient to a variety of imperfections, significantly enhancing model transferability and reliability for optical computing tasks.

1. Introduction

Optically implemented artificial neural networks are attracting growing interest due to their ultra-fast parallel processing capabilities and ultra-low energy consumption. Unlike their electronic counterparts, light propagation and interference inherently enable large-scale matrix operations and convolution computations, offering a novel hardware platform for deep learning inference [1,2,3,4]. Beyond spatial optics-based computing, other physical computing paradigms (microfluidic and micropneumatic computing) have emerged for specialized needs. For microfluidics, Clouse et al. developed a dragonfly-wing-inspired reservoir system (with 91% pattern classification accuracy via RGB dyes) [5], while Ahrar et al. realized electronics-free microfluidic control using pneumatic state machines and microvalves [6]. For micropneumatics, Picella et al. designed coding blocks for soft robot programming, and Hoang et al. created a < USD 1 fault detector—both without electronics [7,8].

While these microfluidic and micropneumatic approaches excel in scenarios like bio-compatible computing, harsh-environment deployment (no electronics), or low-cost fabrication, spatial optics-based computing remains the most promising for high-speed, large-scale parallel inference—particularly in deep learning tasks implemented via architectures like diffractive deep neural networks (D²NNs). D²NNs leverage multi-layer phase or amplitude modulation to sequentially transform free-space propagating optical fields, achieving automated inference tasks such as recognition and classification of complex targets with near-zero latency and minimal power consumption [9,10,11,12,13,14,15]. Figure 1 illustrates two representative hardware routes for realizing D²NN. Panel (a) shows a free-space diffractive deep neural network (S-D²NN) based on multi-plane light conversion (MPLC), where a phase-only SLM and a mirror (M) implement multiple reprogrammable diffractive planes; the assembly is mounted on a multi-axis manual translation stage (MS) for alignment, and the output field is captured by a CCD. The advent of MPLC has given rise to pixel-level reprogrammable devices and very high effective neuron densities [16,17,18]. Panel (b) depicts an integrated diffractive deep neural network (I-D²NN), in which a DOE/photonic chip comprising stacked diffractive layers is placed between two microscope objectives and imaged on a CMOS sensor, consistent with recent on-chip demonstrations [19,20,21]. However, to train such D²NNs, existing approaches predominantly rely on in silico digital twins, where one needs to construct a physical model for accurately describing the light-wave dynamics and the controllable parameters of the model is trained by data-driven backpropagation of a predefined loss function [3]. The optimized phase/amplitude profiles are subsequently loaded onto optical devices such as spatial light modulators (SLMs) or realized by delicately fabricated diffractive optical elements (DOEs) or metasurfaces to satisfy the desired optical forward propagation [22]. However, practical deployment is hindered by a persistent “model–reality” gap between the in silico-modeled expectations and the realistic experimental setup. This gap is primarily a product of fabrication error, device fluctuation, optical system assembly misalignment, wavefront distortion, environmental perturbations, etc. Such imperfections may lead to significant performance (e.g., inference accuracy) degradation simply because the digital twin model cannot be perfectly matched with—and represents only an approximation (which may be good enough) to—the real system [23,24].

To bridge this gap, researchers have essentially adopted several kinds of strategies: (1) incorporating noise or device mismatch/imperfection into the theoretical and simulation models during offline in silico training in order to enhance error resilience [25,26,27]; (2) implementing online or hybrid-model training by integrating real optical measurements into the forward and backward propagation loop for the learning process, thereby reducing dependency on precise hardware modeling [28,29]; and (3) substantially simplifying the architecture of the D²NN by reducing its depth and modulation discretization level. For example, Spall et al. demonstrated that a hardware-in-the-loop (HIL) training scheme can significantly improve robustness against steady-state noise by aligning error characteristics with the physical system [30]. Nonetheless, these HIL-involving methods often require iterative hardware access for sequential updating and/or multiple feed-forward measurements for gradient or loss evaluation. This may result in high computational costs and limited model portability across apparently similar optical devices and systems [21,31,32]. Non-gradient training schemes are possible but also heavily rely on iterative interrogation of the real system. Therefore, both the design space dimension and the speed of the feed-forward process impose strong limitations on their application [2].

To address these critical challenges in transitioning D²NNs via simulation to real system deployment, here, we propose a co-optimization framework that synergistically integrates sharpness-aware minimization (SAM) and aberration-immune learning (AIL) to achieve joint immunity against multi-type perturbations and imperfections, thereby achieving what we call “dual robustness.” Xu et al. were the first to establish a connection between the loss landscape and physical neural network’s robustness; they also verified that it is universally applicable to different physical computing systems including both deterministic errors and non-deterministic errors [33,34]. Motivated by these findings, we also utilize AIL to dynamically inject Legendre aberrations (e.g., tilt and translation) into the input plane during the training stage, emulating fabrication inaccuracy and optical assembly errors to enhance the artificial neural network’s robustness with respect to optical misalignment [35]. The numerical results demonstrate that under certain perturbations, including salt-and-pepper noise ($p_{\mathrm{sp}}$: 0.20), Gaussian noise (variance $\sigma \sim 1.5$), and/or X-axis and Y-axis tilting perturbations (∼$1.0°$ rotation), the SAM–AIL training scheme achieves significant classification accuracy improvements on the MNIST dataset for handwritten digit items [36] relative to the common backpropagation methods. Substantial accuracy improvement was also observed for Fashion-MNIST [37] for D²NNs with one hidden layer. Similar results were obtained for three hidden layers (see Supplementary Document S2 for details). Our work provides practical, efficient, and computationally economical guidance for training and deploying high-robustness D²NNs in real-world applications.

2. The Proposed Training Method

2.1. Modeling Optical Forward Propagation in a D²NN

Let us first consider an image classification task that is implemented via a phase-only modulated optical diffractive network (see inset in Figure 2a). Following the scalar Huygens–Fresnel principle, the continuous propagation from a source point on layer l to an observation point on layer $l+1$ is expressed as a surface integral. For pixelated modulators (SLM/DOE), this integral over the aperture is approximated by a Riemann sum, yielding a discrete “neuron-to-neuron” weighted summation. This discrete formulation directly leads to Equations (1)–(5). The light wave propagation process within D²NN is mathematically formulated as [22,26]

$$t_i^l=a_i^{l}exp\left(\mathrm{j}{\varphi }_i^l\right),$$

(1)

$$n_{i,v}^l=w_{i,v}^l·t_i^l·m_i^l,$$

(2)

$$m_i^l=\sum _k{n}_{k,i}^{l-1},$$

(3)

$$w_{i,v}^l={\displaystyle \frac{z_v-z_i}{r^2}}\left({\displaystyle \frac{1}{2\pi r}}+{\displaystyle \frac{1}{\mathrm{j}\lambda }}\right)exp\left({\displaystyle \frac{\mathrm{j}2\pi r}{\lambda }}\right),$$

(4)

$$r=\sqrt{{(x_v-x_i)}^2+{(y_v-y_i)}^2+{(z_v-z_i)}^2},$$

(5)

where $t_i^l$ is the transmission coefficient of the i-th neuron in layer l, composed of an amplitude term $a_i^l$ and a phase term ${\varphi }_i^l$. In this work, we adopt phase-only modulation by default (i.e., $a_i^l=1$, and ${\varphi }_i^l$ is trainable), while the factorization in Equation (1) also covers amplitude–phase devices if needed. Here, i refers to a neuron of the l-th layer, and v refers to a neuron of the next layer, connected to neuron i by optical diffraction. Additionally, $m_i^l$ represents the input wave to the i-th neuron in layer l, formed by the summation of outputs from all neurons in the previous layer. Here, $n_{i,v}^l$ denotes the output contribution from the i-th neuron in layer l to the v-th neuron in layer $l+1$; $w_{i,v}^l$ is the diffraction weight characterizing wave propagation between these neurons; $\lambda$ is the wavelength; and $j=\sqrt{-1}$.

We note that most usual implementations of D²NNs, as modeled in Equations (1)–(5), are grounded in the Huygens-Fresnel principle and predominantly rely on computer-aided backpropagation to optimize the trainable parameters (i.e., amplitude and phase profiles across layers), enabling ultra-low-energy classification or imaging via multi-layer diffractive phase plates. However, as illustrated in Figure 2a, discrepancies between theoretical models and physical systems may introduce substantial training errors [3].

2.2. Enhancing Device-Level Robustness Through Sharpness-Aware Minimization

The optical modulation at each layer, determined through computational simulations or in situ optical training, is physically realized using either active (e.g., spatial light modulator (SLM) [38,39,40]) or passive (e.g., diffractive optical element (DOE) [41,42,43] or metasurface [44,45,46]) devices. These optical devices often exhibit random phase errors that can be statistically modeled as a combination of continuous Gaussian noise and discrete salt-and-pepper noise. These phase deviations stem from multiple fabrication and environmental mechanisms. For SLMs, experimental deployment is often hindered by parameter drift due to material aging, calibration complexity, and operational instabilities. For DOEs and metasurfaces, systematic inaccuracies arise from fabrication limitations (e.g., quantization errors in digital-to-analog conversion and distortions induced by manufacturing tools) and environmental factors (e.g., temperature fluctuations, humidity variations, material expansion/contraction, and exposure-dose inconsistencies in recording materials). These issues are collectively categorized as device-level imperfections [19,20,47]. As illustrated in Figure 3a, such imperfections are modeled as Gaussian or salt-and-pepper noise in this study.

In the Gaussian noise model, the phase error ${\eta }_G$ at each pixel is treated as a Gaussian random variable

$${\eta }_G\sim \mathcal{N}(0,{\sigma }^2),$$

(6)

meaning it has a mean of zero and a variance of ${\sigma }^2$; its probability density function is

$$f_{{\eta }_G}\left(x\right)={\displaystyle \frac{1}{\sigma \sqrt{2\pi }}}exp\left(-{\displaystyle \frac{x^2}{2{\sigma }^2}}\right),$$

(7)

characterizing small fluctuations about the ideal phase. Such a model is justified because fabrication tolerances—for example, slight errors in pixel height $\Delta h$ or refractive index $\Delta n$—can be assumed to be Gaussian-distributed, and these induce Gaussian-distributed phase deviations via the linearized phase error formula:

$$\Delta {\psi }_{\mathrm{device}}={\displaystyle \frac{2\pi }{\lambda }}(n_r\Delta h+h\Delta n_r).$$

(8)

Here, $\lambda$ is the optical wavelength, $n_r$ is the nominal refractive index of the material of the device is composed of, and h is its design thickness; $\Delta h$ and $\Delta n$ are the stochastic fabrication errors in thickness and refractive index, respectively; and $\Delta {\psi }_{\mathrm{device}}$ denotes the device-induced phase error.

In contrast, the salt-and-pepper noise model accounts for occasional large phase glitches caused by localized defects or malfunctions (e.g., dust on a DOE or a dead/stuck SLM pixel). In this model, each pixel has a probability $p_{\mathrm{sp}}$ of being corrupted to an extreme phase value—either a fixed maximum ${\varphi }_{max}$ or minimum ${\varphi }_{min}$ phase (for instance, saturating to $2\pi$ or 0)—typically with equal likelihood. Formally, one can write

$$P\left({\eta }_{\mathrm{SP}}={\varphi }_{\min }\right)=P\left({\eta }_{\mathrm{SP}}={\varphi }_{\max }\right)={\displaystyle \frac{p_{\mathrm{sp}}}{2}},$$

(9)

$$P\left({\eta }_{\mathrm{SP}}=0\right)=1-p_{\mathrm{sp}},$$

(10)

meaning that a fraction $p_{\mathrm{sp}}/2$ of pixels are randomly set to ${\varphi }_{min}$, and another are set to $p_{\mathrm{sp}}/2$ to ${\varphi }_{max}$ (while the rest remain unperturbed) [19,20,47]. These two noise models thus capture different error mechanisms in phase devices: the Gaussian term describes the aggregate effect of many small independent fabrication errors (yielding a bell-shaped distribution of phase noise), whereas the salt-and-pepper term captures rare, discrete phase jumps due to isolated defects, which manifest as pixels stuck at high or low phase values. We note that the camera-related optoelectronic detection noise is weaker, and it is not discussed in this work [48].

To improve the inference resistance of D²NNs with pure phase modulation, we employed the SAM algorithm for optical parameter training [33,49,50,51]. Traditional phase optimization focuses solely on minimizing the output error $L_s(\Theta )$, where $\Theta =\left\{{\varphi }_i^l\right\}$ denotes the adjustable phase parameters. However, this method tends to converge to sharp local minima (as schematically shown in Figure 2c), which are extremely sensitive to the phase perturbations, leading to a dramatic decline in inference accuracy. The SAM, by jointly minimizing the error value and the “sharpness” of the loss landscape, thus guides the optimization towards “flat” minima (as shown in Figure 2d), ensuring diffraction stability under perturbations. The core idea of SAM originates from robust optimization theory. It not only optimizes the design space parameter configuration $\Theta$ but also ensures that the loss remains low within its neighborhood regime. This is especially important for training optical computing systems, as their parameters are susceptible to perturbations embodied by the phase drift $\delta {\varphi }_i^l$. SAM searches for parameters $\Theta$ that satisfy the following condition: within a neighborhood with a radius of p (defined by the p-norm), the worst-case loss is minimized. This naturally leads to a minimax problem [33]:

$$\underset{\mathbf{\Theta }}{min}\left[\underset{{\parallel \mathit{ϵ}\parallel }_p\le \rho }{max}{L}_S(\mathbf{\Theta }+\mathit{ϵ})\right]+\gamma {\parallel \mathbf{\Theta }\parallel }_2^2.$$

(11)

Here, the inner maximization $\left({max}_{{\parallel \mathit{ϵ}\parallel }_p\le \rho }\right)$ identifies the worst perturbation $\mathit{ϵ}$ that maximizes the loss, thereby quantifying the local sharpness of the error surface. The outer minimization $\left(min\mathbf{\Theta }\right)$ adjusts $\Theta$ to suppress sensitivity.

To minimize $L_s(\Theta )$, an efficient approximation of its gradient ${\nabla }_{\mathbf{\Theta }}{L}_S(\mathbf{\Theta })$ needs to be derived, enabling the application of gradient descent to the SAM optimization objective. The gradient is approximately

$${\nabla }_{\mathbf{\Theta }}{L}_S^{\mathrm{SAM}}(\mathbf{\Theta })\approx {\nabla }_{\mathbf{\Theta }}{L}_S\left(\mathbf{\Theta }+{\mathit{ϵ}}^{*}\right),$$

(12)

where ${\mathit{ϵ}}^{*}$ represents the worst-case perturbation associated with the model weights $\Theta$,

$${\mathit{ϵ}}^{*}\approx \rho ·{\displaystyle \frac{sign\left({\nabla }_{\mathbf{\Theta }}{L}_S(\mathbf{\Theta })\right){\left|{\nabla }_{\mathbf{\Theta }}{L}_S(\mathbf{\Theta })\right|}^{q-1}}{{∥{\nabla }_{\mathbf{\Theta }}{L}_S(\mathbf{\Theta })∥}_q^{q/p}}},$$

(13)

where $1/p+1/q=1$, the p-norm constrains the perturbation range, and the q-norm normalizes the gradient magnitude.

Figure 2b schematically illustrates a single step of SAM parameter updating.

2.3. Enhancing System-Level Robustness Through Aberration-Immune Learning

As illustrated in Figure 3b, in addition to intrinsic errors caused by constituent optical devices, misalignments of these optical components (e.g., planar shifts along the X/Y-axis or minor tilts with respect to the different axes) in experimental systems would also introduce unpredictable wavefront distortions, which we term system-level environmental aberrations in this work. Such imperfections can be represented by a combination of low-order Legendre polynomials, characterized by gradient-continuous phase profiles [52]. Conventional methods, such as those proposed by Hoshi et al. [53], enhance robustness by superimposing Legendre distortions on phase modulation layers [35,54]. While partially mitigating fabrication errors, these approaches remain sensitive to stochastic salt-and-pepper or Gaussian noise because random noise exhibits high-frequency oscillations, whereas Legendre distortions feature relatively smooth variations.

To accommodate both device-level and system-level inaccuracy, we propose an AIL strategy. Instead of superimposing distortions on the phase modulation layers, AIL injects Legendre-based phase aberrations into the input plane. As illustrated in Figure 2e, the injected phase is constructed from the first ten two-dimensional Legendre polynomials $L_1$–$L_{10}$. Low-order two-dimensional Legendre polynomials provide an orthonormal basis for modeling systematic misalignment-induced wavefront aberrations over a rectangular aperture [55]. Each Legendre mode corresponds to a specific misalignment pattern and generates a characteristic smooth phase profile across the aperture. For example, as shown in Figure 3b, the mode $L_2(x,y)$ with coefficient $\alpha$ (representing the tilt-induced phase amplitude) models a slight tilt about the X-axis, whereas $L_3(x,y)$ with coefficient $\gamma$ models a tilt about the Y-axis.

A small tilt by an angle of $\Delta \theta$ introduces an additional linear phase ramp, whose peak-to-center phase difference is

$$\Delta {\psi }_{\mathrm{t}}={\displaystyle \frac{N}{2}}d{\displaystyle \frac{2\pi }{\lambda }}\Delta \theta ,$$

(14)

where N is the number of diffractive pixels along one axis, d is the pixel pitch, and $\lambda$ is the operating wavelength. Under this small-angle approximation, the tilting-induced phase coefficient satisfies $\alpha \approx \Delta \psi$ (and likewise $\gamma \approx \Delta \psi$ for $L_3$), establishing a direct link between the physical tilt and its Legendre weight. Higher-order Legendre modes $L_n(x,y)$ with coefficients ${\beta }_n$ (coefficients of the Legendre polynomials) capture more complex or subtle distortions, enabling representation of high-order alignment errors or smoothly varying airflow-induced phase patterns. The overall aberration phase is therefore expressed as

$$\Delta {\psi }_{\mathrm{c}}(x,y)=\sum _n{\beta }_n{L}_n(x,y).$$

(15)

where ${\psi }_{\mathrm{c}}(x,y)$ is the aberration phase (in radians), and $L_n(x,y)$ denotes the orthonormal Legendre basis functions defined over the square aperture. During D²NN training, AIL dynamically incorporates wavefront aberrations generated from the Legendre polynomials and random perturbations into each training batch dataset. By multiplying these aberrations by the complex amplitude of the input optical field, AIL emulates cumulative effects from device misalignments and assembly errors [55].

This approach enables D²NNs to adaptively compensate for diverse systematic errors while preserving SAM’s noise immunity. The simulated results demonstrate that AIL significantly reduces performance degradation caused by cumulative wavefront distortions without compromising robustness against random noise inherited from the optical devices (see Figure 4, and there are more details in Section 3).

2.4. Joint SAM–AIL Co-Optimization for Phase-Only D²NNs

SAM promotes flat minima of the loss landscape by optimizing within a small parameter neighborhood, thereby reducing the sensitivity of phase-only D²NNs to device-level stochastic phase errors (e.g., Gaussian and salt-and-pepper noise). AIL improves tolerance to systematic, low-order, physically grounded wavefront aberrations by injecting Legendre-mode distortions in the input plane during training, encouraging the network to learn misalignment compensation. Building on these complementary strengths, rather than applying SAM and AIL sequentially (e.g., “SAM then AIL” or vice versa), we couple them within a single training loop. For each mini-batch, AIL first injects physically grounded aberrations in the input plane, which reshapes the loss landscape observable by the optical phase parameters; then, SAM explicitly searches for flat minima under these aberration-induced distributions by perturbing only the phase variables. This realizes a joint objective—minimizing the worst-case loss in a SAM neighborhood while marginalizing over AIL aberrations—which empirically behaves differently from a naive combination. The complete training flow is summarized in Algorithm 1, which details the batch-wise integration of AIL’s aberration injection and SAM’s perturbation-based phase optimization for phase-only D²NNs.

Algorithm 1 Joint SAM–AIL training for D2NNs

Require: Training set $\mathcal{S}={\left\{(x_i,y_i)\right\}}_{i=1}^n$; network parameters $\mathbf{\Theta }=\left\{{\varphi }_i^l\right\}$ (amplitudes fixed $a_i^l=1$); SAM radius $\rho$; step size $\eta$; Legendre basis $\left\{L_n\right\}$ and coefficient $\left\{{\beta }_n\right\}$; loss $L(·)$.

Ensure: D2NN trained with the joint SAM–AIL strategy

1: Initialize parameters ${\mathbf{\Theta }}_0$; $t\leftarrow 0$

2: while not converged do

3:       Sample a mini-batch $\mathcal{B}={\left\{(x_j,y_j)\right\}}_{j=1}^b$

4:       # AIL injection (input plane, per batch)

5:       Sample $\left\{{\beta }_n\right\}$; build $\Delta {\psi }_{\mathrm{c}}(x,y)={\sum }_n{\beta }_n{L}_n(x,y)$

6:       Aberrate inputs: ${\tilde{x}}_j\leftarrow x_j·exp\{\mathrm{j}\Delta {\psi }_{\mathrm{c}}(x,y)\}$ for all $j\in \mathcal{B}$

7:       # Forward/backward at current parameters

8:       Compute ${\nabla }_{\mathbf{\Theta }}{L}_{\mathcal{B}}(\mathbf{\Theta })$ using $\left\{{\tilde{x}}_j\right\}$

9:       # SAM: worst-case perturbation (optical specialization)

10:       Compute ${\mathit{e}}^{*}$ via Equation (13); ${\mathit{e}}^{*}$ has the same shape as $\mathbf{\Theta }$

11:       Apply perturbation: ${\mathbf{\Theta }}^{+}\leftarrow \mathbf{\Theta }+{\mathit{e}}^{*}$

12:       # Gradient of SAM objective at ${\mathbf{\Theta }}^{+}$ under the same AIL batch

13:       $g\leftarrow {\nabla }_{\mathbf{\Theta }}{L}_{\mathcal{B}}(\mathbf{\Theta }){|}_{{\mathbf{\Theta }}^{+}}$

14:       # Parameter update

15:       Update $\mathbf{\Theta }\leftarrow \mathbf{\Theta }-\eta g$

16:       $t\leftarrow t+1$

17: end while

18: return $\mathbf{\Theta }$

3. Results and Discussion

Based on the scheme outlined in the preceding section, we systematically validated the synergistic advantages of integrating SAM and AIL under scenarios with both device-level noise and system-level distortion. All training procedures were conducted on a computational platform equipped with an Intel Xeon Gold 5115 CPU (2.40 GHz), 32 GB of RAM, and an NVIDIA V100 GPU, implemented using PyTorch (v1.13.0). The network was optimized using 60,000 training images from the MNIST handwritten digit dataset over 20 training epochs. We further evaluated a single-layer D²NN on the Fashion-MNIST dataset. The optical diffractive neural network classifier contains a hidden-layer spacing of $8\mathrm{cm}$, a light-source wavelength of $532\mathrm{nm}$, and a neuron resolution of $256\times 256$ [56]. During training, the cross-entropy loss function was employed to optimize network parameters. For SAM, the perturbation radius hyperparameter was set to $\rho =1$. In the AIL strategy, each training iteration dynamically generated a combination of the first 10 Legendre polynomials, with coefficients ${\beta }_n$ initially sampled from a uniform distribution (range: $-5$ to 5) and subsequently linearly normalized so that the resulting phase distribution lay within $[-\pi ,\pi ]$. This normalization ensured that the simulated aberration magnitudes were physically consistent with experimentally reported ranges of coefficients in optical calibration studies [57,58].

Network performance was evaluated via numerical simulations on the MNIST test set (10,000 images). The numerical simulations included salt-and-pepper noise ($p_{\mathrm{sp}}$: 0.05–0.2, step: 0.05), Gaussian noise ($\sigma$: 0.5–2.0, $\mu =0$, step: 0.5), and input plane tilt (X/Y-axis offsets: 0.25°–1.0°, step: 0.25°). For details on the parameter selection rationale, please refer to the cited literature [26,40,59,60]. The noise parameters were configured based on experimental noise data measured from the actual devices utilized in typical experiments [26,40,41]. Results were averaged over 10 runs, verifying the trained phase masks on 10,000 test images, with error bars reflecting post-convergence stability (±0.5–4.8%). Error bars represent the variability observed across 10 independent test runs, corresponding to the range between the minimum and maximum accuracy obtained across epochs.

Figure 4a,b show that when solely using the Adam optimizer [56], classification accuracy drops sharply from 90.56% ($p_{\mathrm{sp}}$: 0.05) to 75.78% under salt-and-pepper noise ($p_{\mathrm{sp}}$: 0.20). For Gaussian noise ($\sigma =1.5$), accuracy plummets from 91.77% ($\sigma =0.5$) to 30.57%, highlighting the extreme noise sensitivity of traditional training methods like Adam. In contrast, with the SAM strategy, the classification accuracy under salt-and-pepper noise ($p_{\mathrm{sp}}$: 0.20) increases to 88.56%, representing a 12.78% gain over Adam. For Gaussian noise ($\sigma =1.5$), the classification accuracy reaches 60.55% (around 29.98% improvement), showing SAM’s effectiveness against stochastic noise.

We note that under Adam optimization, input-plane tilt degrades classification accuracy severely, as illustrated in Figure 4c,d. For example, X-axis tilt ($1.0°$) reduces the total accuracy from 90.74% ($0.25°$) to 43.16% ($\Delta =47.58\%$), while Y-axis tilt causes a drop from 89.81% to 33.13% ($\Delta =56.68\%$). With AIL alone (e.g., Adam with AIL), the accuracy under a $1.0°$ X-axis tilt improves by 46.85% (90.01% vs. 43.16%), and that under a $1.0°$ Y-axis tilt improves by 47.72% (80.85% vs. 33.13%), demonstrating robust compensation for systematic errors.

Noticeably, the combined SAM–AIL strategy retains the individual advantages of both methods without much of a compromise. More specifically, SAM seeks final results with a flat loss landscape to suppress phase fluctuation sensitivity, while AIL injects wavefront aberrations during training to emulate assembly misalignments. The confusion matrix analysis in Figure 5 and Figure 6a–d on the MNIST (10,000 images) shows that SAM–AIL reduces misclassification rates across all digit number classes (0–9) under extreme perturbations. The Hessian matrix eigenvalue analysis further quantifies the robustness. The maximum eigenvalue (${\lambda }_{\max }$) decreases from 1250.47 (Adam) to 35.78 (SAM), confirming the role of flat minima in perturbation immunity. Furthermore, we also compared the training time of the standard Adam optimizer and the proposed SAM with AIL method on both MNIST and Fashion-MNIST datasets in Supplementary Document S1.

The results indicate that SAM–AIL decouples noise resilience and misalignment compensation, enabling mutual reinforcement. Accordingly, we believe that this framework provides a theoretical and experimental foundation for deploying photonic neural networks in dynamic environments, particularly for high-precision optical computing tasks that are quite sensitive to device imperfection and environmental perturbations.

Figure 7 compares the performance of the conventional Adam algorithm and the SAM–AIL strategy for the handwritten digit “1” under both perfect conditions and tilt perturbations. Figure 7a–c show the input digit, the predefined detector area layout, and the optimized phase mask modified by the respective schemes used for testing the example, respectively. As shown in Figure 7d–f, the Adam-trained D²NN fails when the digit is tilted by $1^{\circ }$ about both the X- and Y-axes and misclassifies the sample as “4” and “5”, whereas when tested under additive Gaussian phase noise ($\sigma =1$), it predicts “4” and “6”. By contrast, the SAM–AIL-trained network correctly recognizes the digit in every case.

To systematically validate the synergistic advantages of the SAM and the AIL approaches in scenarios with various types of device noise and input distortion, we extended the numerical experiments from MNIST to the Fashion-MNIST dataset, which represents a more challenging yet computationally comparable classification task. Figure 8a shows that with SAM, a classification accurcy reached 76.55 % under salt-and-pepper noise ($p_{\mathrm{sp}}=0.15$), representing a 13.55 % improvement over Adam. For Gaussian noise ($\sigma =1.0$), SAM achieved classification accuracy of 66.05 % (Figure 8b), surpassing Adam by 22.78 %, showing its strong robustness against stochastic noise. With standalone AIL (Figure 8c), accuracy under a $0.5°$ tilt of the input plane about the x-axis improved by 40.20 % (80.45 % vs. 40.25 %), while a $0.5°$ tilt about the y-axis yielded a 31.59 % increase in accuracy (81.02 % vs. 49.43 %), demonstrating its exceptional ability to compensate for systematic misalignments (see Figure 8d).

Given the relatively higher complexity of Fashion-MNIST compared to MNIST, we conducted confusion matrix analysis on the 10,000-image test set to compare pure Adam and the SAM–AIL. The results are shown in Figure 6e–h and Figure 9. It is evident that the Adam approach exhibits a maximum Hessian eigenvalue ${\lambda }_{max}=978.24$, while SAM reduced this to ${\lambda }_{max}=69.36$. These results align with the MNIST case, suggesting that the SAM–AIL synergy maintains noise robustness and misalignment compensation in classification tasks with much higher complexity.

4. Conclusions

In this work, we demonstrate a robust and transferable diffractive optical neuromorphic system powered by a combined sharpness-aware method and aberration-immune learning strategy. The core contribution lies in leveraging SAM to mitigate device-specific variances, thereby allowing an offline-trained D²NN to retain high accuracy across different hardware instances. In addition, by promoting flatter loss landscapes and incorporating aberration-immune training, our approach yields D²NN models that remain resilient to optical noise, alignment errors, and other post-deployment perturbations without the need for delicate system calibration. This enhanced robustness against system noise and errors underpins reliable real-world deployment of diffractive neural networks.

Beyond algorithmic improvements, the proposed SAM–AIL framework opens pathways to a wide range of practical applications, including optical computing for edge-AI devices, intelligent visual inspection in industrial settings, and medical-image-based decision-support systems, where robustness and power efficiency are crucial. The demonstrated transferability across device variations makes it suitable for hardware reuse and scalable production of optical neural modules.

Nevertheless, the present study has certain limitations. Our experiments were conducted primarily on the MNIST and Fashion-MNIST datasets, which, while standard, are relatively simple compared to those used in complex optical imaging tasks. Future work will therefore extend the joint SAM–AIL strategy to more challenging datasets (e.g., CIFAR and ImageNet) and to physical experimental prototypes. Furthermore, although the proposed SAM–AIL framework has been numerically validated under a wide range of device- and system-level perturbations, these evaluations were conducted within simulation environments. Future work will therefore focus on implementing and experimentally validating the algorithm on physical diffractive prototypes to confirm its real-world robustness and exploring engineering aspects such as fabrication scalability and integration feasibility in order to develop deployable optical-AI systems.