Incoherent Optical Neural Networks for Passive and Delay-Free Inference in Natural Light

Abstract

Optical neural networks are hardware neural networks implemented based on physical optics, and they have demonstrated advantages of high speed, low energy consumption, and resistance to electromagnetic interference in the field of image processing. However, most previous optical neural networks were designed for coherent light inputs, which required the introduction of an electro-optical conversion module before the optical computing device. This significantly hindered the inherent speed and energy efficiency advantages of optical computing. In this paper, we propose a diffraction algorithm for incoherent light based on mutual intensity propagation, and on this basis, we established a model of an incoherent optical neural network. This model is completely passive and directly performs inference calculations on natural light, with the detector directly outputting the results, achieving target classification in an all-optical environment. The proposed model was tested on the MNIST, Fashion-MNIST, and ISDD datasets, achieving classification accuracies of 82.32%, 72.48%, and 93.05%, respectively, with experimental verification showing an accuracy error of less than 5%. This neural network can achieve passive and delay-free inference in a natural light environment, completing target classification and showing good application prospects in the field of remote sensing.

1. Introduction

Traditional remote sensing detection methods, particularly electronic computing-based image processing algorithms, face numerous challenges, including slow computation speed and high energy consumption when handling large-scale remote sensing data. These issues make it difficult to meet the demands for real-time performance and low power consumption. As remote sensing technology rapidly advances, the number of remote sensing platforms and sensors continues to increase and is constantly being optimized, leading to a rapid growth in the volume of massive multi-source remote sensing data [1,2,3]. Therefore, developing an efficient and low-energy image processing technology has become an important research direction in the field of remote sensing detection. In recent years, optical neural networks (ONNs), as hardware neural networks implemented based on physical laws, have shown great potential in the field of deep learning [4,5,6,7]. Compared with traditional electronic computing, optical neural networks have advantages such as high speed, low energy consumption, and resistance to electromagnetic interference, making them particularly suitable for handling large-scale parallel computing tasks. They have already achieved good development in fields such as computer vision [8,9,10,11,12,13,14,15,16,17,18,19,20], optical communication [21,22,23,24,25], and multimodal signal processing [26,27,28,29,30,31,32,33,34,35,36]. However, most existing optical neural networks are designed for coherent light inputs, which means that in practical applications, an electro-optical conversion module needs to be introduced before the optical computing device. This conversion not only increases the complexity of the system but also significantly hinders the inherent speed and energy efficiency advantages of optical computing, limiting the application of optical neural networks in actual target detection tasks.

To overcome this limitation, researchers have begun to explore the design methods of incoherent optical neural networks (ICONN). Incoherent optical neural networks aim to directly receive natural light input for inference calculations, avoiding the step of electro-optical conversion, thereby preserving the high speed and low energy consumption characteristics of optical computing [37]. M. Kleiner et al. discussed how the performance of diffractive neural networks is affected by the coherence length and coherence time of the light source, and they proposed optimization schemes for network design with incoherent light inputs [38]. M.S.S. Rahman et al. studied the use of spatially incoherent diffractive processors to achieve universal linear intensity transformations, demonstrating the application potential of incoherent optical neural networks in fields such as imaging and signal processing [39]. In the same year, the team also investigated the use of diffractive optical neural networks for signal restoration in the presence of arbitrary opaque occlusions, showing the application of robust optical communication systems in complex environments [40]. B. Rahmani et al. proposed a method for transmitting full-optical images through multimode fibers using incoherent light, employing diffractive optical neural networks to compensate for distortions caused by multimode fibers, indicating the feasibility of using incoherent light in diffractive neural-network-based optical communication and imaging systems [7]. The method proposed by S. Sun’s team, based on partially coherent optical neural networks (PCONN), although theoretically promising, is limited in practical applications due to the complexity of coherence measurements, which restricts its performance [41]. The incoherent all-optical methods mentioned above still have defects such as difficulty in training, large simulation errors, and the need for electrical domain activation.

In response to the aforementioned issues, this paper proposes a diffractive algorithm for incoherent light based on mutual intensity propagation [42,43,44] and establishes a model of an incoherent optical neural network on this basis. The model is completely passive, directly performing inference calculations on natural light, and it does not require any electro-optical conversion modules before outputting the light intensity at the detection surface, thereby achieving target classification in an all-optical environment. The main contributions of this paper include (1) proposing a new diffractive algorithm for incoherent light that can effectively handle natural light inputs; (2) establishing an incoherent optical neural network model based on this algorithm, achieving passive inference, and conducting simulation tests on the MNIST, Fashion-MNIST, and ISDD datasets, achieving accuracies of 82.32%, 72.48%, and 93.05%, respectively, and analyzing various factors that may affect the model’s performance; (3) setting up an optical platform to experimentally verify the proposed model, with an error in simulation accuracy of less than 5%, proving the model’s effectiveness and robustness in a natural light environment.

The method proposed in this paper still has room for further development. First, the model has currently only been tested and verified on image classification datasets, and in the future, it can be further tested and optimized for object detection tasks on large-scale remote sensing datasets. Second, because the model directly receives natural light input, it is difficult for the network to generate nonlinear activation, which limits the network’s performance on complex tasks. Possible solutions include exploring weak-light nonlinear mechanisms for natural light or introducing a small amount of electrical domain computation to provide nonlinearity. Future research will focus on solving these problems to further enhance the performance of incoherent optical neural networks in practical applications.

2. Methods

2.1. Subsection

There are two commonly used analytical algorithms for the diffractive transmission of incoherent light. One is based on Fourier optics, which calculates the output through spectral conversion [37,45,46]. This method is usually suitable for simple imaging systems; otherwise, the propagation modes of different spectral components will vary, causing the system to be space-variant and resulting in errors. The other is the random phase superposition method based on statistics [39,47]. This method interprets the input of incoherent light fields as the superposition of multiple coherent lights with random initial phases. These coherent lights are transmitted independently, and then the field strength is calculated separately at the output surface before performing an incoherent average. To obtain a stable output, this method typically requires a high number of superimposed layers, resulting in high computational power demand. Moreover, both of the above algorithms require the input to be completely incoherent light for calculation. In other words, when there are changes in coherence within the system, they cannot perform segmented optimization. In practical applications, especially in the field of remote sensing, long-distance transmission often causes the coherence of the light field to change in various ways. In this paper, we adopted a transmission theory for incoherent light fields based on mutual intensity propagation. This method does not rely on statistical laws, which reduces the computational power demand to a certain extent. Moreover, since the coherence of the light field is fully preserved, the calculation can start at any point in the system. Therefore, the system can be modeled in segments, corresponding to the multi-layer network in neural network algorithms, allowing for layer-by-layer optimization. The following is the derivation of this transmission model.

In nature, targets can be regarded as incoherent light at the initial plane, that is, the degree of partial coherence is 0. Therefore, the input light field can be established as

$$\mathrm{J}({\mathrm{P}}_1,{\mathrm{P}}_2)=\mathrm{I}({\mathrm{P}}_1)\mathsf{\delta }({\mathrm{P}}_1-{\mathrm{P}}_2)$$

(1)

In Equation (1), J represents the mutual intensity, I represents the light field intensity, P₁ and P₂ represent the initial plane of the light field emitted by the target, and the delta function indicates that the value of J is equal to I only when P₁ and P₂ overlap. When P₁ and P₂ do not overlap, J = 0, indicating complete incoherence.

The general propagation formula for partial coherence can be obtained from the statistical law of Rayleigh Sommerfeld diffraction formula, that is,

$$\begin{array}{c}\mathrm{J}({\mathrm{Q}}_1,{\mathrm{Q}}_2)=〈\mathrm{u}*({\mathrm{Q}}_1)\mathrm{u}({\mathrm{Q}}_2)〉\hfill \\ ={\displaystyle \frac{1}{4{\mathsf{\pi }}^2}}{\displaystyle \iint {\displaystyle \iint \mathrm{J}({\mathrm{P}}_1,{\mathrm{P}}_2)}\exp [\mathrm{j}\mathrm{k}({\mathrm{r}}_2-{\mathrm{r}}_1)]\frac{\mathrm{K}({\mathsf{\theta }}_1)}{{\mathrm{r}}_1}\frac{\mathrm{K}({\mathsf{\theta }}_2)}{{\mathrm{r}}_2}(\frac{1}{{\mathrm{r}}_1}+\mathrm{j}\mathrm{k})(\frac{1}{{\mathrm{r}}_2}-\mathrm{j}\mathrm{k})}\mathrm{d}{\mathrm{S}}_1\mathrm{d}{\mathrm{S}}_2\end{array}$$

(2)

In Equation (2), J(Q₁, Q₂) represents the distribution of mutual intensity at the output plane; r₁ and r₂ represent the distances between P₁, Q₁ and P₂, Q₂, respectively; and K(${\mathsf{\theta }}_1$) and K(${\mathsf{\theta }}_2$) represent the tilt factors in diffraction. The above equation represents the mapping relationship between the mutual intensity of two planes in free space, and there is no approximation, which follows the space-invariant principle. Therefore, it can be established as a convolution formula:

$$\begin{array}{c}\mathrm{J}({\mathrm{Q}}_1,{\mathrm{Q}}_2)={\displaystyle \iint \mathrm{J}({\mathrm{P}}_1,{\mathrm{P}}_2)}\mathrm{h}({\mathrm{P}}_1){\mathrm{h}}^{*}({\mathrm{P}}_2)\mathrm{d}{\mathrm{S}}_1\mathrm{d}{\mathrm{S}}_2\\ =\mathrm{J}({\mathrm{P}}_1,{\mathrm{P}}_2)\ast \mathrm{h}({\mathrm{P}}_1,{\mathrm{P}}_2,{\mathrm{Q}}_1,{\mathrm{Q}}_2)\end{array}$$

(3)

In Equation (3), h is the point spread function of mutual intensity in free space. Further solving the Fourier transform of this point spread function allows us to transfer Equation (2) to the frequency domain calculation, which means that we can greatly accelerate our calculation process using the fast Fourier transform (FFT) algorithm. As mentioned earlier, this method can perform segmented optimization on the network.

After the light field has been transmitted, we can process the light field characterized by partial coherence according to the system requirements to obtain the desired information. In our task, what we need is the intensity of the final detection plane. Assuming the output surface coordinates are Q′(x, y), it can be obtained by setting Q′₁ = Q′₂:

$$\mathrm{I}(\mathrm{Q}{\prime }_1)=\mathrm{J}(\mathrm{Q}{\prime }_1,\mathrm{Q}{\prime }_2){|}_{\mathrm{Q}{\prime }_1=\mathrm{Q}{\prime }_2}$$

(4)

In addition, we also note that in remote sensing scenarios, the targets are usually very far from the system, and this distance usually satisfies the Fresnel approximation. Then, we can derive the Van Cittert–Zernike theorem in partial coherent optics:

$$\mathrm{J}({\mathrm{x}}_1,{\mathrm{y}}_1;{\mathrm{x}}_2,{\mathrm{y}}_2)={\displaystyle \frac{\exp (\mathrm{j}\mathsf{\psi })}{{(\overline{\mathsf{\lambda }}\mathrm{z})}^2}}{\displaystyle \iint \mathrm{I}(\mathsf{\alpha },\mathsf{\beta })\exp \{-\mathrm{j}\frac{2\mathsf{\pi }}{\overline{\mathsf{\lambda }}\mathrm{z}}(\Delta \mathrm{x}\mathsf{\alpha }+\Delta \mathrm{y}\mathsf{\beta })\}\mathrm{d}\mathsf{\alpha }\mathrm{d}\mathsf{\beta }}$$

(5)

In Equation (5), α and β are the input surface coordinates, and x and y are the output surface coordinates. The meaning of this theorem is that when the size of the incoherent target itself and the observation plane are much smaller than the distance between them, the mutual intensity of the observation plane is only related to the relative distance between the two points. Specifically, the mutual intensity is proportional to the Fourier transform of the target intensity. This theorem is also important in our network, as it greatly simplifies the computational complexity of the input light field when mutual intensity is used as the propagation dimension, allowing us to directly establish the neural network model from the first acceptance plane of the system.

2.2. Principle of ICONN

The ICONN model designed in this paper is mainly divided into two modules: the optical input module and the optical inference module, as shown in Figure 1. In the left block diagram, the optical field input module is depicted, where the first lens serves as the system’s optical aperture, primarily used for imaging. According to the resolution requirements, an aperture is set on the imaging plane of this input module to filter out a specified small field of view. The small field of view is then transformed into partially coherent light by the second lens and output to the subsequent inference module. The main functions of this input module are light collection, image plane segmentation, and preserving the spatial coherence distribution of the light field.

The inference module mainly consists of multiple layers of incoherent optical neural networks. The optical neural network is composed of discrete diffractive devices. The spatial light field is modulated by the diffractive devices with linear phase modulation, and after diffracting a certain distance in free space, it is modulated again by the next layer of diffractive devices. After multiple layers of modulation and diffraction, the label distribution preset during the algorithm training phase can be obtained. The label is further measured by the output module. The modulation range of the diffractive devices is from 0 to 2π. During the training phase, the data type of the phase modulation quantity is set to float32, and it is re-quantified in the subsequent output manufacturing stage. We will discuss the impact of this quantization on network performance later in the text.

Here, we consider one modulation plus one free-space diffraction as one layer of the optical neural network. Therefore, a single-layer optical neural network can be modeled as

$${\mathrm{w}}_{\mathrm{i},\mathrm{j}}^{\mathrm{l}}({\mathrm{P}}_{\mathrm{i}},{\mathrm{P}}_{\mathrm{j}})={({\displaystyle \frac{{\mathrm{z}}_{\mathrm{l}}}{\overline{\mathsf{\lambda }}{\mathrm{r}}_{\mathrm{i}}{\mathrm{r}}_{\mathrm{j}}}})}^2\exp [{\displaystyle \frac{\mathrm{j}2\mathsf{\pi }({\mathrm{r}}_{\mathrm{j}}-{\mathrm{r}}_{\mathrm{i}})}{\overline{\mathsf{\lambda }}}}]$$

(6)

$${\mathrm{n}}_{\mathrm{i},\mathrm{j}}^{\mathrm{l}}({\mathrm{P}}_{\mathrm{i}},{\mathrm{P}}_{\mathrm{j}})={\mathrm{w}}_{\mathrm{i},\mathrm{j}}^{\mathrm{l}}({\mathrm{P}}_{\mathrm{i}},{\mathrm{P}}_{\mathrm{j}})\cdot {\mathrm{t}}_{\mathrm{i}}^{\mathrm{l}}(:,:,{\mathrm{x}}_{\mathrm{i}},{\mathrm{x}}_{\mathrm{i}})\cdot {\mathrm{t}}_{\mathrm{j}}^{{\mathrm{l}}^{*}}({\mathrm{x}}_{\mathrm{j}},{\mathrm{x}}_{\mathrm{j}},:,:)\cdot {\displaystyle \sum _{\mathrm{m},\mathrm{n}}{\mathrm{n}}_{\mathrm{m},\mathrm{n}}^{\mathrm{l}-1}({\mathrm{P}}_{\mathrm{m}},{\mathrm{P}}_{\mathrm{n}})}$$

(7)

In Equation (6), ${\mathrm{w}}_{\mathrm{i},\mathrm{j}}^{\mathrm{l}}({\mathrm{P}}_{\mathrm{i}},{\mathrm{P}}_{\mathrm{j}})$ represents the mutual intensity propagation mode of the four-dimensional sub-wave source $({\mathrm{P}}_{\mathrm{i}},{\mathrm{P}}_{\mathrm{j}})$, where l denotes the layer number in the network, and i and j represent a pair of points on the P-plane. ${\mathrm{z}}_{\mathrm{l}}$ is a constant representing the center distance between the l-th layer and the next layer. $\overline{\mathsf{\lambda }}$ represents the center wavelength of the optical field under the quasi-monochromatic approximation, and ${\mathrm{r}}_{\mathrm{i}}$ and ${\mathrm{r}}_{\mathrm{j}}$ represent the distances from the sub-wave source to the output point. In Equation (7), ${\mathrm{n}}_{\mathrm{i},\mathrm{j}}^{\mathrm{l}}({\mathrm{P}}_{\mathrm{i}},{\mathrm{P}}_{\mathrm{j}})$ represents the output function of the (i, j)-th neuron formed by $({\mathrm{P}}_{\mathrm{i}},{\mathrm{P}}_{\mathrm{j}})$ in the l-th layer of the network, where ${\mathrm{t}}_{\mathrm{i}}^{\mathrm{l}}$ denotes the mutual intensity modulation coefficient of the diffractive element.

There is a filter in front of the diffractive device, whose main function is to improve the temporal coherence of the light field and provide quasi-monochromatic light input for the modulation device. The final output result of the system is received by the detector, and the classification result is determined by the label area set during the training phase. The loss of the network is represented as

$$\mathrm{L}\mathrm{o}\mathrm{s}\mathrm{s}=\mathrm{M}\mathrm{S}\mathrm{E}({\mathrm{I}}_{\mathrm{g}\mathrm{t}}\cdot scale,\mathrm{I})$$

(8)

In Equation (8), we adopted the mean squared error (MSE) method to calculate our loss. I_gt represents the label distribution, and different classification tasks correspond to different label settings. I_real represents the network output result, and scale represents a scaling factor for the output. Since the output of the optical neural network usually does not perfectly match the label setting, appropriately scaling the label can help the network converge faster.

Optical neural networks typically have a small field of view, and our system also has this characteristic because we placed an aperture on the focal plane of the first lens to perform image plane filtering. However, we can achieve a larger field of view by arraying the system or through line scanning and area scanning methods. That is, our system can perform more complex tasks in a natural light environment, such as object detection, without the need for prior electro-optical conversion, followed by image cutting and using lasers and spatial light modulators to provide input for the subsequent inference model. Our method is expected to achieve real-time processing of massive remote sensing data.

3. Simulation and Experiment Results

3.1. Simulation Strategy

The training and simulations were carried out in a CentOS 7.9.2009 environment, using an Nvidia@4090 graphics card and an AMD EPYC 7282 CPU. The simulation tests included performance evaluations of ICONNs with different numbers of layers, while the experimental tests focused on a single-layer ICONN. We will see in the following text that the simulation and experimental results for the single-layer case are in good agreement.

We conducted simulation tests on the ICONN model proposed earlier, with the detector ground truth settings shown in Figure 2. The ground truth is circularly distributed around the center of the effective detection area. The average light intensity within each square detection region corresponds to the score of a category, and the category with the highest average light intensity is the output category.

During the simulation and experimental testing phases, we used the general image classification datasets MNIST and Fashion MNIST, as well as an infrared ship detection dataset, ISDD, to evaluate the performance of the ICONN model. The first two image classification datasets are ten-class, corresponding to the ground truth in Figure 2a. For the ISDD dataset, we performed preprocessing to conduct binary classification, i.e., ship present and ship absent, corresponding to the ground truth in Figure 2b. The specific processing method is as follows: First, we read the labels (boxes) from the annotation files in the dataset and extracted fixed-width images centered on the label boxes as positive samples. Here, we set the width to 40 pixels, which has been verified to cover over 95% of the target bounding boxes. Then, we randomly generated images from the large image that do not overlap with the labeled bounding boxes and whose pixel grayscale values are not all less than 10 as negative samples. The ratio of positive to negative samples was approximately 1:1. The extracted small images were used as the new classification dataset for subsequent simulation and experimental testing. All datasets were divided into training and testing sets in a 4:1 ratio, with a batch size of 20. The loss function was the mean squared error (MSE) between the network output and the ground truth. The network operated entirely in the optical domain without the addition of activation functions, and the learning rate was set to 0.001.

3.2. Simulation Result and Analysis

First, we conducted tests on the general image classification datasets MNIST and Fashion MNIST. The training was iterated for 30 epochs, and the performance of networks with 1 to 3 layers was tested. The simulation test accuracies were 79.75%, 82.11%, and 82.32% for MNIST, and 69.76%, 71.67%, and 72.48% for Fashion MNIST. The training results and inference demonstration are shown in Figure 3 and Figure 4. From the test results, we can observe that the performance of the ICONN network increased with the number of layers. However, since the ICONN network is a purely linear light intensity mapping, this increase is not significant compared to fully connected networks in the electrical domain or CONN/PCONN with one layer of nonlinear activation. Nevertheless, our simulation demonstration clearly shows that after increasing the number of layers, the label distribution on the detection surface was closer to the ideal ground truth. In other words, most of the energy was concentrated within the label boxes, and the proportion of light intensity for the correct label was higher. This can avoid misjudgments caused by insufficient detector sensitivity, noise, pixel alignment errors, etc., and is still helpful in practical detection tasks.

We also conducted tests on an open-source infrared remote sensing ship detection dataset. The network was trained for 30 epochs, and the performance of networks with 1 to 3 layers was tested. In line with the application characteristics of optical neural networks, we processed the remote sensing images based on the annotations in the dataset, with the processing method described in Section 3.1. The training parameters for the network were consistent with those in the previous section. The simulation test accuracies for networks with 1 to 3 layers were 92.15%, 92.89%, and 93.05%, respectively. The training results and inference demonstration are shown in Figure 5. Similar conclusions can be drawn from the test results: the performance of ICONN did not significantly improve with an increase in the number of layers, but the label distribution became more ideal and easier to measure. Moreover, the excellent performance of our model on the ISDD dataset demonstrated its great potential in the field of remote sensing detection.

In addition to the number of layers, we conducted simulation tests on other parameters that may affect the performance of ICONN, including modulation plane pixel density, layer spacing, and phase modulation bit depth. The modulation plane pixel density refers to the number of pixels contained in the modulation plane when the area of the modulation layer remains unchanged. This value can be adjusted by zero-padding the input image. The layer spacing refers to the diffraction distance of light in free space after passing through one layer of modulation. The phase modulation bit depth refers to the modulation precision of the modulation plane. In each of the above tests, the method of controlling variables was used, and the size of the input image and the size of the modulation layer remained unchanged. The results are shown in Figure 6.

The results show that for the three image classification datasets we tested, the original image resolution and the modulation device resolution had little impact on the network’s performance. However, the layer spacing, or more precisely, the Fresnel number F, which is related to the diffraction effect, had a significant impact on the network’s performance. The Fresnel number is defined as follows:

$$\mathrm{F}={\displaystyle \frac{{\mathrm{d}}^2}{\mathsf{\lambda }\mathrm{L}}}$$

(9)

In Equation (9), d represents the width of the diffraction plane, i.e., the modulation device, and L represents the diffraction distance, i.e., the layer spacing. When the area of our modulation device is fixed, if the layer spacing is set too small, the diffraction effect between layers is weak, and the light field propagation is more akin to geometric optics. The energy allocation effect brought by phase modulation is very weak, making it difficult for the model to converge. As the layer spacing increases and the diffraction between layers approaches Fresnel diffraction, the phase modulation can have a more pronounced energy allocation effect, making the output results more likely to converge to the preset label region. For this reason, in diffractive neural networks, increasing the pixel density of the modulation plane, or the number of neurons, has a limited impact on ICONN. This is because ICONN is driven by physical computation of diffraction effects, rather than the mathematically driven computation of electrical neural networks based on parameter scale. The physical propagation laws of light will somewhat limit the input–output mapping relationship. Our simulation results indicate that the network performance is optimal and stable when the Fresnel number is in the range of approximately 25 to 250. Generally, a shorter layer spacing can lead to a more compact and stable system. Therefore, it is sufficient to take the maximum value of the Fresnel number. This conclusion applies to optical neural networks based on diffraction effects.

The impact of phase modulation bit depth on network performance is also limited. Our network uses torch.complex64 type data during training, corresponding to a phase modulation precision of 24 bit. However, when we reduced the modulation precision to 3 bit during the testing phase, the network performance only dropped by about 1 percentage point. When reduced to 2 bit, the network performance dropped by 1–3 percentage points. Three bit means the network can only provide eight modulation phases: −3π/4, −π/2, −π/4, 0, π/4, π/2, 3π/4, π. Similarly, 2 bit can only provide four modulation phases: −π/2, 0, π/2, π. This demonstrates the remarkable robustness of ICONN to the modulation precision of modulation devices.

3.3. Experiment Result and Analysis

We constructed an ICONN experimental platform to verify our model, with the experimental optical layout depicted in Figure 7. The right side represents the optical input module, where we utilized a white light LED (MNWHL4, THORLABS), a polarized beam splitter, and an amplitude SLM (HDSLM80RA Plus, UPOlabs) loaded with the target intensity distribution to simulate incoherent light input from a natural environment. The left side comprises the optical inference module, which consists of a filter (532 nm ± 20 nm), a 4f system, a beam splitter, a polarizer, a phase SLM (PLUTO-2.1, HOLOEYE), and a CCD. The polarizer is employed to meet the polarization sensitivity requirements of the inference device, while the filter is used to obtain quasi-monochromatic light to satisfy the modeling needs. The 4f system is designed to cut the image plane and map the mutual intensity spectrum plane of the target onto the phase SLM. The phase SLM serves as the modulation layer in the neural network, modulating the input incoherent light field by loading the phase distribution trained in the electrical domain.

The input image had a pixel density of 80 × 80, with a pixel size of 32 μm. The modulation plane pixel density was set to 40 × 40. The distance from the target (amplitude SLM) to the front focal plane of the 4f system was set to 0.308 m, which satisfied the Fresnel approximation. According to the Van Cittert–Zernike theorem, the sampling interval of the mutual intensity spectrum plane is 64 μm. Since the mutual intensity spectrum is a function of the difference in distance, the length is halved when mapped to a two-dimensional space. Therefore, the sampling width is 40 × 40, resulting in an inference device size of 256 μm × 256 μm. The magnification of the 4f system was set to 1. The spacing between the phase SLM and the detector was set to 0.1 m, and the phase modulation bit depth was consistent with the training results, set to 32 bit. A comparison of the experimental and simulation results is shown in Figure 8a,b. The single-layer network achieved the best accuracy of 87.57% on the ISDD dataset, demonstrating good agreement between our experimental and simulation results.

The output of the optical neural network’s detection surface also involves a pixel alignment issue. Especially when the number of network layers is small, the convergence effect of the output surface is not good, and a large amount of energy does not fall within the label area. At this time, the accuracy of the label area selection may also have an undeniable impact on the network performance. Here, we conducted tests based on the images obtained from the actual detector, performing a sliding test of the label area on the actual image to observe the impact of measurement area offset on network performance. The results are shown in Figure 8c. It can be seen from the figure that when the label area was selected at its best, the test accuracy was 87.57%. When the offset was about 70 μm, the network performance was still greater than 85%. This offset is already comparable to the modulation pixel size (64 μm), showing that ICONN has good robustness to measurement area offset.

4. Discussion

We have proposed an incoherent optical neural network based on mutual intensity propagation theory, which is more readily convergent. The network operates entirely in the optical domain and can directly perform computational inference on natural light, thus featuring ultra-high real-time capability and truly zero energy consumption. Except for the detector at the final output plane, all other components can operate passively (the SLM used in the experiment can be replaced by a pre-fabricated DOE component). Therefore, in the future, more energy can be devoted to reconfigurability, or a portion of electrical domain computation can be added to provide nonlinearity and enhance performance. Even in that case, our system would still possess significant advantages in terms of speed and energy efficiency.

Our network also achieved excellent classification performance in remote sensing data tests, which is expected to address the data explosion problem currently faced by the remote sensing field. For example, it can expand the field of view through array integration, serve as a feature extraction module to compress data, or directly perform target detection tasks.

We also conducted a series of simulations and experimental tests to investigate factors that may affect the performance of ICONN. The results show that within a certain range, the pixel density of the modulator has a negligible impact on network performance. In contrast, the Fresnel number, which is controlled by the layer spacing, has a significant impact on the network. Specifically, the network performance is optimal and stable when the Fresnel number is in the range of approximately 25 to 250. Additionally, the modulation depth of the modulator has a limited impact on network performance, and the performance stabilizes when the modulation bit depth is greater than 3 bit. Pixel alignment at the detection plane also affects network performance. When the deviation is less than one pixel width, the simulation and experimental error is approximately 2–3 percentage points.

In summary, the ICONN network proposed in this paper is capable of directly performing inference using natural light. It features all-optical passive operation, ease of convergence, zero energy consumption, and zero latency. Moreover, it has demonstrated excellent robustness and consistency in both simulations and experimental tests. This paves the way for low-cost and high-efficiency optical computing systems. The network holds great potential for applications in portable devices, edge computing, infrared remote sensing, and onboard data processing for satellites.