Three-Dimensional Scanning Virtual Aperture Imaging with Metasurface

Abstract

Metasurface-based imaging is attractive due to its low hardware costs and system complexity. However, most of the current metasurface-based imaging systems require stochastic wavefront modulation, complex computational post-processing, and are restricted to 2D imaging. To overcome these limitations, we propose a scanning virtual aperture imaging system. The system first uses a focused beam to achieve near-field focal plane scanning, meanwhile forming a virtual aperture. Secondly, an adapted range migration algorithm (RMA) with a pre-processing step is applied to the virtual aperture to achieve a 3D high-resolution reconstruction. The pre-processing step fully exploits the feature of near-field beamforming that only a time delay is added on the received signal, which introduces ignorable additional calculation time. We build a compact prototype system working at a frequency from 38 to 40 GHz. Both the simulations and the experiments demonstrate that the proposed system can achieve high-quality imaging without complex implementations. Our method can be widely used for single-transceiver coherent systems to significantly improve the imaging depth of field (DOF).

1. Introduction

Metasurface-based imaging is emerging as a promising technology in security screening, nondestructive detection, and the remote monitoring of people [1,2,3,4,5]. Compared with conventional radio-frequency imaging systems, metasurface-based imaging possesses the advantages of lower hardware costs and reduced system complexity. The mainstream metasurface-based imaging modality is called coded aperture imaging [6]. Low-correlated illumination patterns are generated to encode the scene information into a few measurements, enabling image reconstruction through computation. Coded aperture imaging usually requires fewer measurements than conventional imaging modality, resulting in a faster imaging speed. However, the modulation pattern design and computational burden still pose challenges.

The simplest method of radiation modulation is to use random illuminations by introducing an extra mask, such as a metallic mask [7] or a spinning disk [8]. Due to its capability to arbitrarily manipulate the wavefront of electromagnetic waves, the metasurface can generate random illuminations more effectively. Frequency-diverse metasurface antennas have been widely used, achieving high imaging resolution and speed [9,10]. Since frequency-diverse metasurface antennas require an ultra-wide working bandwidth, dynamic metasurface antennas [11,12,13], and frequency-polarization sensitive metasurface antennas [14,15] are proposed for more flexible applications. To achieve a high image reconstruction SNR and accuracy, the modulation patterns must be measured carefully to form a precise forward model. A more effective strategy is to use orthogonal pattern sets such as Hadamard basis [16,17,18] or Fourier basis [19]. Despite the elimination of the cumbersome calibration process, these methods still require complex computational reconstruction, such as matched filter or iterative optimization.

Besides the complexity of the modulation pattern design, measurement, and preprocess computation, many metasurface-based imaging methods are only suited for two-dimensional (only cross-range) [20,21,22] but are not straightforward to be extended to three-dimensional. Recently, considerable research has achieved significant progress in 3D reconstruction [23,24,25,26,27]. The common basic idea is to convert the metasurface-based imaging problem into a conventional synthetic aperture radar (SAR) imaging problem. Specifically, a pre-processing step is performed to decompress the raw received signal as if it had been transmitted and received from a set of uniformly distributed antennas. However, there are still some drawbacks inherent in these methods. First, random illumination patterns are adopted, which require careful designing to ensure their orthogonality and completeness. Second, all the random patterns need precise measurements to calibrate a precise forward model. Typically, tens, or even one hundred, measurements are required. This procedure can be very time-consuming since each measurement requires scanning at the whole region of interest (ROI) by an extra receiver. Third, when converting the forward model to apply the SAR algorithm, inverse matrix calculations are performed for each different modulation pattern and frequency, resulting in a huge computation and memory burden. All these drawbacks restrict the applications of the above methods, leading to mere simulations or poor experimental results.

Here, we propose a scanning virtual aperture (SVA) imaging system to overcome the aforementioned limitations. Our system combines the advantages of both a raster-scanning system and a synthetic aperture system, yielding 3D high-resolution imaging results by simple illumination modulation without any complex calibrations or computations. For the illumination, we use a focused beam to scan the scene, leading to a 2D image with a very limited DOF. In other words, the metasurface is used as a focusing lens with a fixed focal length. Noticing that the focal plane composes a new virtual aperture, we multiplex raw acquisitions using digital coherent superposition to provide high resolution at all depths. In order to accelerate the reconstruction speed, we develop an adjusted RMA to reconstruct the 3D image. Benefitting from the unique near-field focusing modulation pattern, there is only one extra phase shift step. Compared with the current 3D imaging method, our system avoids any complex modulation pattern design, calibration, and matrix computation, providing a feasible and effective solution to achieve high-quality reconstruction of the whole 3D scene.

2. Methods

Figure 1 illustrates the principle of SVA imaging. The metasurface consists of several programmable elements, with agile phase modulation. The desired phase modulation can be generated by tuning the active components on the programmable elements. Based on the scalar diffraction model in wave optics, the echo signal neglecting attenuation can be expressed as:

$$S(\phi ,k)={\displaystyle \iiint \rho (x,y,z)\cdot \exp (-j2kr)\cdot \exp (j2\phi )\cdot \exp (j2{\phi }_0)dxdydz},$$

(1)

where $k$ is the wavenumber, $S(\phi ,k)$ is the echo signal when the modulation phase is $\phi$, and ${\phi }_0$ is the initial phase caused by the incident angle of the feed horn. This system uses the metasurface as a focusing lens with a fixed focal length $f$ to scan the plane $(x,y,z0)$. Therefore, the modulation phase on the metasurface should be a converging spherical wave, which can be expressed as

$$\begin{array}{cc}\phi (x,y)& =k_c{r}_{mf}-{\phi }_0\hfill \\ & =k_c\sqrt{{\left(x_m-x\right)}^2+{\left(y_m-y\right)}^2+f^2}-{\phi }_0,\hfill \end{array}$$

(2)

where $k_c$ is the central wavenumber. This modulation phase converts $S(\phi ,k)$ into a focused image in the cross-range $S(x,y,k)$. Next, inverse fast Fourier transform (IFFT) is applied to obtain a range-focused image, which can be represented as:

$$\rho (x,y,z){|}_{z0}={\mathrm{FT}}_k^{-1}\left\{S(x,y,k)\right\},$$

(3)

where ${\mathrm{FT}}_k^{-1}$ denotes IFFT along the $k$ dimension. However, this image is only well-focused in the focal plane and is otherwise defocused due to beam diffraction. A straightforward way to get well focused 3D images is to change the focal length to different distances, whereas very time-consuming. Here, we note that the focal plane can be viewed as a new virtual aperture, allowing for the application of a synthetic aperture technique. For the target point $\left(x_r,y_r,z1\right)$ far away from the focal plane, it can be digitally refocused by coherent superposition of the echo signals, which can be expressed as:

$$\rho (x,y,z){|}_{z1}={\mathrm{FT}}_k^{-1}\left\{{\displaystyle \iint S(x,y,k)\cdot \exp (2j{k}_c\sqrt{{\left(x_r-x\right)}^2+{\left(y_r-y\right)}^2+d^2})dxdy}\right\},$$

(4)

where $d$ is the distance between the focal plane and the refocusing plane. In fact, (4) is a back projection algorithm. It constructs an identified matched filter for each pixel sequentially, resulting in high complexity. To improve the efficiency, we further develop an adjusted RMA. Our method only requires an extra phase shift to adapt the RMA, which can be expressed as:

$$S^{VA}(x,y,k)=S(x,y,k)\cdot \exp (-2\mathrm{j}kf),$$

(5)

where $S^{VA}$ is the signal on the virtual aperture. According to the Fourier relationship between time and frequency, (5) adds a temporal delay to the captured signal. Hence, all the waves seem to be emanating from the focal plane. Ignoring the different time delays at different scanning points due to the relatively small scanning area, this pre-processing step corrects the mismatch between the wavefront and the conventional imaging model. Therefore, we can use RMA to reconstruct the 3D image with spatially invariant resolution, which is only dependent on the size of the metasurface. Due to the page limitations, we do not review RMA in detail here [28,29], but directly provide the reconstruction formula as:

$$\rho (x,y,z){|}_{z>f}={\mathrm{FT}}_{3D}^{-1}\left\{{\mathrm{Interp}}_{k\to k_z}\left[S^{VA}(x,y,k)\right]\right\},$$

(6)

where $k_z$ is the projection of the spatial wave vector on the axial direction, and Interp is the Stolt interpolation. Finally, as shown in Figure 2, the 3D reconstruction result can be composited by the scanning image (3) and the RMA reconstruction (6):

$$\rho (x,y,z)=\left\{\begin{array}{l}{\mathrm{FT}}_k^{-1}\left\{S(x,y,k)\right\}, z=f\\ {\mathrm{FT}}_{3D}^{-1}\left\{{\mathrm{Interp}}_{k\to Q_z}\left[S^{VA}(x,y,k)\right]\right\}, z>f.\end{array}\right.$$

(7)

The system’s lateral resolution is dependent on the real aperture size of the metasurface, which is given by:

$${\mathrm{\delta }}_x={\displaystyle \frac{{\lambda }_c}{4sin\left(\theta /2\right)}},$$

(8)

where ${\lambda }_c$ is the central wavelength, and $\theta$ is the angle subtended by the metasurface aperture. The axial resolution is given by:

$${\mathrm{\delta }}_z= \mathrm{c}/2\mathrm{B},$$

(9)

where c is the light speed, and B is the bandwidth of the signal.

3. Results

3.1. Simulation

First, a simulation is conducted to evaluate the performance of the proposed algorithm. The parameters except the working frequency are slightly different from those of the experimental prototype system for better presentation. The linear frequency sweep from 38 GHz to 40 GHz, with 31 sampling points, is applied in the simulation. The bandwidth of 2 GHz leads to the range resolution of 7.5 cm. The metasurface has 51 × 51 elements, with ${\lambda }_c/2$ spacing. The imaging FOV and scanning area are both 25 × 25 cm². The resolution step size and sampling interval are both set to 5 mm. The focal length is set as 30 cm. The scene consists of three points at (0, −6, 30), (0, 0, 50), and (0, 6, 90) cm. The initial phase and quantization of the metasurface are ignored.

Figure 3 shows the simulation results using near-field beamforming (NBF) and SVA with the same illumination modulation. While NBF only focuses the target in the focal plane, SVA satisfactorily reconstructs all targets at different distances. The DOF of NBF is defined as twice the Rayleigh length [30], which can be calculated as:

$$\Delta L_c=\pi {\delta }_x^2/\lambda ={\displaystyle \frac{\pi \lambda }{16 {\sin }^2(\theta /2)}}=1.62 \mathrm{cm},$$

(10)

which is very limited. As a result, these targets not in the focal plane are all defocused. SVA obviously improves the DOF, leading to high resolution at all distances (Figure 3f). We take the middle target for resolution analysis quantitatively. The full-width at half-maxima (FWHM) is 1.2 cm (Figure 3c) with little deviation from the theoretical value ${\delta }_x=1 \mathrm{cm}$ according to (8).

Next, we evaluate the performance with different SNRs. The parameters are kept the same with channel SNR set as 0 dB, 5 dB, and 10 dB. The target is a letter “T” at the distance of 50 cm. As shown in Figure 4, defocused NBF fails to recover the shape. However, SVA successfully recovers the “T” shape with any SNR. The noise mainly generates artifacts in the background with little impact on the target contour. We further compare the SVA with the in-focus NBF results, where the focal length is set as equivalent to the focal length of 50 cm. The SVA and in-focus NBF results are very similar, as shown in the middle and right columns of Figure 4. Therefore, we demonstrate that the proposed method is very robust to noise with high reconstruction accuracy.

3.2. Experiment

The experiment setup is shown in Figure 5. Unlike most experimental prototype systems with separated transmitter and receiver placed side by side, our system introduces a circulator to form a quasi-monostatic configuration. This design is crucial for creating a reversible electromagnetic wave path, leading to a high signal-to-noise ratio (SNR) for confocal scanning. The vector network analyzer (VNA) connected with the transceiver is used to generate linear stepped-frequency signals from 38 GHz to 40 GHz. The metasurface comprises 32 × 32 elements with a whole aperture size 12 × 12 cm². The main structure of each element is shown in Figure 5c. Each radiating element is connected by a driver circuit consisting of two PIN diodes and switches. The circuit is controlled by the field programmable gate array (FPGA) with 12 V applied voltage. By applying a bias signal, we control the resistance and capacitance in the switch to turn the diodes to ON/OFF. Finally, we can obtain four different phase states by using different combinations of the diodes states. These phases are experimentally tested to be 0°, 144°, 294°, and 87°. Due to the high power loss of the last state, we only use the first three phases, i.e., 1.5 bit quantization. When generating the focusing beam, we directly set the closest value to the theoretical value, according to (2). In general, higher quantization accuracy leads to better focus quality. Given only 1.5 bit quantization, whether the beam can be focused as expected is a concern. To validate the focusing quality, we measured the beam focus at 10 different lateral and axial locations. Figure 5d shows three typical beam spots in the lateral plane. All the beam spots are well focused, thus proving that 1.5 bit quantization is sufficient to implement the desired phase. Since our method can be applied to arbitrary quantization, we do not pay special attention to any further optimization. The metasurface is illuminated by a feed horn antenna at about 30° and 25 cm for appropriate beam coverage. The scanning range is the same as the imaging FOV to be 10 × 10 cm². The sampling interval is set to 2.5 mm in each axis thus 41 × 41 points are scanned in total. It takes about 0.2 s for the data reading and writing of the VNA, and 0.3 s for the phase change of the metasurface, adding up to 0.5 s to collect the data for one point. Ultimately, it takes about 15 min to complete an entire scan. The speed can be significantly improved by substituting the VNA with a high-speed RF link and optimizing the control software. Since both the metasurface and the VNA exhibit high stability, there is no extra time cost for system calibration during our experiments.

We validate the 2D imaging capability at different distances. To reduce environmental noises, we sample the background signal first and subtract it from the target signal. The focal length is fixed at 20 cm. We set the targets at 20 cm, 30 cm, and 40 cm, respectively. As shown in Figure 6, our system displays the highest resolution and SNR at the distance of $z=20 \mathrm{cm}$.The main shape along with some fine structures of the targets are well reconstructed. For example, the hole in the metal letter “zero” is just discernible (Figure 6b). Since the lateral size of the hole is 1.5 cm, we can estimate that the resolution of our system is ${\delta }_x<1.5 \mathrm{cm}$, which is consistent with the measured spot size (Figure 5d). The imaging result of SVA is the same as NBF when $z=f$, both obtained by direct scanning. For cases of greater distance, the imaging results of NBF are obviously defocused, leading to a low resolution and SNR. However, SVA results remain relatively good. We evaluate the quality of the reconstructed image $\mathrm{I}(x,y)$ with the normalized mean squared error (NMSE):

$$\mathrm{NMSE}={\displaystyle \frac{{\displaystyle \sum _{x,y}{\left(\mathrm{I}(x,y)-{\mathrm{I}}_{\mathrm{ref}}(x,y)\right)}^2}}{{\displaystyle \sum _{x,y}{\mathrm{I}}_{\mathrm{ref}}^2(x,y)}}},$$

(11)

where the imaging result at $z=20 \mathrm{cm}$ is used as the reference image ${\mathrm{I}}_{\mathrm{ref}}(x,y)$. The results are listed in

Table 1. Comparison of NMSE.

	NBF	SVA	NBF	SVA
	(z = 30 cm)		(z = 40 cm)
letter “T”	0.455	0.181	0.362	0.187
letter “0”	0.268	0.059	0.396	0.188
Two batteries	0.199	0.096	0.469	0.180

. The NMSE value of SVA is much lower than that of NBF, demonstrating better imaging quality. We also compare the computational time between SVA and other typical 3D reconstruction methods, as shown in

Table 2. Comparison of the computation time (s) with that of previous methods.

	SVA	[21]	[24]
Pre-processing	0.001	0.25	3.97
RMA	0.3	0.01	11.77
Ratio	0.003	25	0.337

. SVA takes 0.001 s for the pre-processing and 0.3 s for RMA on a 64-bit 3 GHz CPU. Due to the different computing platform, we further consider the time ratio between pre-processing and RMA for a fair comparison. This ratio of SVA is only 0.003, showing great superiority over the other methods. In addition, it is emphasized that no extra time is required for measuring and calibrating the modulation patterns before pre-processing, making SVA an effective 3D imaging with metasurface.

Finally, we demonstrate the capability of the proposed system for real 3D imaging, while previous studies mainly report results using a single target at a fixed distance. The experiments are conducted with objects at different distances, as shown in Figure 7. The focal length is maintained at 20 cm. For the case letter “UC”, the letter “U” is placed at 20 cm, while the letter “C” is placed at 30 cm. For the two-battery case, the front battery is placed at 25 cm, while the back battery is placed at 35 cm. Our system successfully provides a 3D reconstruction of the scene, revealing the shape and location of each object.

4. Discussion

The proposed SVA system exhibits many connections and distinctions with the conventional focal plane scanning (FPS) system and the SAR. Both SVA and FPS employ raster-scanning with a focused beam, but SVA has a larger DOF owing to the application of the synthetic aperture algorithm. On the other hand, SAR creates aperture from the platform motion, whereas there is no motion in SVA. SVA creates aperture with the metasurface, which is acting like a scanning lens. In this regard, SVA is similar to interferometric synthetic aperture microscopy (ISAM) [31]. ISAM uses a convex lens and a synthetic aperture algorithm to obtain a high resolution in all planes, which is equivalent to the resolution at the focal plane. However, ISAM still requires the mechanical motion of the target or the galvo-scanner. Instead, SVA fully exploits the metasurface, resulting in a compact system and the potential for real-time imaging.

The ability of SVA to significantly increase the DOF and reduce the required resources is highly useful for coherent imaging system with a single transceiver across the electromagnetic spectrum. For the single channel system using dynamic beamforming and scanning, there is always a tradeoff between the DOF and the resolution. The key point is to create a virtual aperture to bridge the connection between the synthetic aperture algorithm and the scanning mechanisms, which facilitates a large DOF reconstruction. Although we demonstrate this technique in a millimeter-wave band with a metasurface, it is potentially more broadly transformative. For example, in terahertz band, quasi-optical systems [32,33] usually require a fixed distance for the best focus quality. Following the idea of SVA, these systems can obtain an extended DOF since the optical focal plane is naturally a virtual aperture.

5. Conclusions

In conclusion, we propose a metasurface-based SVA imaging system to produce a 3D high-resolution reconstruction without complex implementations, which avoids stochastic modulation, time-consuming measurement and burdensome computation. SVA first uses a focused beam to obtain a scanning image at the focal plane, meanwhile forming a virtual aperture. Next, an adjusted RMA is applied on the virtual aperture to refocus the targets at all distances. Through the composition of the scanning result and reconstruction result, SVA obtains a high-resolution 3D reconstruction for the whole scene. SVA opens up a new horizon for metasurface-based imaging using simple modulation and digital manipulation, paving the way to practical applications of metasurface-based sensing. Furthermore, it has the potential to significantly improve other single-transceiver beam-scanning coherent imaging setups, which have a very limited DOF. As a proof of principle, our prototype system uses only a small-sized metasurface without optimizing the imaging speed. In the future, we will increase the scale and speed of the system to achieve a higher resolution and a larger FOV in real-time for practical applications.