A Non-Iterative Method Combined with Neural Network Embedded in Physical Model to Solve the Imaging of Electromagnetic Inverse Scattering Problem

Abstract

The main purpose of this paper is to solve the electromagnetic inverse scattering problem (ISP). Compared with conventional tomography technology, it considers the interaction between the internal structure of the scene and the electromagnetic wave in a more realistic manner. However, due to the nonlinearity of ISP, the conventional calculation scheme usually has some problems, such as the unsatisfactory imaging effect and high computational cost. To solve these problems and improve the imaging quality, this paper presents a simple method named the diagonal matrix inversion method (DMI) to estimate the distribution of scatterer contrast (DSC) and a Generative Adversarial Network (GAN) which could optimize the DSC obtained by DMI and make it closer to the real distribution of scatterer contrast. In order to make the distribution of scatterer contrast generated by GAN more accurate, the forward model is embedded in the GAN. Moreover, because of the existence of the forward model, not only is the DSC generated by the generator similar to the original distribution of the scatterer contrast in the numerical distribution, but the numerical of each point is also approximate to the original.

1. Introduction

Electromagnetic inverse scattering is an accurate imaging method in the field of non-destructive testing. It determines the distribution characteristics of the physical parameters of scatterers by measuring the distribution of scattering fields. Additionally, it is widely used in science, engineering, and the military and medical fields [1,2,3,4,5,6,7]. Due to the inherent nonlinearity of ISP, regularization iterative optimization methods are usually used to solve ISP, such as the Born iterative method [8], the distorted Born iterative method [9], contrast source inversion (CSI) method [10,11], subspace optimization method (SOM) [12,13,14], and so on. Although it has been proven that these methods can provide satisfactory results for objects of medium size and contrast, their computational cost is too high and not suitable for real-time imaging. Because of the limitation of computation, it is still a great challenge to apply it to some large scenes. Up to now, due to the influence of multiple scattering effects, the nonlinear electromagnetic inverse scattering technology has mainly been used for low-contrast objects.

Over the past few years, with the development of the neural network, the neural network has been one of the most influential methods in the field of computer vision and natural language processing, such as image classification [15,16,17], image segmentation [18], and speech recognition [19,20]. Most recently, in the field of biological imaging, the neural network has also achieved excellent achievements, such as magnetic resonance imaging [21] and computational optical imaging [22]. Additionally it has been found that the method based on neural networks is superior to the traditional image reconstruction technology in improving image quality and reducing computational cost through experiments [21,23,24,25].

The imaging of electromagnetic inverse scattering can also be solved by the neural network. DeepNIS is a neural network composed of multiple complex-valued cascading residual convolutional blocks [23]. The complex-valued residual convolutional is used to approximately characterize the multi-scattering physical mechanism. Additionally, the inputs of DeepNIS come from back-propagation images. DeepNIS greatly reduces the computational time compared with the traditional iterative method. CVP2P constructs a complex-valued network with reference to the Generative Adversarial Networks (GAN) [24]. The generator of CVP2P adopts a multilayer complex-valued convolution neural network that can calculate complex-valued convolution. The back-propagation images are also used to be the inputs of CVP2P. This algorithm is mainly used to reconstruct binary images, and the reconstruction time is significantly reduced. Wei et al. constructed three different imaging schemes based on U-Net [26]. The first scheme is to directly input the scattering field measurement data into the neural network to calculate the final imaging results. The second scheme is to obtain the initial image by using the BP algorithm and then input the initial image into the neural network for imaging. The last one is to obtain the initial contrast by using the dominant current scheme method, and then it is inputted into the neural network for imaging. A fully connected artificial neural network is used to solve ISP [27]. Firstly, the scattered field samples collected at the receiver are processed, and the estimated value of the scatterer contrast is provided by the neural network in the case of strong nonlinearity. In the training process, a random shape generator affected by the statistical distribution of breast biological tissue is used to assist the training. It also obtained an excellent imaging result. Sanghvi proposed the contrast source network to solve the problem of getting trapped in false local minima when recovering objects with high permittivity [28]. The contrast source network refines the iterative solution by learning the noise spatial component of the radiation operator and estimating the spatial component of the signal directly from the data. When the traditional technology fails and the computational time does not significantly increase, the correct results are obtained.

In this paper, we propose a simple but effective method, the diagonal matrix inversion method (DMI), to get an initial estimate of the distribution of scattering contrast. We also refer to the GAN idea to establish a neural network to decrease the error between the initial estimation and the real contrast. The generator of the GAN is similar to a “U-Net”-based architecture [18], and for the discriminator, a “PatchGAN”-like classifier is used for auxiliary discrimination [29]. The generator obtains the bottom features through multiple down-sampling layers and then restores the bottom features to the real contrast distribution through up-sampling. The features obtained from the down-sampling layers are connected with the features restored by the up-sampling layer through skip-connection so that the information lost in the process of down-sampling can be collected and the real contrast can be restored more fully. When training, we make the physical model between the generator and the discriminator provide assistance. The theoretical measured value can be calculated by the forward model with the contrast distribution obtained by the generator. Additionally, the discriminator then judges whether the theoretical measured value is close to the real measured value. This makes the generator eliminate errors more accurately. Obtaining the initial contrast through the DMI and using the generator to make the initial contrast close to the real contrast is a non-iterative method for solving ISP. After our experimental verification, the GAN combined with the DMI method is superior to the traditional imaging scheme in both the imaging effect and speed. The main contributions of this paper are as follows:

• A non-iterative method (DMI) is proposed to estimate the initial contrast, which is more accurate than the previous schemes;
• A neural network based on physical model training is constructed, which can obtain a better imaging effect than the traditional scheme.

2. Problem Statement

In this paper, we only considered the two-dimensional scalar electromagnetic field. As shown in Figure 1, the TM polarized incident plane wave irradiates the target region.

Several TM polarized incident plane wave-generating devices are evenly distributed at the edge of field D. It can transmit and receive electromagnetic waves. One of them transmits electromagnetic waves, which form a scattered field after scattering by the object. Then, all devices receive scattered field data so that the scattered field data can be present by the following equation:

$$E_{z,sca}\left(r\right)=k_0^2{\int }_{\Sigma }G(r,r^{\prime })\chi \left(r^{\prime }\right)E_z\left(r^{\prime }\right)dSr\in \partial D,r^{\prime }\in \Sigma ,$$

(1)

$$E_z\left(r\right)=E_{z,in}\left(r\right)+k_0^2{\int }_{\Sigma }G(r,r^{\prime })\chi \left(r^{\prime }\right)E_z\left(r^{\prime }\right)dSr\in \Sigma ,r^{\prime }\in \Sigma ,$$

(2)

in which r and $r^{\prime }$ represent the field point and the source point. $E_{z,sca}$, $E_{z,in}$, and $E_z$ are the scattered, incident, and total fields, respectively. $k_0$ is the wavenumber in the background medium. $G(r,r^{\prime })=\frac{i}{4}{H}_0^{\left(1\right)}(k_0|r-r^{\prime }\left|\right)$ is the two-dimensional Green’s function for the case of a homogeneous background, and $H_0^{\left(1\right)}(\ast )$ is the first kind of zero-order Hankel function. $\chi \left(r^{\prime }\right)={ϵ}_r\left(r^{\prime }\right)-1-i(\left(\sigma \left(r^{\prime }\right)\right)/\left({ϵ}_0\omega \right))$ denotes a quantitative relationship between the contrast of the object $\chi \left(r^{\prime }\right)$ and relative permittivity ${ϵ}_r\left(r^{\prime }\right)$. $\sigma \left(r^{\prime }\right)$, ${ϵ}_0$, and $\omega$ are the conductivity, vacuum permittivity, and angular frequency, respectively. $dS$ is an area unit on $\Sigma$. In order to facilitate the numerical experiment, we divide $\Sigma$ into a $K\times K$ square grid by the method of moments aiming to obtain the discretized version of Equations (1) and (2). For a single illumination, one of the electromagnetic waves generating devices generates, and all of them receive. The received data is the discrete version of the scattered field. Then, combined with Equations (1) and (2), we obtain the following discretized equation about a single illumination:

$${\mathbf{E}}_{\mathbf{S}}^{\mathbf{j}}={\mathbf{G}}_{\mathbf{m}}\mathbf{\chi }{\mathbf{E}}_{\mathbf{z}}^{\mathbf{j}},$$

(3)

$${\mathbf{E}}_{\mathbf{z}}^{\mathbf{j}}={\mathbf{E}}_{\mathbf{in}}^{\mathbf{j}}+{\mathbf{G}}_{\mathbf{s}}\mathbf{\chi }{\mathbf{E}}_{\mathbf{z}}^{\mathbf{j}},$$

(4)

where ${\mathbf{E}}_{\mathbf{z}}^{\mathbf{j}}\in {\mathbf{C}}^{{\mathbf{K}}^{\mathbf{2}}\times \mathbf{1}}$, ${\mathbf{E}}_{\mathbf{in}}^{\mathbf{j}}\in {\mathbf{C}}^{{\mathbf{K}}^{\mathbf{2}}\times \mathbf{1}}$, and ${\mathbf{E}}_{\mathbf{s}}^{\mathbf{j}}\in {\mathbf{C}}^{{\mathbf{K}}^{\mathbf{2}}\times \mathbf{1}}$ denote the discrete version of the total fields, the incident fields, and the scattered field for the j-th illumination, respectively. The incident fields can be calculated by the Green Function. $\mathbf{\chi }\in {\mathbf{C}}^{{\mathbf{K}}^{\mathbf{2}}\times {\mathbf{K}}^{\mathbf{2}}}$ is the discrete version of the contrast in the imaging domain $\Sigma$. $\mathbf{\chi }$ is a diagonal matrix, and the element on the diagonal is the divided contrast on $\Sigma$. ${\mathbf{G}}_{\mathbf{s}}\in {\mathbf{C}}^{{\mathbf{K}}^{\mathbf{2}}\times {\mathbf{K}}^{\mathbf{2}}}$ and ${\mathbf{G}}_{\mathbf{m}}\in {\mathbf{C}}^{\mathbf{N}\times {\mathbf{K}}^{\mathbf{2}}}$ denote the Green’s object and data matrix, respectively, and $\mathbf{N}$ is the number of the electromagnetic waves generating antennas.

When each antenna acts as a transmitter, every vector ${\mathbf{E}}_{\mathbf{S}}^{\mathbf{j}}$, ${\mathbf{E}}_{\mathbf{z}}^{\mathbf{j}}$, and ${\mathbf{E}}_{\mathbf{in}}^{\mathbf{j}}$ were added to the measurement data matrix ${\mathbf{E}}_{\mathbf{S}}$, the total field matrix ${\mathbf{E}}_{\mathbf{z}}$, and the incident field matrix ${\mathbf{E}}_{\mathbf{in}}$ as one column, respectively. The equations become the following:

$${\mathbf{E}}_{\mathbf{S}}={\mathbf{G}}_{\mathbf{m}}\mathbf{\chi }{\mathbf{E}}_{\mathbf{z}},$$

(5)

$${\mathbf{E}}_{\mathbf{z}}={\mathbf{E}}_{\mathbf{in}}+{\mathbf{G}}_{\mathbf{s}}\mathbf{\chi }{\mathbf{E}}_{\mathbf{z}}$$

(6)

The inverse scattering problem is the use of the measured scattered field ${\mathbf{E}}_{\mathbf{S}}$ to estimate the contrast distribution $\mathbf{\chi }$ on $\Sigma$.

3. Methods

3.1. Motivation

Equations (5) and (6) are nonlinear in the process of inversion. The solution may not be unique, especially when the target is high-contrast objects. As a result of this, from the measured data ${\mathbf{E}}_{\mathbf{S}}$ to the object contrast $\mathbf{\chi }$, the process of directly solving Equations (5) and (6) is very difficult. Although it has been proven in previous experiments that neural networks can estimate the distribution of contrast, most of them cannot estimate the true contrast value of a certain point. Accordingly, we propose an easy but effective method named the diagonal matrix inversion method (DMI) to solve the nonlinear equations, and a neural network with the forward model to estimate the distribution of contrast and the true contrast value of a certain point. Next, we will explain the solution process of DMI and the implementation of this neural network.

3.2. Diagonal Matrix Inversion Method

For Equations (5) and (6), solving the contrast $\mathbf{\chi }$ directly is a nonlinear problem. However, if solving the inverse matrix of $\mathbf{\chi }$ directly, it can be regarded as a linear problem. Combining Equation (5) with Equation (6), we can get the following equation:

$${\mathbf{E}}_{\mathbf{S}}={\mathbf{G}}_{\mathbf{m}}{({\mathbf{\chi }}^{-\mathbf{1}}-{\mathbf{G}}_{\mathbf{s}})}^{-\mathbf{1}}{\mathbf{E}}_{\mathbf{in}}.$$

(7)

This equation is the forward function, and it will be embedded in the neural network as the forward model. For Equation (7), it is easy to derive the following equation:

$${\mathbf{\chi }}^{-\mathbf{1}}={\mathbf{E}}_{\mathbf{in}}{{\mathbf{E}}_{\mathbf{S}}}^{\Psi }{\mathbf{G}}_{\mathbf{m}}+{\mathbf{G}}_{\mathbf{s}},$$

(8)

where ${{\mathbf{E}}_{\mathbf{S}}}^{\Psi }$ is the generalized inverse matrix of ${\mathbf{E}}_{\mathbf{S}}$ and ${\mathbf{\chi }}^{-\mathbf{1}}$ is the inverse matrix of $\mathbf{\chi }$. ${{\mathbf{E}}_{\mathbf{S}}}^{\Psi }$ can be calculated by singular value decomposition. If a singular value decomposition is conducted on ${\mathbf{E}}_{\mathbf{S}}$, it can obtain Equation (9):

$${\mathbf{E}}_{\mathbf{S}}=\sum _{\mathbf{n}}{\mathbf{u}}_{\mathbf{n}}{\mathbf{s}}_{\mathbf{n}}{\mathbf{v}}_{\mathbf{n}}^{\mathbf{H}},$$

(9)

where ${\mathbf{u}}_{\mathbf{n}}$, ${\mathbf{v}}_{\mathbf{n}}$, and ${\mathbf{s}}_{\mathbf{n}}$ represent the left, right, and singular vectors and singular values of ${\mathbf{E}}_{\mathbf{S}}$. If ${\mathbf{s}}_i(0<i\le n)$ is less than $\mathbf{\lambda }max\left({\mathbf{s}}_{\mathbf{n}}\right)$, $s_i$ will be set to zero. $\mathbf{\lambda }$ is a constant that is used to filter small singular values. After discarding small singular values, the ${{\mathbf{E}}_{\mathbf{S}}}^{\Psi }$ can be calculated by the following equation:

$${{\mathbf{E}}_{\mathbf{S}}}^{\Psi }=\sum _{\mathbf{n}}{\left({\mathbf{v}}_{\mathbf{n}}^{\mathbf{H}}\right)}^{\mathbf{T}}{\mathbf{s}}_{\mathbf{n}}{\mathbf{u}}_{\mathbf{n}}^{\mathbf{T}}.$$

(10)

Because $\mathbf{\chi }$ is a diagonal matrix, the inverse matrix of a diagonal matrix must be a diagonal matrix and have the following property:

$${\mathbf{\chi }}_{i,i}=\frac{1}{{\left({\mathbf{\chi }}^{-1}\right)}_{i,i}},$$

(11)

so it is easy to calculate $\mathbf{\chi }$ from ${\mathbf{\chi }}^{(-\mathbf{1})}$. However, for the scattered field ${\mathbf{E}}_{\mathbf{S}}$, although it is a square matrix, the condition number is too large to calculate the generalized inverse of ${\mathbf{E}}_{\mathbf{S}}$ and it will suffer from some calculation errors. Because of these calculation errors, ${\mathbf{\chi }}_{\mathit{e}}^{-\mathbf{1}}$ is not an exact diagonal matrix, but the distribution of the elements on its diagonal is similar to ${\mathbf{\chi }}_{\mathit{r}}^{-\mathbf{1}}$s. (${\mathbf{\chi }}_{\mathit{e}}$ is the contrast calculated by DMI and ${\mathbf{\chi }}_{\mathit{r}}$ is the contrast calculated by the real contrast with Equation (11)). After the calculation of Equations (8) and (11), a contrast estimate appeared, and this will be the initial guess.

The above method of estimating contrast is called the diagonal matrix inversion method (DMI).

4. Implementation Details of the GAN

4.1. Structure and Central Idea

The main purpose of the GAN is to eliminate the error introduced by DMI. The GAN refers to the design of pix2pix. Pix2pix has been proven to be effective in image translation. Additionally, the essence of the image is a matrix, so it can say that pix2pix performs well from matrix to matrix and GAN may have great performances from the initial guess contrast matrix to the real contrast matrix. In terms of the generator, the GAN and pix2pix both adopt “U-Net”-based architecture, but the up-sampling layers and down-sampling layers used by pix2pix are seven layers, and our GAN is five layers. For the discriminator, since the data of each column in the scattering field measurement data matrix do not affect each other, the filter we used is $1\times N$, while pix2pix selects $N\times N$. It is different from pix2pix as it directly inputs the generated result into the discriminator, and we input them into the discriminator after the forward model calculation.

The GAN mainly contains three parts: the generator, forward model, and discriminator. The generator is of the “U-Net”-like structure, and the discriminator is of the “PatchGAN”-like structure. The generator tries to learn the mapping from the initial guess contrast to the real one. Additionally, the forward model can help the discriminator to more easily distinguish the contrast generated by the generator.

The generator architecture is shown in Figure 2. The inputs have two channels, where one is the real part of the contrast and another is the imaginary part, and so are the outputs. Different from the conventional U-Net, pooling layers are not used in the generator, and a full convolution structure is adopted. In the process of down-sampling, leakyrelu is adopted as the activation. Additionally, in the process of up-sampling, relu is adopted as the activation. The 4 × 4 kernel size is adopted for all layers. Additionally, the strides were set to 2.

The same idea as the patchGAN, the discriminator outputs a matrix as an evaluation of the overall generation results, but different from the traditional patchGAN, considers that each column in the scattering field measurement matrix is the measurement value generated by one scattering. Only each column is convoluted, and the column dimension of the final output matrix and the input matrix is reduced and the row dimension is unchanged. Different from the generator, the inputs only have one channel and the imaginary part is spliced below the real part. All the parameters of the discriminator can be seen in

Table 1. Parameters of each layer in the discriminator.

Layer Name	Inputs Shape	Outputs Shape	Filters	Kernel	Stride
Conv1	$48\times 24$	$24\times 24$	64	$4\times 1$	$2\times 1$
Conv2	$24\times 24$	$12\times 24$	128	$4\times 1$	$2\times 1$
Conv3	$12\times 24$	$6\times 24$	256	$4\times 1$	$2\times 1$
Conv4	$6\times 24$	$3\times 24$	1	$4\times 1$	$2\times 1$

.

In the process of training, the generator and the discriminator confront each other so that the generator and the discriminator achieve a balance. Due to the existence of a forward model, the discriminator determines the generation effect of the generator by distinguishing the measured value of the scattered field from the actual measured value. Additionally, it is not only the distribution of contrast, but also the contrast of each pixel that can be generated by the generator. It can be seen that the greatest advantage of GAN is to embed the forward model between the generator and the discriminator, so that the discriminator has a stronger ability to judge the results generated by the generator.

4.1.1. Loss Function

The loss function of the GAN is similar to traditional pix2pix. The loss function of cGAN with L1 distance is used for the generator, and the binary cross-entropy loss function is used for the discriminator. Additionally, the loss function of the cGAN and L1 distance can be expressed as the following Equations (12) and (13), respectively:

$${\mathbf{L}}_{\mathbf{cGAN}}(\mathbf{G},\mathbf{D})={\mathbf{E}}_{\mathbf{x},\mathbf{y}}\left[\log \mathbf{D}(\mathbf{x},\mathbf{y})\right]+{\mathbf{E}}_{\mathbf{x},\mathbf{G}\left(\mathbf{x}\right)}\left[\log (\mathbf{1}-\mathbf{D}(\mathbf{x},\mathbf{G}\left(\mathbf{x}\right)))\right]$$

(12)

and

$${\mathbf{L}}_{\mathbf{l}\mathbf{1}}\left(\mathbf{G}\right)={\mathbf{E}}_{\mathbf{x},\mathbf{y}}\left[\right||\mathbf{y}-{\mathbf{G}\left(\mathbf{x}\right)\left|\right|}_{\mathbf{1}}].$$

(13)

Combining Equations (12) and (13), the final loss is:

$${\mathbf{L}}_{\mathbf{G}}=\arg \underset{\mathrm{G}}{min}\underset{\mathrm{D}}{max}{\mathbf{L}}_{\mathbf{cGAN}}\lambda {\mathbf{L}}_{\mathbf{l}\mathbf{1}}\mathbf{G},$$

(14)

where $\lambda$ controls the relative importance of the two loss functions.

4.1.2. Network Training

The GAN and DMI, applied for nonlinear inverse scattering, can be seen in Figure 3. The input data of the generator come from the DMI algorithm after normalization.

The training procedure for the GAN is as follows:

• In the first step, the real measured data can be calculated by the forward model (Equation (7)) with the ground truths;
• In the second step, the initial contrasts can be obtained by DMI with the measurement data. Additionally, the initial contrasts are divided into the real part and the imaginary part, and these two parts are different channels for the inputs of the generator after normalization. The generator converts the normalization into the contrast closed to the real;
• In the third step, the measured data obtained in the second step is sent to the discriminator to compare with the real measured data;
• In the final step, according to the results of the generator and the discriminator, the discriminator and generator are optimized to balance them.

The experimental results show that the GAN with the forward model has a good effect on eliminating the error introduced by DMI.

5. Numerical Simulation and Experimental Results

The performance of the GAN and DMI is assessed from the two aspects of “simulation” and “experiment”. Additionally, we also compare with the Multiplicative Regularization Contrast Source Inversion (MR-CSI) method [11], and both of them use the measured data calculated by Equation (7). The imaging of electromagnetic inverse scattering technology is mainly used in the early detection of breast cancer. Its features are ellipses, circles, and long bars. These features happen to be included in handwritten numerals. Therefore, we choose the MNIST as the training dataset.

5.1. Training and Testing on MNIST Datasets

The experiment uses MNIST handwritten digits to evaluate the GAN. A total of 8000 examples were used to train the GAN, and 2000 examples were used for testing. Additionally, The pixel value was between 0 and 255 in MNIST. All the examples were converted from $28\times 28$ to $64\times 64$ pixels. Additionally, in this experiment, each image was converted to float64 and normalized to [0, 1].

Referring to Figure 1 and considering the experimental environment, the imaging region $\mathbf{\Sigma }$ is a square with a size of $0.07\times 0.07$ m, and it is composed of $64\times 64$ cells on average. A total of 24 transmitting–receiving antennas were evenly distributed on the circular region D, including the imaging region $\mathbf{\Sigma }$. The relative permittivity ${ϵ}_r$ was set to 1. The radius of the circular region $\mathbf{D}$ is represented by $R=0.1m$. Additionally, the system works at 4.4 GHz.

The training of the GAN was optimized by the ADAM optimization method [30]. The learning rate of the generator was set to ${10}^{-5}$, and the discriminators were set to $2\times {10}^{-4}$. The filters were initialized randomly. All calculations were performed on a small server configured with 256 GB of memory with Intel(R) Xeon(R) CPU E5-2690 v4 and Tesla K80. The GAN and DMI were implemented by a tensorflow library [31]. Additionally, the MR-CSI algorithms were also implemented by the tensorflow and numpy library [31,32]. The DMI algorithm took about 4 s, and the MR-CSI algorithm, with 500 iterations, took about 5 min. The training time of the GAN was about 24 h and reconstructed process was about 5 s.The numerical simulation process is as follows:

• First, the scattered field measurements can be calculated by Equation (7) with ground truths;
• Second, the initial contrast images can be calculated by Equations (8) and (9) (DMI) with the scattered field measurements that are calculated by the first step;
• Finally, the initial contrast images are the inputs of the GAN, and the outputs are the real images.

In this example, the real parts of the contrast are the ground truths and the imaginary parts are all set to 0. The contrast is used for the forward model to calculate the measurement data. Then, the measurement data are used for DMI to estimate the initial contrast with errors. The initial contrast is used to generate the real contrast. Figure 4 shows the ground truths of the MNIST, the image of the contrast distribution calculated by DMI, the image of the contrast distribution generated by GAN, and the result calculated by MR-CSI with 500 iterations, respectively. Figure 5 shows the ground truths and the distribution images of scattered field measurements that were calculated by the ground truths in Figure 4. In order to compare the effects of different methods on imaging quality, the scattering field reconstruction error (SRE) and structural similarity (SSIM) were used to evaluate the imaging quality. The formula to calculate SRE and SSIM are as follows:

$$\mathrm{SRE}=\sum _{i=0}^K\sum _{j=0}^K|{\mathbf{X}}_{ri,j}-{\mathbf{X}}_{si,j}{|}^2,$$

(15)

$$\mathrm{SSIM}(\mathrm{x},\mathrm{y})=\frac{(2{\mu }_x{\mu }_y+C_1)(2{\sigma }_x{\sigma }_y+C_2)}{(2{\mu }_x^2{\mu }_y^2+C_1)(2{\sigma }_x^2{\sigma }_y^2+C_2)},$$

(16)

where ${\mathbf{X}}_r$ is the real distribution of contrast and ${\mathbf{X}}_s$ represents the reconstruction distribution of contrast (including the real part and imaginary part). In Equation (16), x represents the real part of the reconstruction, and y is the real part of the original. The constant $C_1$ and $C_2$ is included to avoid instability when the denominator is very closed to zero. $\mu$ and $\sigma$ represent the luminance of the image and the contrast of the image, respectively. $\mu$ and $\sigma$ could be calculated by the following formula:

$${\mu }_x=\frac{1}{N}\sum _{i=1}^n{x}_i,$$

(17)

$${\sigma }_x={(\frac{1}{N-1}\sum _{i=1}^n{(x_i-{\mu }_x)}^2)}^{\frac{1}{2}}.$$

(18)

Table 2. SRE results for the reconstructions in Figure 4.

Ground Truths	DMI	GAN	MR-CSI
	111.7656	45.28245	61.52228
	105.4205	31.68619	48.43685
	94.92069	21.66978	22.43685
	130.9329	12.30507	19.54651
	66.78501	22.64007	23.60733
	214.5837	39.78957	42.28401
	29.08561	8.097077	11.39325
	140.8501	51.77450	72.77665
	151.5352	53.31921	72.77665
	178.7540	21.95760	43.64304
	183.6041	43.37700	53.37468
	124.1460	29.11482	32.13470

and

Table 3. SSIM results for the reconstructions in Figure 4.

Ground Truths	DMI	GAN	MR-CSI
	0.7138551	0.8100136	0.7487772
	0.7740224	0.8982024	0.8435692
	0.7785497	0.8807378	0.8555204
	0.7513792	0.9175278	0.9015835
	0.8042963	0.8906798	0.8541093
	0.6552282	0.8431529	0.8194608
	0.9351476	0.9759904	0.9055687
	0.6925802	0.7931228	0.7598765
	0.6980058	0.8094963	0.7807395
	0.6422808	0.8665296	0.7998289
	0.6691707	0.8308884	0.7951557
	0.8663689	0.7061396	0.8453851

show the corresponding SRE and SSIM of different methods, respectively.

For SRE, a smaller value means a better imaging effect. However, for SSIM, a higher value means a better imaging effect. As can be seen from the above, the GAN reduces the error introduced by DMI to a certain extent, and the imaging effect is better than traditional methods. In ISP, the reconstruction result is usually compared with the real part of the ground truth. However, only considering the reconstruction effect of the real part is not complete, and the imaginary part is also a very important factor. The SRE both considers the real part and the imaginary part, and it can better explain the reconstruction effect. In the following experiments, we will pay more attention to the results of SRE. For Table 2, the GAN significantly reduced the SRE results of DMI. Additionally, compared with the MR-CSI method, the SRE results of the GAN are also smaller.

5.2. Testing on EMNIST Dataset

In order to verify the superiority of this method, another dataset is tested. In this test, the EMNIST [33] dataset is used to test with the model which was trained successfully by MNIST. The EMNIST dataset contains digits, uppercase and lowercase handwritten letters, and some of them are used to test. The pixel value is also between 0 and 255. Similarly, all test examples were enlarged to $64\times 64$ pixels, converted to float64, and normalized to [0, 1].

In this example, the real parts of the contrast are the ground truths in Figure 6 and the imaginary parts were also set to 0. Figure 6 shows the reconstruction results based on different methods. The SRE is used to evaluate the imaging effects of these three methods. Additionally, the results are shown in

Table 4. SSIM results for the reconstructions in Figure 6.

Ground Truths	DMI	GAN	MR-CSI
	103.2088	19.04672	29.22322
	116.3270	51.63381	71.20323
	99.05278	16.26200	20.75824
	213.6186	38.95768	82.97401
	49.62779	12.97790	26.17604
	152.3258	21.10506	35.66177
	114.4213	32.43655	32.50585
	15.60317	10.26880	11.28294

. Through the above experiments, it can be concluded that although the GAN is only trained by the MNIST dataset, excellent reconstruction results can still be obtained by using the trained model for objects with different characteristics. This shows that GAN has good generalization ability and can eliminate the calculation error introduced by DMI. Using DMI combined with the GAN is an excellent method to solve ISP.

5.2.1. Comparing the Imaging Ability with CVP2P

We also wanted to know whether the imaging ability of the GAN is better than other neural networks. We decided to compare the imaging results with CVP2P [24]. CVP2P is mainly used for the image which is composed of 0 and 1. Thus, we converted the ground truths from [0, 1] to 0 and 1. If the pixel value is greater than 0.5, we set its real parts to 1, and if the pixel value is less than 0.5, we set its real parts to 0. All the imaginary parts were set to 0. The ground truths were used for calculating the measurement data by the forward model. Additionally, the DMI method was used to estimate the initial images. The initial images are the inputs of the GAN. The imaging results of CVP2P and the GAN are shown in Figure 7. Additionally, the SRE results are shown in

Table 5. SRE results for the reconstructions in Figure 7.

Ground Truths	CVP2P	GAN
	523.0	269.1223
	450.0	361.1265
	849.0	387.2867
	306.0	104.5572
	483.0	268.2372
	420.0	168.0862

.

5.2.2. Testing with Lossy Scatterers

In order to verify the effectiveness of this method on lossy scatterers, the contrast range of the real part was set to [0, 1] and the imaginary part of the target was set to [1, 2]. In this example, the real part and the imaginary part of the contrast are all ground truths in Figure 8. Figure 8 shows the imaging results of these methods on lossy scatterers. The imaging results include real and imaginary parts. The SRE result of the GAN is 98.12068 and that of the MR-CSI is 182.1479.

5.2.3. Testing with More Complex Images

Handwritten letters and numbers have different characteristics from human tissues. In order to verify the feasibility of this method, we use a more complex image composed of multiple circles as test samples to test the DMI and GAN. In this example, the real part of the contrast is the ground truth in Figure 9, and the imaginary part was set to 0. The test results are shown in Figure 9. The SRE result of the GAN is 109.2299 and the MR-CSI is 127.7074. Additionally, the SSIM result of the GAN is 0.75476 and the MR-CSI is 0.7223. It can be verified that the GAN with DMI still has better imaging ability than MR-CSI even on images with different features through this test.

5.2.4. Testing with Varying Degrees of Noise

In this experiment, the white noise will be added to the scattered field measurements. The real parts are the handwriting digital image, and the imaginary parts are set to 0. The scattered field measurements are calculated by Equation (7) and varying degrees of noise are added to them. Figure 10 shows the experimental results.

It can be seen from the experimental results that when the noise is lower than 10%, the GAN can still complete the reconstruction task well. When the noise is higher than 10%, the reconstruction ability of the GAN is greatly weakened. This is because the DMI method cannot obtain a good initial result, resulting in the deviation of the GAN reconstruction.

5.2.5. Testing Trained Networks by Experimental Data

In order to test the imaging effect of DMI and the GAN in the actual experiment, a self-made experimental system is used to obtain the measured data of the scattering field for verification.

The whole experimental system can work at 3–5 GHz. A total of 24 balanced Vivaldi antennas were evenly distributed on a cylinder with a radius of $0.1$ m as electromagnetic wave-generating devices. Each Vivaldi antenna is 7 cm long and $7.3$ cm wide. The maximum imaging area is a square area with a side length of 7 cm. The model of the vector network analyzer is kc901v and connected to the antenna through an Agilent coaxial matrix switch to control the reception and transmission of electromagnetic waves. Additionally, the analyzer provides port isolation of greater than 100 dB over the frequency range of interest. The computer is connected to the vector network analyzer through a USB and automatically collects measured data through the Agilent coaxial matrix switch control device. The target is a square wooden block with a side length of 5 cm, and its contrast is 2. The same as the previous numerical simulation, the imaging area D is evenly divided into $64\times 64$ small cells. The target is shown in Figure 11.

The measurement data of the target are shown in Figure 12. The GAN trained by the MNIST dataset is used to test the experimental data. Figure 13 shows the generation results of the ground truth, DMI, MR-CSI, and the GAN at $4.4$ GHz operating frequency. The SRE result of the GAN is 76.97342 and the MR-CSI is 140.7981. Although the experimental data are quite different from the simulated data, the results of the GAN are satisfactory and better than MR-CSI in imaging effect. Adding the actual experimental data to MNIST for training, the trained GAN can reduce the SREs for actual experimental measurements. MNIST may not be perfect for application. It would be better if some actual experimental data were added for training. For a particular application, it is best to add some data with features that are similar to the application to MNIST datasets for training.

6. Conclusions

In this paper, by deriving the formula of the forward process, an equation that is easier to obtain the approximate contrast was obtained, and a neural network framework was constructed to convert the obtained approximate contrast into a closer and more real contrast. In addition, the quantitative comparison between the simulation and experimental data showed that DMI can easily obtain better contrast, and that the GAN can further optimize this result to get better results. Additionally, in the solving process of DMI with the GAN were non-iterative processes, and the reconstruction speed was much better than MR-CSI.