A Machine Learning-Assisted Inversion Method for Solving Biomedical Imaging Based on Semi-Experimental Data

Abstract

Machine learning approaches have been extensively utilized in the field of inverse scattering problems. Typically, the training dataset is generated synthetically using ideal radiation sources such as plane waves or cylindrical waves. However, the testing data often consist of experimental data that take into account the antenna port couplings and waveform distortions within the system. While noise can be artificially added to synthetic data, it may not accurately represent the real experimental noise. Consequently, the application of machine learning-assisted inversion techniques may encounter challenges when the training dataset differs significantly from the experimental data. In this paper, we propose an experimental system specifically designed for human body imaging. A semi-experimental training dataset is constructed using full-wave simulation software, incorporating the relative permittivities of common human tissues. Furthermore, the system noise is meticulously considered through full-wave simulation, enhancing the authenticity of the dataset. A back-propagation scheme is firstly employed to obtain the rough reconstructed images. Then, the U-net convolutional neural network (CNN) is employed to map these rough images to high-resolution ones. Numerical results demonstrate that, in comparison to networks trained solely on synthetic data, the network trained using semi-experimental data achieves superior reconstruction results with lower errors and improved image quality.

1. Introduction

Inverse scattering problems (ISPs) have been widely applied in geology exploration [1], biomedical imaging [2], nondestructive evaluation [3] and microscopy [4]. The permittivities of unknown scatterers can be reconstructed by the measured scattering data, which will be considered one of the ideal detection approaches in the future due to its quantitative and nondestructive imaging ability [5]. In a biomedical application scenario, the imaging techniques based on electromagnetic ISPs are able to recognize the slight variations of electromagnetic properties of human tissue such as breast [6,7,8], head [9] and leg [10], which helps to detect early stage cancer or soft tissue damage. Traditional methods to solve ISPs, such as the distorted born iterative method (DBIM) [11] and subspace-based optimization method (SOM) [12,13], have considered all the multiple scattering effects and are low in computational speed as the time consuming iterative optimization is involved. The past decades witnessed the huge development of computer technology which inspired researchers applying deep learning methods to solve ISPs [14,15,16] and they have achieved satisfying results. Li et al. [17] have combined deep neural network (DNN) architecture with the iterative method of inverse scattering. Ye et al. [18] have proposed a real-time imaging method for inhomogeneous background problems using the generative adversarial network (GAN) with an attention mechanism. Wei et al. [19] have proposed a physics-inspired neural network to solve the ISPs. The contrast matrices of scatterers are firstly calculated by the back-propagation scheme (BPS) [20], then the trained U-net CNN recovers the high resolution information of the coarse image obtained by the BPS. However, in all these works, the scattered field used in the training dataset is obtained by a numerical forward solver that neglects the impact of system noise. Even though artificial noise can be added to the synthetic data, it may be quite distinct from the real experimental one. As a result, the machine learning assisted inversion may fail due to the difference of the training dataset and the experimental one.

In this paper, the physics-assisted machine learning approach is applied to explore biomedical imaging using semi-experimental data generated by full wave simulation. The imaging system is a three-dimensional one, and by properly designing the polarization of the antenna and shape of the scatterers, the system can be solved by the 2D approximation. The handwriting numbers in MNIST datasets [21] are used to generate the cylindrical scatterers embedded in the imaging system. A back-propagation scheme is firstly used to generate the coarse images of the relative permittivities, which are used as the inputs of the network. A U-net CNN [22] is applied to map between the coarse image and high resolution image of the reconstructed relative permittivity. A comparison is made between the networks trained by synthetic data and semi-experimental data. The experiments show that the proposed dataset which is built up by the full wave simulation software according to the relative permittivities of the common human tissue can effectively improve the imaging accuracy. In addition, the proposed method is able to quantitatively image scatterers with relative permittivities of common human tissue, which reveals a promising biomedical imaging application.

The contributions of the proposed method are summarized as follows. A real and valid biomedical imaging system is established using full wave simulation software, incorporating a suitable matching medium and a designed antipodal Vivaldi antenna array. System noise of the whole imaging system is fully taken into account through the full wave simulation, enhancing the authenticity of the imaging process. Meanwhile, the system is specifically designed for human body imaging, and thus the scatterers are generated based on the actual constitutive parameters of human tissue. Both the scatterers and the system exhibit three-dimensional characteristics. By considering the TM polarization of the Vivaldi antenna and the cylindrical shape of the scatterers, the entire imaging system can be effectively approximated using a 2D model. To further validate the implementability of the proposed method, synthetic data generated via full wave simulation are employed. The system noise includes antenna coupling noise, multi-reflection caused by system walls, a non-ideal waveform generated by the antenna and the inherent error resulting from the 2D approximation. This comprehensive consideration enables a thorough evaluation of the proposed method’s effectiveness in handling experimental data.

This paper is organized as follows. In Section 2, the formulation of the forward problem and the simulation system are introduced. In Section 3, the back-propagation scheme (BPS) and the structure of the U-net CNN are introduced. In Section 4, the details of implements are given, including the settings of simulation and the calibration method. In Section 5, the numerical results are given to verify the generalization ability of the network. Finally, we summarize this paper in Section 6.

2. Forward Problem and Simulation System

2.1. Forward Problem

In this paper, a 2D transverse magnetic (TM) case is considered. The unknown scatterers are located in the domain of interest (DOI), which is illuminated by $N_i$ cylindrical waves that are located at ${\mathbf{r}}_p^i,p=1,2,\cdots ,N_i$. For each incidence, there are $N_s$ receiving antennas placed circularly to measure the scattered field in the S domain, which are located at ${\mathbf{r}}_q^s,q=1,2,\cdots ,N_s$. The method of moment (MoM) with the pulse basis function and the delta testing function is used to calculate the synthetic scattered field. The domain D is discretized into $M\times M$ square subunits, the centers of which are located at ${\mathbf{r}}_1,{\mathbf{r}}_2,\cdots ,{\mathbf{r}}_{M^2}$. The experimental setup of typical ISPs are as shown in Figure 1. The total field ${\overline{E}}^t$ in domain D and the scattered field ${\overline{E}}^s$ in domain S can be described by the discretized form of the electric field integral equation, respectively:

$${\overline{E}}^t={\overline{E}}^i+{\overline{\overline{G}}}_D·\overline{J}$$

(1)

$${\overline{E}}^s={\overline{\overline{G}}}_S·\overline{J}$$

(2)

where ${\overline{\overline{G}}}_D$ means the 2D Green’s function in domain D and ${\overline{\overline{G}}}_S$ is the Green’s function that maps the current in domain D to the scattered field on the receivers. The dimensions of the matrices ${\overline{\overline{G}}}_D$ and ${\overline{\overline{G}}}_S$ are $M^2\times M^2$ and $N_s\times M^2$, respectively.

The induced current $\overline{J}$ is defined as

$$\overline{J}=\overline{\overline{\chi }}·{\overline{E}}^t$$

(3)

where $\overline{\overline{\chi }}$ is a diagonal matrix, the nth diagonal element is represented as $\overline{\overline{\chi }}(n,n)=-i\omega {\epsilon }_0[{\epsilon }_r\left({\mathbf{r}}_n\right)-{\epsilon }_b\left({\mathbf{r}}_n\right)]$ and ${\epsilon }_r\left({\mathbf{r}}_n\right)$ and ${\epsilon }_b\left({\mathbf{r}}_n\right)$ denote the relative permittivity of the scatterer and background located at location ${\mathbf{r}}_n$, respectively.

2.2. The Imaging System

An improved antipodal Vivaldi antenna is proposed which is suitable for biomedical imaging [23]. Considering the fact that the method of moment uses the line source which is different from the actual wave generated by the antenna, a calibration method should be introduced [24].

The relative permittivities of skin and air are significantly different (1 compared to 40) which is not conducive to the propagation of electromagnetic waves and most of the incident waves are reflected by the skin [25]. Thus, we add the matching medium, the permittivity of which is close to the skin, between the scatterers and the antennas [26]. The loss tangent of the matching medium is set as 0.148, which demonstrates the best performance to inhibit the coupling between antennas and make sure that the electromagnetic wave propagates in the form of a cylindrical wave.

3. Inverse Problem and U-Net

3.1. Back-Propagation Scheme (BPS)

The back-propagation scheme (BPS) assumes that the induced current is proportional to the BP field

$$\overline{J}=\gamma ·{\overline{\overline{G}}}_S^H·{\overline{E}}^s$$

(4)

The loss function is defined using the squared error of the scattering field

$$F\left(\gamma \right)={∥{\overline{E}}^s-{\overline{\overline{G}}}_S·(\gamma ·{\overline{\overline{G}}}_S^H·{\overline{E}}^s)∥}^2$$

(5)

To minimize the loss function, the derivative of $\gamma$ is required to be zero and the analytical expression of $\gamma$ can be obtained

$$\gamma =\frac{{\left({\overline{E}}^s\right)}^T·{({\overline{\overline{G}}}_S·({\overline{\overline{G}}}_S^H·{\overline{E}}^s))}^{*}}{{∥{\overline{\overline{G}}}_S·({\overline{\overline{G}}}_S^H·{\overline{E}}^s)∥}^2}$$

(6)

where T means the transpose and * means the complex conjugate. The induced current can be obtained using (4) once $\gamma$ is solved. The total field of domain D can be expressed as

$${\overline{E}}^t={\overline{E}}^i+{\overline{\overline{G}}}_D·\overline{J}$$

(7)

For each incident p, the contrast $\overline{\chi }$ of permittivity and the induced current ${\overline{J}}_p$ satisfy the relation

$${\overline{J}}_p=\mathrm{diag}\left(\overline{\chi }\right)·{\overline{E}}_p^t$$

(8)

Applying (8) into all the incidents results in a least squares problem and the analytical expression for the nth element of $\overline{\chi }\left(n\right)$ can be obtained

$$\overline{\chi }\left(n\right)=\frac{{\sum }_{p=1}^{N_i}{\overline{J}}_p\left(n\right)·{\left[{\overline{E}}_p^t\left(n\right)\right]}^{*}}{{\sum }_{p=1}^{N_i}{∥{\overline{E}}_p^t\left(n\right)∥}^2}$$

(9)

3.2. U-Net Convolutional Neural Network

The U-net structure is firstly proposed to solve the segmentation of medical images [22], which is also proved to be effective for pixel-based mapping [19].

The U-net architecture with the skip connection is chosen to improve the resolution of the coarse images obtained by the BPS. The high-resolution information can be learned by the neural network effectively without resourcing to the iteration. The structure of the network is as shown in Figure 2. The network includes two components: a contracting path and an expansive path, designed to facilitate efficient information flow. The contracting path encompasses $3\times 3$ convolutions, batch normalization (BN), and rectified linear unit (ReLU) activations, followed by a $2\times 2$ max pooling operation. Conversely, the expansive path mirrors the contracting path, with the max pooling operation replaced by a $3\times 3$ upconvolution. Furthermore, the expansive path involves two concatenations with the correspondingly cropped feature maps from the contracting path. The number of input channels and output channels are both 2, representing the real part and imaginary part of the relative permittivity of the scatterers. The number of channels C is set as 64 in this paper.

4. Details of Implements

4.1. Simulation Setup of the Imaging System

The schematic of the simulation system is as shown in Figure 3 and the side view is as shown in Figure 4. The imaging system is composed of 16 Vivaldi antennas and the matching medium. The polarization of the electric field is along the z-axis, and the whole system can be approximated by a cylindrical one if the scatterer is long enough along the z-axis. The incident field is collected by the antennas without scatterers, and the scattered field is obtained by extracting the incident field from the total field with the scatterer presenting in the system.

The domain of interest is of size $0.1\times 0.1$${\mathrm{m}}^2$, which is discretized into $40\times 40$ pixels. There are 16 antennas evenly distributed on the circle of radius 0.119 m centered at (0,0) m, which work as both the transmitters and receivers. The frequency of incident is set as 635 MHz where the transfer efficiency of the antenna performs best. The relative permittivity of human tissues in the microwave frequency band are as shown in

Table 1. The relative permittivity of human tissues in the microwave frequency band.

Tissues	Skin	Blood	Fat	Muscle
Real part	44.9149	63.2572	11.5400	56.4454
Imaginary part	26.1865	49.7333	3.06836	29.5676

. Considering the fact that the human body is mostly composed of water (the dielectric constant of which is 80), the real part of relative permittivity of scatterers in the training set is randomly set between 10 and 80, the imaginary part is randomly set between 0 and 50. The relative permittivity of matching medium is 37.7–5.6i, which can be realized by mixing the pure water with alcohol of volume portion 72.6%.

We use the MNIST dataset with a random circular-cylinder for the training dataset. In order to generate a large number of training sets quickly, a joint simulation method based on MATLAB and High Frequency Structure Simulator (HFSS) was used.

4.2. Calibration Method

There are 16 transmitting antennas and the scattered field of each transmitting antenna is measured by 15 receiving antennas. The scattered field at the transmitting antenna is interpolated by the scattered field measured by two adjacent receiving antennas. The scattered field calculated by the cylindrical incident wave using the method of moments (MoM) is different from the one generated by the imaging system. Thus, the calibration method should be used to obtain the accurate semi-measurement data.

For each transmitting antenna, the calibration parameter can be expressed as follows

$$\mathrm{Cali}=\frac{E_{\mathrm{MoM}\left(\mathrm{op}\right)}^{\mathrm{inc}}}{E_{\mathrm{sim}\left(\mathrm{op}\right)}^{\mathrm{inc}}}$$

(10)

where $E_{\mathrm{sim}\left(\mathrm{op}\right)}^{\mathrm{inc}}$ and $E_{\mathrm{MoM}\left(\mathrm{op}\right)}^{\mathrm{inc}}$ mean the simulated incident field received by the antenna at the opposite position to this transmitting antenna and the one simulated by the ideal line source, respectively.

The calibrated scattered field ${\overline{E}}_{\mathrm{cal}}^{\mathrm{sca}}$ of this transmitting antenna can be expressed as follows, where ${\overline{E}}_{\mathrm{sim}}^{\mathrm{sca}}$ means the simulated scattered field:

$${\overline{E}}_{\mathrm{cal}}^{\mathrm{sca}}=\mathrm{Cali}·{\overline{E}}_{\mathrm{sim}}^{\mathrm{sca}}$$

(11)

The signal-to-noise ratio (SNR) can be considered as the ratio of the theoretical scattered field, which is the useful signal, and the difference between the simulated and theoretical scattered field, which can be defined as the noise. The average SNR of the whole system can be calculated by

$$E_{rr}=20·{\log }_{10}(\frac{1}{N_i\times N_s}·\sum _{p=1}^{N_i}\sum _{q=1}^{N_s}\frac{|{\overline{E}}_{\mathrm{MoM},p,q}^{\mathrm{sca}}|}{|{\overline{E}}_{\mathrm{cal},p,q}^{\mathrm{sca}}-{\overline{E}}_{\mathrm{MoM},p,q}^{\mathrm{sca}}|})$$

(12)

After calibration, the amplitude and phase of the simulated scattered field are able to reach a good agreement with the theoretical scattered field. Figure 5 and Figure 6 show the comparison of the scattered field calculated by MoM and HFSS of the first sample in the training dataset. The first picture means the magnitude and phase of the 16 receiving antennas while the first transmitting antenna is working. As a continuation, Figure 5 and Figure 6 illustrate the calibration status of the phase and magnitude of the scattering field received by the receiving antenna when the 1st to 8th and 9th to 16th transmitting antennas are operating, respectively. The average SNR of the whole training dataset is 0.78 dB.

4.3. Training Process

We used a personal computer with one piece of GeForce RTX3090 GPU and CPU Intel Xeon Gold 6248 to train the neural network.

In the training process, the reconstruction results of BPS are fed into the U-net CNN and the outputs are the ground truth relative permittivity in domain D. We used the MNIST dataset with a random circular-cylinder added for both training and testing. The whole dataset is composed of 3000 samples, 90% are used for training and 10% are used for testing. The hyperparameters for training are as follows: the batch size is 10; the learning rate is initially set to 1 $\times {10}^{-2}$ and the learning rate of each parameter group decays by 0.1 once the number of epochs reaches 30 and 80; the ADAM optimizer is adopted with ${\beta }_1$ = 0.9, ${\beta }_2$ = 0.999. The loss function used for training is the mean squared error (MSE). The training epoch is set as 300 as the loss function curve tends to flatten.

5. Numerical Results

5.1. First Example: MNIST Data with Random Circular-Cylinder

In the first example, the scatterers to be reconstructed are four samples of MNIST data with a random circular-cylinder which are previously unseen in the training process. The reconstructions of the MOM-based and HFSS-based dataset using the U-net CNN are as shown in Figure 7. The results show that the HFSS-based dataset performs better than the MOM-based dataset. We calculated the average relative error and SSIM of 100 samples which are not included in both the training and validating datasets as listed in

Table 2. The comparison of the proposed method and traditional method of MNIST data.

	Average Relative Error	SSIM
MOM + U-net CNN	14.01%	0.8501
HFSS + U-net CNN	12.93%	0.8540

. The results have shown that the robustness of the U-net CNN network and the HFSS-based dataset can improve the performance of the network in the relative error and SSIM.

5.2. Second Example: Circular-Cylinder

In the second example, a scatterer with the shape of a circular-cylinder is tested. The radius of the cylinder is 0.015 m centered at (0,0) m. The relative permittivity of the cylinder is 80–0i. This example has never been seen in the training and validating datasets. The reconstruction images for the relative permittivity of the MOM-based and HFSS-based U-net CNN are as shown in Figure 8. The comparison of the proposed method and traditional method are listed in

Table 3. The comparison of the proposed method and traditional method of the second example.

	Average Relative Error	SSIM
MOM + U-net CNN	1.00%	0.9411
HFSS + U-net CNN	0.58%	0.9438

, which demonstrates the better performance of the semi-experimental data trained network.

5.3. Third Example: Austira

In the third example, we use a HFSS to simulate the scattered field of profile ’Austria’, which consists of two cylinders and one ring. The center of the ring is located at (0, −0.01) m, the inner radius and outer radius of the ring are 0.015 m and 0.03 m, respectively. Both of the cylinders have a radius of 0.01 m and are centered at (0.015, 0.03) and (−0.015, 0.03) m. This example is special because the shape of the ring is extremely different from the digitals in the training data set. The reconstructions of the MoM-based and HFSS-based U-net are shown in Figure 9. The comparison of the proposed method and traditional method is listed in

Table 4. The comparison of the proposed method and traditional method of the third example.

	Average Relative Error	SSIM
MOM + U-net CNN	5.27%	0.7958
HFSS + U-net CNN	4.39%	0.8045

. This example has verified the generalization ability of the proposed method and demonstrates that the proposed method promises a real-time application future of solving biomedical imaging.

6. Conclusions

A U-net-based inversion method is proposed to solve the biomedical imaging problem. The method includes two parts. The first part is the dataset, which is generated according to the relative permittivities of the common human tissue, and system noise is fully taken into account through the full wave simulation of the whole imaging system, which is more authentic in terms of the imaging of the real system. It strikes an average between the theoretical simulation data and experimental data. The former completely ignore the noise, which is not in line with reality and the latter is hard to obtain and time-consuming. The second part is the U-net CNN-assisted inversion method, which outputs high-resolution imaging by using the coarse input image obtained by BPS. The results have shown the robustness of the network. The proposed method can further improve the performance of the network in relative error and SSIM. The numerical experiments have shown that the proposed method outperforms the traditional method in accuracy and generalization ability, which promises a real-time application future of solving biomedical imaging. Our next step in future work will be building up a real system to provide the experimental data and developing a 3D deep learning-assisted imaging algorithm with lower approximation noise.