Real-Time Nonlinear Image Reconstruction in Electrical Capacitance Tomography Using the Generative Adversarial Network

Abstract

This study investigated the potential of the generative adversarial neural network (cGAN) image reconstruction in industrial electrical capacitance tomography. The image reconstruction quality was examined using image patterns typical for a two-phase flow. The training dataset was prepared by generating images of random test objects and simulating the corresponding capacitance measurements. Numerical simulations were performed using the ECTsim toolkit for MATLAB. A cylindrical sixteen-electrode ECT sensor was used in the experiments. Real measurements were obtained using the EVT4 data acquisition system. The reconstructed images were evaluated using selected image quality metrics. The results obtained using cGAN are better than those obtained using the Landweber iteration and simplified Levenberg–Marquardt algorithm. The suggested method offers a promising solution for a fast reconstruction algorithm suitable for real-time monitoring and the control of a two-phase flow using ECT.

1. Introduction

The primary use of electrical capacitance tomography is in monitoring the two-phase flow, particularly when a cylindrical tomography sensor is attached to a horizontal pipeline section in a chemical or process engineering plant [1,2]. Typical structures observed in multiphase flow include annular, stratified, and slug patterns, where one or two circular cross-section elements may appear in the resulting image [3,4].

There is a critical need to develop and evaluate the measurement and image reconstruction methods to keep pace with the high-speed flow of multiphase materials in industrial installations [2,5,6]. In such environments, it is crucial that the imaging system not only captures accurate data but also processes it rapidly to provide real-time monitoring and control [7]. Though effective, nonlinear approaches such as the Levenberg–Marquardt algorithm (LM) are computationally intensive [8,9,10]. For example, the LM algorithm requires repeated updates of the sensitivity matrix calculated by forward problem solving, i.e., complex calculations of the electric field distributions. On the other hand, fast, traditional reconstruction techniques based on linear approximation, such as the linear back projection (LBP) or Landweber iterations, do not assure sufficient image quality [11,12].

Given these challenges, we explored the potential of deep neural network-based methods, which have shown promising results in recent years for solving nonlinear problems [13,14,15]. The essential advantage of once-trained neural networks is their ability to perform rapid reconstructions, making them a viable option for real-time applications [16,17,18]. In this study, we focused on evaluating a conditional generative adversarial network (cGAN) due to its increasingly recognized effectiveness in reconstructing high-quality images at a speed that is necessary for industrial flow monitoring and control [19,20,21].

This paper presents experimental verification of adversarial network image reconstruction in electrical capacitance tomography (ECT) of two-phase flows. The experiments were conducted numerically using a computer simulation of tomographic measurements. The numerically generated dataset for supervised learning was divided into training, testing, and validation parts. The validation of the cGAN reconstruction was also performed using real measurements of two-phase flow phantoms. For this purpose, the data for four phantoms mimicking typical flow patterns were acquired using a 16-electrode sensor.

The study aimed to verify whether image reconstruction based on an artificial neural network (cGAN) allows the reconstruction of acceptable ECT images and the detection of flow regions. The results were compared with those obtained using the Landweber algorithm and the simplified Levenberg–Marquardt (sLM) algorithm (without a sensitivity matrix update).

2. Materials and Methods

2.1. Numerical Simulation of the Training Dataset

In supervised learning, the training dataset consists of elements represented as ordered pairs of model inputs and their corresponding outputs. For a computed tomography model, each data pair includes an image of the tomographic cross-section of the object being tested and the tomographic projections recorded for that object. However, when addressing the inverse problem, the order of the data pairs is reversed. In electrical capacitance tomography, the input data comprises inter-electrode capacitance measurements, while the output data corresponds to the pixels representing the permittivity distribution image within the examined space.

A large training dataset was required due to the size of the inverse problem considered in ECT. Performing measurements for several thousand known objects does not seem to be achievable in a reasonable time. Only a numerical simulation allowed us to collect the required several thousand cases in the dataset. The numerical simulations were performed using the ECTsim toolbox for MATLAB R2024a [22,23]. The software developed by our team is available under an open-source license on GitHub platform owned by GitHub Inc. (San Francisco, CA, USA) [24]. ECTsim enables the highly efficient modeling of electrical tomography problems, simulating electric field distributions and sensitivity matrices, making it well-suited for preparing large datasets. The software includes basic image reconstruction methods and intuitive image display functions [24].

In ECTsim, numerical simulations are executed on a dense, non-uniform mesh, whereas reconstructions are carried out on a simplified mesh. In our study, the reconstructed image matrix had a resolution of 32 × 32 pixels. Since only the pixels within the sensor’s field of view were reconstructed, the total number of pixels was reduced to 608 for the chosen model. The number of reconstructed pixels in the image matrix corresponded to the number of output nodes of the neural network. The number of the input nodes of the neural network corresponded to the number of measured inter-electrode capacitances, which was equal to 120 for a 16-electrode sensor.

2.2. Training Dataset with the Two-Phase Flow Patterns

Determining the optimal number of samples for the training dataset is challenging, as there is no precise method to estimate the required dataset size. Nevertheless, our experiments indicate that a model with an input size of 120 and an output size of 608, when trained on a dataset of approximately 150,000 samples, yields satisfactory results.

Figure 1 illustrates the classes of the flow patterns. A single element of the training dataset consists of the image of the distribution of electric permittivity inside a cylindrical sensor and the matrix of the inter-electrode capacitances normalized between the minimum (empty sensor) and the maximum (sensor filled with a material with relative permittivity equal 3). The capacitance measurements are shown in Figure 1 in the form of a 2D color map. The 16 × 15 pixels map size corresponds to the number of exciting and sensing electrodes in the tomographic scan. During scanning, one of the electrodes acts as a voltage-forcing electrode, and the others (here, 15) act as current-measuring electrodes. In the tomographic scan, voltage forcing is carried out successively for each of the 16 electrodes. Since the order of electrodes in the capacitance measurement does not change the measured value, we have only 120 independent measurements for the 16-electrode sensors. In our experiments, the two measurements made with one pair of electrodes were averaged, giving a 120-element input vector.

In the experiments, the training dataset, which includes four flow patterns (75,000 elements), was extended by a class of random circles (75,000 elements), giving five classes of images in total. This approach ensures that the image patterns are most likely well-represented while preventing the model from overfitting these patterns. Since the imaging was limited to a two-phase flow, the electric permittivity values were restricted to two specific values (ε_r1 = 1 and ε_r2 = 3) corresponding to the gas and oil present in the flow. A randomly selected part (25%) of the training set was reserved as a testing subset used in the learning process.

2.3. ECT Sensor and Test Objects for Real Measurements

The EVT4 data acquisition system, developed by our team, has been used for real measurements [25]. The system uses the single-shot high-voltage (SSHV) technique to measure capacitances [26]. It achieves a high signal-to-noise ratio (SNR), ranging from 30 to 65 dB for capacitance measurements between 1 fF and 1 pF. The EVT4 supports a frame rate of 1000 with a 16-electrode sensor [27].

Figure 2 illustrates the view (2a) and the sketch (2b) of the 2D sensor used in our experiment. The sensor’s wall, made of PVC, features an inner diameter of 152 mm. Sixteen thin copper foil electrodes (200 µm) are mounted on the inner side of the sensor wall. These electrodes are insulated with a layer of PVC foil (80 µm) adhered to the inner surface. With the excellent SNR provided by the EVT4, it was possible to use electrodes with a height of only 50 mm, which is one-third of the sensor’s diameter, thereby achieving good resolution along the sensor’s Z-axis. The SNR values achieved in measurements conducted using the developed sensor are presented in Figure 3.

We designed four static test objects to simulate the cross-sections of the two-phase flows within a cylindrical pipeline. For this purpose, we used polypropylene (PP) pellets, which have an electrical permittivity of approximately 2.2, similar to that of crude oil. The images of these test objects are shown later in the Section 3.

2.4. Neural Network Architecture and Training Procedure

During our previous research [28,29], we found conditional generative adversarial networks (cGAN) with the U-Net [30] based generator being very promising in electrical impedance tomography image reconstruction. They provide satisfactory image quality and enough resistance to overtraining. The main component of the cGAN is the generator shown in Figure 4. The U-Net-like network structure implies having an image on the input and an image on the output. Because capacitance measurements in electrical tomography are represented by a 1D vector, it is necessary to add a linear layer with an input size corresponding to the number of measurements (120 in our case) and an output size corresponding to the total number of pixels in the image (1024). As the network layers are deterministic functions, they provide the same output for the given input. To successfully train the network, it is necessary to allow for output variability. In the traditional GAN approach, this is achieved by providing a Gaussian noise to the generator network input [31]. This works well in the case of a small number of measurements [32]; however, when increasing the size of the input vector and output image, the network requires an additional source of entropy to maintain its stability. We found that the Pix2Pix approach, originally proposed for large image translation [33], allows us to achieve satisfactory network stability and generalization ability. It uses dropout layers as a source of additional entropy. Such an approach allows us to eliminate latent vectors when using a sufficient number of dropout layers.

The U-Net network consists of encoder and decoder parts. In our case, the encoder part consists of five convolutional layers with Leaky ReLU (rectified linear unit) activation. All layers, except the first one, are also stacked with 2D batch normalization layers [10], dramatically decreasing the training time. The decoder comprises five deconvolutional layers with Leaky ReLU activation stacked with 2D batch normalization layers. The first two layers are also stacked with dropout layers with a factor of 0.2, meaning 20% of the pixels are randomly zeroed to provide enough randomness. At the output of the decoder we have a 32 × 32 × 32 tensor, and it is necessary to process this tensor into a 32 × 32 image using a convolutional layer.

The GAN approach for training an artificial neural network implies using a binary classifier, called a discriminator, aimed to distinguish the generated images from the ground-truth images. The conditional GAN method, shown in Figure 5, also requires adding a condition to the input of the binary classifier. In our case, we use a linear layer to combine the 1D measurement vector with the 2D image in the same way as it was done with the generator’s input. The traditional approach supposes using a binary cross-entropy as a loss function. However, it becomes difficult to prevent discriminator overtraining when the image size increases. It is possible to overcome this challenge by modifying a loss function using the already mentioned Pix2Pix approach. Modification is based on including the L1 and L2 components into the loss function, comparing the obtained image and a ground-truth image in the following way:

$$f\left(l_r,l_p,y_r,y_p\right)=\mathrm{B}\mathrm{C}\mathrm{R}\left(l_p,l_r\right)+100\mathrm{M}\mathrm{S}\mathrm{E}\left(y_r,y_p\right)+100\left|y_r-y_p\right|$$

(1)

where $l_r$ is the reference value of the probability that the image is true, and $l_p$ is the probability value of the image being true, determined by the discriminator. $y_r$ and $y_p$ are the reference and predicted images, respectively. BCR is a binary cross-entropy, and MSE is a mean square error.

The network was implemented using Python programming language version 3 with the PyTorch library version 2.4. The network was trained, and the results were analyzed using an additionally prepared validation dataset (10,000 elements). Tuning the generator and discriminator learning rates’ initial values and using a cosine annealing learning rate schedule allowed us to obtain discriminator and generator loss curves for 50 training cycles (epochs), as shown in Figure 6a,b. The mean relative image error was calculated on each epoch, as shown in Figure 6c. The total training time was 40 min using a Tesla P100 GPU with 16 Gb of memory (Nvidia corp., Santa Clara, CA, USA). However, it is important to mention that network training should be conducted only once for a given ECT sensor configuration and the expected set of materials inside the sensor.

2.5. Reference Image Reconstruction Algorithms

The results generated by the cGAN network were compared with those obtained using two iterative image reconstruction methods commonly used in electrical capacitance tomography:

I.

The Landweber method, which is given by the formula:

$${\mathsf{\epsilon }}_{(k+1)}={\mathsf{\epsilon }}_{\left(k\right)}-{\alpha }_{\left(k\right)}{\mathbf{S}}^T(\mathbf{S}{\mathsf{\epsilon }}_{\left(k\right)}-\mathbf{C}),$$

(2)

where $\alpha$ is the self-relaxing step length, $\mathbf{S}$ is the sensitivity matrix calculated for a uniform distribution inside the sensor, $\mathbf{C}$ is a vector of measured capacitances, and $k$ is the iteration number. A discrepancy norm, given by the formula

$${\mathbf{d}\mathbf{i}\mathbf{s}}_{\left(k\right)}={‖\mathbf{S}{\mathsf{\epsilon }}_{\left(k\right)}-\mathbf{C}‖}^2$$

(3)

is calculated in each step of the algorithm. If the error fails to decrease from one iteration to the next, the algorithm may reduce the step size to maintain progress. The calculations are stopped if the step length falls below the assumed minimal level.

II.

The simplified Levenberg–Marquardt (sLM) algorithm (also with a self-relaxing step length) is given by the formula:

$${\mathsf{\epsilon }}_{(k+1)}={\mathsf{\epsilon }}_{\left(k\right)}-{\alpha }_{\left(k\right)}{({\mathbf{S}}^T\mathbf{S}+\lambda \mathbf{I})}^{-1}{\mathbf{S}}^T(\mathbf{S}{\mathsf{\epsilon }}_{\left(k\right)}-\mathbf{C}),$$

(4)

where $\lambda$ is a regularization parameter that balances the trade-off between the solution’s stability and accuracy. To optimize the performance of the sLM method, we conducted several reconstructions on the entire validation dataset and determined that a $\lambda$ value of $5 {\times 10}^{-3}$ yielded the best results. The sensitivity matrix $\mathbf{S}$ was not updated to reduce computation time, making this a linearized version of the sLM method, which is more comparable in execution time to the neural network-based reconstruction. In the literature, this approach is also called iterative Tikhonov regularization [34]. The stopping criterion is consistent with that used in the Landweber method.

Validation was also conducted using real measurements. For this, a tomographic sensor was prepared, corresponding to the numerical model used in the simulations. Also, several test objects were designed to simulate a two-phase flow. These physical setups allowed us to compare the results obtained from the neural network and traditional methods, validating the effectiveness of our approach in practice.

3. Results

This experiment aimed to compare the reconstruction results obtained using the cGAN network with those from the Landweber method and sLM algorithm. The experiment consisted of two parts: in the first part, a quantitative comparison was performed on the additionally generated validation set, and in the second part, real measurements were conducted.

3.1. Analysis of the Simulated Dataset

The validation dataset consisted of two subsets, each containing 5000 elements: one with flow patterns and the other with random circles. The noise was added to the measurements to achieve 30 dB for the opposing electrodes, which reflects the SNR of our actual measurement system. Figure 7 shows the reference pattern and the images reconstructed using the Landweber method, sLM algorithm (λ = 5 × 10⁻³), and cGAN neural network.

The performance of the cGAN was evaluated using a commonly adopted and recognized approach for measuring the quality of ECT images [35]. This approach relies on straightforward yet objective pixel-by-pixel comparisons to gauge the difference between the reconstructed and original ground-truth images. The quality of the reconstructed images was assessed using several metrics: mean square error (MSE), peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and correlation coefficient. Each metric relates to a reference cross-sectional image, the basis for generating the dataset. The mean, median, and standard deviation of these norms were calculated for the validation dataset (

Table 1. The mean, median, and standard deviation of image quality metrics for the testing datasets, including MSE, SSIM, correlation, PSNR, and reconstruction time, are presented. The method with the best performance on each metric is highlighted in bold.

		Flow Patterns			Random Circles
		Lanweber	sLM	cGAN	Lanweber	sLM	cGAN
MSE	µ	0.762	0.859	0.034	0.372	0.381	0.018
	mdn	0.409	0.516	0.010	0.361	0.358	0.012
	σ	0.605	0.713	0.048	0.083	0.115	0.018
SSIM	µ	0.342	0.321	0.867	0.318	0.315	0.945
	mdn	0.320	0.298	0.879	0.315	0.312	0.951
	σ	0.087	0.099	0.075	0.055	0.056	0.027
correlation	µ	0.845	0.829	0.965	0.780	0.779	0.985
	mdn	0.862	0.858	0.994	0.788	0.787	0.990
	σ	0.063	0.077	0.064	0.056	0.057	0.017
PSNR	µ	2.524	2.166	20.36	4.388	4.345	19.21
	mdn	3.886	2.869	20.16	4.430	4.463	19.08
	σ	3.381	3.628	7.812	0.906	1.101	4.008
time [ms]	µ	3.1	5.3	2.7	3.1	5.8	2.7
	mdn	3.0	5.8	2.7	3.0	6.0	2.7
	σ	0.4	1.3	0.1	0.6	1.4	0.2

).

The average value helps us evaluate the overall reconstruction quality by quantifying the discrepancy between the real and reconstructed image datasets. However, this does not capture the issues related to specific elements, known as outliers. Therefore, the distributions of the MSE and correlation coefficient across the testing dataset were also analyzed and presented as histograms in Figure 8.

The distributions of the time required for reconstruction were also calculated and presented in Figure 9 for all three reconstruction methods.

3.2. Reconstruction from Real Measurements

For the real measurements, we used polypropylene (PP) pellets to create test objects designed to simulate a two-phase flow. The versatility of forming PP pellets into various shapes enabled us to construct detailed but static phantoms that could replicate the characteristics of two-phase flow patterns (annular, stratified, and slug) without needing an industrial setup. PVC pipes were used to shape the PP pellets into the desired forms. For measurements involving random circles, the test object was constructed using polyamide rods (${\epsilon }_r=4$) of varying diameters (10 mm, 20 mm, and 30 mm) placed vertically in the sensor. The results are shown in Figure 10. In addition to the photos of the object setup inside the sensor, we included their numerical models to illustrate the expected reconstruction results better. Real measurements were conducted using the EVT4 data acquisition system. Normalized capacitance values are presented as a two-dimensional set of tomographic projections (sinograms). We compared the images obtained using a neural network based on cGAN to those reconstructed using the Landweber method (with step size relaxation) and those obtained using the sLM algorithm (without a sensitivity matrix update). The results demonstrate the superiority of the cGAN-based neural network over the iterative algorithm.

4. Conclusions

Implementing ECT in industrial settings requires a rapid and computationally efficient reconstruction method. Our research indicates that this can be achieved using an artificial neural network. Through numerical simulations and real measurements, we have demonstrated that a cGAN-based method produces promising results. The reconstruction speed is comparable to iterative linear algorithms, and the quality of the reconstructions with the cGAN-based approach, evaluated using image distance metrics, is superior.

Utilizing neural networks like cGAN for image reconstruction necessitates the availability of large training datasets. In our study, we employed a dataset containing 150,000 elements. Generating such a large dataset through real-world measurements would have been impractical; thus, it was only achievable through precise numerical simulations. We used our custom software, ECTsim, which allows for a rapid and accurate simulation of measurements in electrical capacitance tomography. We have decided to make ECTsim available under an open-source license, enabling other researchers and practitioners interested in preparing large training datasets to benefit from our tool.

We were particularly focused on validating the effectiveness of our network (trained on the simulated dataset) in real-world measurements. To achieve this, we prepared a measurement sensor and various test objects. The measurements were carried out using our custom-built tomography system, EVT4, which can capture 1000 images with a 16-electrode sensor. The reconstructions were performed on individual measurements. The average time for reconstruction using the C++ cGAN neural network implementation was 2.7 ms on our laboratory computer, indicating that we could easily achieve up to 200 reconstructed images per second using this method. Furthermore, the quality of the reconstructions in case of real measurements was superior to that of the linear iterative algorithms with comparable computation times.