1. Introduction
During the 1980s, based on the computed tomography (CT) technique of medical images, researchers proposed electrical capacitance tomography (ECT) [
1]. Because of its low cost and accuracy, ECT has been widely used in industrial process monitoring in reactors, pipelines, and containers, and wherever non-conductive components of a dielectric nature can be used. Knowing the internal distribution of materials inside an industrial process container or pipe is essential in many applications. Tomography plays a very important role in several industrial fields. Typical examples of the use of this technology include the food industry, industrial tomography, biomedical processes [
2], gas–fluid flow [
3], chemical and pharmaceutical processes [
4,
5], and non-destructive evaluations of invisible objects in dams and flood embankments [
6].
The electrical capacitance tomography (ECT) can be defined as the use of electrodes to measure capacitance changes that are transformed into two-dimensional images as visual outputs using image reconstruction algorithms [
7]. Typically, the electrode numbers in the ECT sensor controls the number of independent capacitance measurements (usually 28 to 496) and the acquisition rate varies from a few up to several thousand images per second [
8]. Then, one or more high-performance PCs collaborating together, using mathematical models, can process the collected data and implement dedicated image reconstruction algorithms to make the appropriate diagnostic decision to effectively process control and automation [
7].
The ECT can be implemented both in real-time [
9] and offline mode [
10]. The choice of the image reconstruction algorithm has a crucial role in the ECT process since it has a direct impact on the image quality [
11]. The ECT image reconstruction process can be implemented through iterative algorithms, e.g., iterative Landweber method (ILM) [
12], Newton Raphson [
13], and Tikhonov regularization [
14], as it can also be implemented through non-iterative methods, such as linear back projection (LBP) [
15]. The speed and the simplicity of the non-iterative methods was not an argument for wide use because, in the same time, they suffer from deformations in the reconstructed images [
16]. In comparison, iterative methods can generate higher quality images. They are, however, computationally very expensive, thus, more useful for offline processing. The need for tools that can compromise the trade-off between high quality reconstructed images and computational efficiency, is currently the main interest of machine learning (ML) [
17,
18], more specifically, deep neural network (DNN) methods [
19]. DNN methods have been utilized in many fields due to their ability to map complex nonlinear functions [
20,
21]. DNN algorithms have been transferred and adapted such as in image reconstruction methods based on the convolutional neural network (CNN) [
22], multi-scale CNNs [
23], long short-term memory (LSTM) [
24], and autoencoder [
25]. To solve the forward problem and to estimate the capacity measures, Deabes et al. used a capacitance artificial neural network (CANN) system [
26,
27]. Thanks to its ability to effectively use specific geometric relationships hidden in commonly used unstructured grid models, the authors in [
28] proposed to use the graph convolutional network(s) (GCN), to increase the quality of the ECT image. Moreover, a long short-term memory image reconstruction (LSTM-IR) algorithm was implemented to map the capacitance measurements to accurate material distribution images [
24].
Generative adversarial networks (GANs) are very interesting techniques that have been recently developed in ML [
29,
30]. These networks allowed obtaining new results that were previously thought to be difficult to achieve: text to image generation [
31], text generation in different styles [
32], generation and defense against fake news [
33], conversion of sketches to images [
34], generation of photo-realistic images [
35], and even game designs learned by watching videos [
36]. The conditional generative adversarial network (CGAN) [
37], which is a particular version of the standard GAN, allowed better control over the output of generative adversarial models. Subsequently, this kind of GAN was applied in medicine to the CT of soft tissues [
38] as well as to tomography of the structure of materials with synchrotron radiation [
39,
40].
A novel post-processing adversarial resolution enhancement (ARE-ECT) model for ECT reconstructed image quality improvement is proposed in this paper. The proposed model is inspired by the deep learning networks for image super-resolution [
41,
42]. Principally, we assumed that a CGAN can be trained to enhance the reconstructed low-resolution ECT images from few capacitance measurements. Particularly, a CGAN is trained in generator and discriminator networks to produce high-resolution images from lower resolution reconstructions. As a result, when trained with pairs of ECT image reconstructions of a simulated phantom and a phantom itself, the CGAN model learns how to enhance the resolution of the inputs. Accordingly, the proposed adversarial model achieves better results than the recent complex, time-consuming non-linear ECT image reconstruction methods, and brings the reconstructed images closer to the phantom reference quality.
The contributions of this paper can thus be summarized as follows:
The adversarial resolution enhancement (ARE-ECT) model was developed in the problem of the ECT image reconstruction quality improvement.
The proposed model aimed to predict enhanced ECT image reconstructions from the lower quality ones.
Our CGAN-based approach produces qualitative and quantitative improved results in ECT image resolution better than current complex and time-consuming non-linear reconstruction algorithms.
The remainder of this paper is organized as follows:
Section 2 covers the ECT image construction problems.
Section 3 describes the DNN models, including GAN and CGAN.
Section 4 introduces a new ARE-ECT model to enhance the ECT image construction.
Section 5 describes the dataset used to train, test, and evaluate the proposed model.
Section 6 discusses the experimental results and the validity of the proposed model. Finally,
Section 7 presents our conclusions.
2. Problem Statement
The ECT problem is a typical image reconstruction problem. Particularly, given input data measurements, a higher resolution image is to be reconstructed. The input measurements could be any input data that are correlated to the reconstructed image. The modalities of the input data do not necessarily have to be the same of the output data. In the ECT problem, the input data are a few sensor reading numbers that are fed into the reconstruction algorithm as the input signal. The ECT sensor generates readings via a number of electrodes (
), which are evenly mounted around the imaging area.
Figure 1 illustrates the sensor setup. To capture the variations in the permittivity of the inner distribution, the mutual capacitance of each pair of these electrodes are measured independently [
43]. This pairwise measurement process results in a total number of capacitance measurements of
. To keep the uniformity in the electric field, decrease the external coupling, and eliminate any interference, the electrodes are separated by insulating guards [
44].
The distribution of the permittivity of the inner material within the area of interest affects the distribution of the electric field, which is defined according to the Poisson linear partial differential equation, as shown in Equation (
1).
where
is the distribution of permittivity,
is the potential distribution, and
denotes the charge distribution.
The mutual capacitance between electrode pairs is given by Equation (
2).
where
identifies the mutual capacitance between two electrodes
u and
v,
denotes the charge on the sensing electrode, which is defined according to the Gaussian law,
denotes the potential difference,
represents a closed path embracing a detection electrode, and
stands for a unit vector normal to
.
The ECT image reconstruction involves solving two types of problems: the forward and inverse. The forward problem refers to the numerical computation of the capacitance measurements from the sensor reading, according to Equation (
3):
where
C is the calculated capacitance,
S is the sensitivity matrix,
N = 16,384 is the number of image pixels, and
G is the permittivity distribution. The sensitivity matrix is the Jacobian of the capacitance with respect to pixels evaluated at
.
The ECT inverse problem refers to estimating the permittivity distribution,
G, given the capacitance measurement,
C, and the sensitivity matrix,
S. A non-iterative solution can be obtained directly from Equation (
3) using non-iterative algorithms, e.g., LBP, as shown in Equation (
4).
However, the obtained images using such a paradigm suffer from poor quality. This shortcoming could be dealt with using iterative algorithms, e.g., the Landweber algorithm (LW), as shown in Equation (
5).
where
is the relaxation parameter,
is the forward problem solution, and
k is the iteration number. However, despite the significant improvement achieved in the reconstructed images quality, it comes with high computational costs.
4. ARE-ECT Model
As explained in
Section 2, the main objective of the ECT image reconstruction problem is to generate a high quality permittivity distribution image, given a lower resolution distribution input image. Therefore, the first step of the proposed ARE-ECT model is to prepare the input image for the generator operation. This preparation is performed in a preprocessing phase, as shown in
Figure 2. The input to this preprocessing phase is the capacitance reading set. The ECT capacitance sensor produces a
raw vector data, i.e.,
. Afterwords, the input image is generated using traditional LW of Equation (
5) with
. The initial image of the permittivity distribution is provided by some fast matrix multiplication. The input image resulted from the preprocessing phase is fed to a generator. This generator could be a traditional autoencoder. However, although autoencoders are capable of reconstructing such patterns, the spatial information of the input signals are not modeled with sufficient accuracy. Given that the spatial information of the inner distributions is essential for the reconstruction of the flow pattern image, another generator that can preserve such spatial representation is mandatory. UNet is a good candidate to satisfy this requirement [
48]. Therefore, we adopted UNet to construct the flow pattern in the generator module.
Figure 3 illustrates the details of the used UNet in ARE-ECT. Four blocks were used on the encoder side, and similarly, four blocks were placed on the decoder side. The latent vector size was eight. The input layer’s low resolution image, generated by the preprocessing phase, was concatenated with the generated image by the final layer. Similarly, each input of the hidden layers on the decoder side was concatenated with the output of the corresponding layer from the encoder side.
The UNet generator module produces a flow pattern, which is considered a fake sample for the discriminator training. A synthetic data generator was developed to generate real samples,
, for the purposes of discriminator training. As shown in
Figure 2, the architecture of our UNet generator was designed with two sections: down- and upsampling. The main idea of UNet is to map a low resolution input image at a size of
to a 1-D vector and then reconstruct it back to a high quality image. The contraction of the downsampling (encoder) applies a
convolutional layer, batch normalization, and Relu activation followed by a
max pooling in each step. This stage generates a downsized image of a size equal to
with 128 features, and it continues to the latent vector size of
with 1024 features. The layers at the decoder (upsampling) section employ a
upsampling layer after convolution. During the upsampling process, the corresponding feature maps from the downsampling part are reused to reduce the distortion of images. They are appended directly after the upsample layer. The proposed model is designed for a 12-electrode ECT sensor setup. If any change in this setup, in terms of the number of sensors occurs, a new dataset must be generated. Therefore, every generated dataset is valid only for its underlying hardware configuration. This is because the resolution of the initially generated low-resolution images varies with the number of installed sensors.
5. ECT Dataset
We implemented a MATLAB GUI software package to build different configurations of ECT sensors. Various flow patterns can be simulated and their forward problems can be solved to generate the corresponding capacitance measurements. An extensive ECT benchmark dataset was developed for training and testing of the proposed ARE-ECT. A traditional image reconstruction algorithm was used to reconstruct the permittivity distributions, which used the initial image
x for the deep learning ARE-ECT model. In this paper, we used the LW algorithm as the inversion algorithm to generate the initial input image. The dataset consisted of 320 k samples, each one was a pair of an actual permittivity distribution vector as a ground truth, and the reconstructed image of the LW algorithm corresponding to each capacitance measurement vector. The sizes of the actual distribution, and the LW reconstructed image were
= 16,384. The ECT sensor was composed of 12 electrodes as shown in
Figure 1. The sensor pipe was made from PVC material with a relative permittivity of 2. The diameter and the thickness of the pipe was 100 and 2 mm, respectively. The electrodes were separated by gaps of 4 degrees, and the span angle of each electrode was 26 degrees. The dataset contained five different flow patterns, 10 k ring patterns, annular with 20 k patterns, 10 k stratified patterns, 1–3 circular bars with 140 k patterns, and 140 k patterns of 1–3 square bars.
Figure 4 shows some samples of various flow patterns from the generated ECT dataset. The low phase was air with a relative permittivity value equal to 1, and the relative permittivity of the high phase glass was
. Random variables were used in building the dataset. For instance, a uniform random variable with a range of
to
of the imaging area’s radius was applied to the ring’s width of the annular flow. The stratified flow height was assigned to a uniform random variable in a range of 5–95% of the diameter of the sensing field. The number of circular and square bars varied from 1 to 3. The generated data have some discrepancies in the number of instances within each type to reflect varying degrees of randomness. Additionally, every flow pattern had a different number of attributes that determined its geometric specifications. For instance, the attributes that characterized a ring flow pattern were just two—the inner and outer radii, while those of the square bar patterns were the number of bars, their lengths, widths, and planner locations. This large attribute dimensionality variation implies consequent large variations in the number of generated instances that represented the input data space.
7. Conclusions
In this paper, a new ARE-ECT model based on the CGAN deep neural network was proposed to enhance the resolution of the ECT reconstructed images. The generator was built using UNet. For evaluation purposes, a big dataset was developed. It contained simulation data of 320 k capacitance measurements–flow image pairs for training, validating, and testing. For generalization and feasibility of ARE-ECT, data instances, to which the model was not exposed during the training phase, were included in the evaluation dataset. The experimental results proved the superiority of the proposed ARE-ECT over the state-of-the-art, both quantitatively and qualitatively. Efficiency evaluation results showed that ARE-ECT succeeded in beating existing high-quality methods in terms of execution speed by ’several tens of times’, particularly from 28× to 135×. Briefly, ARE-ECT achieved better performance than the computationally-expensive methods, yet with the same execution time order of the low-resolution reconstruction method, e.g., the well-known LBP. In terms of the overall generalization, the ARE-ECT exhibited good capabilities. Hopefully, the work presented herein will inspire researchers in the ECT field to further investigate other deep learning-based approaches to reconstruct the flow patterns in the sensing field of the multi-phase flow.