Study on Quality Assessment Methods for Enhanced Resolution Graph-Based Reconstructed Images in 3D Capacitance Tomography

Abstract

This paper proposes a novel approach to assessing the quality of 3D Electrical Capacitance Tomography (ECT) images. Such images are typically represented as irregular graphs. Thus, image quality metrics typically used with raster images do not straightforwardly apply to them. However, given the recent advancements in Graph Convolutional Neural Networks (GCNs) for improving ECT image reconstruction, reliable Quality Assessment methods are essential for comparing the performance of different GCN models. To address this need, this paper applied some existing image quality and similarity assessment methods designed for raster images to the graph-based representation of 3D ECT images. Specifically, attention was paid to the Peak Signal-to-Noise Ratio (PSNR), the Structural Similarity Index Measure (SSIM), and measures based on image histograms. The proposed adaptations resulted in the development of tailored Graph Quality Assessment (GQA) techniques specifically designed for the graph-based nature of ECT images. The proposed GQA techniques were validated on 1042 phantoms and their corresponding Low-Quality (LQ) and High-Quality (HQ) reconstructions through a robust GQA benchmarking system, enabling a systematic comparison of various GQA methods. The evaluation of the proposed methods’ performances across this diverse dataset, by analyzing overall trends and specific case studies, is presented and discussed. Finally, we present our conclusions regarding the effectiveness of the proposed GQA methods, and we identify the most promising approach for assessing the quality of graph-based ECT images.

1. Introduction

Industrial processes often hide crucial information within closed vessels and pipelines. In such scenarios, Electrical Capacitance Tomography (ECT) offers a non-invasive window into these hidden realities, providing a non-invasive means to visualize and analyze processes hidden in pipes. Unlike X-rays and other radiation-based techniques, ECT utilizes electrical capacitance measurements to reconstruct images, enabling real-time monitoring and control of vital industrial operations and processes. While ECT has primarily been applied in industrial settings, recent studies have also investigated its potential for medical imaging applications [1,2].

Recently, Graph Neural Networks (GCNs) have offered a promising approach to enhancing the spatial resolution of 3D ECT image reconstruction. GNNs are a powerful class of neural networks designed to process data represented as graphs. Unlike traditional neural networks that operate on grid-like data (like images or text), GNNs can handle complex non-Euclidean data structures where relationships between data points are represented as nodes and edges. In the context of ECT image reconstruction, GNNs offer a novel approach to enhancing reconstruction quality. By representing the 3D finite element grid as a graph, where nodes correspond to grid vertices and edges encode spatial relationships, GCNs can effectively learn complex patterns within the data. This graph-based representation allows GCNs to capture intricate relationships between neighboring grid points and their associated electrical permittivity values. By leveraging this information, GCNs can refine the results of advanced non-linear reconstruction algorithms, such as those presented in [3], leading to more accurate and detailed ECT images.

However, the recent integration of Graph Convolutional Networks into the reconstruction process of ECT images presents a new challenge, i.e., reconstructed image Quality Assessment. Traditional image quality metrics, designed for pixel-based data, struggle to effectively evaluate the reconstructed images represented as graphs.

As highlighted in the seminal work by Zhou Wang and Alan C. Bovik [4], the need for objective image Quality Assessment methods has become paramount with the explosion of digital images and videos. This paper builds upon the established knowledge, acknowledging the limitations of existing methods when applied to the unique context of graph-based images in 3D ECT.

The fundamental difference lies in the nature of the data. Classical image Quality Assessment thrives on the well-defined structure of raster images, where pixels form a predictable two-dimensional grid. In contrast, the structure of a graph-based image is far less rigid. Nodes, analogous to pixels, connect irregularly, making neighborhood relationships more complex and unpredictable. Another significant challenge arises from the shifting object problem. In 3D ECT scans, the reconstructed object might retain its shape across two images, but its position might differ slightly. This scenario renders direct comparisons between corresponding nodes ineffective. Even with perfect shape preservation, metrics like Mean Squared Error or Pearson Correlation Coefficient, which are simple but still prevalent in the field of ECT, can be significantly skewed by the object’s shift. This highlights the need for a Quality Assessment method that remains insensitive to such positional variations.

Furthermore, a robust metric is required to effectively compare different Graph Quality Assessment (GQA) methods. Ideally, this metric would involve comparing a Low-Quality (LQ) image, a High-Quality (HQ) image, and a reference phantom image (PH). A successful GQA method should consistently identify the higher degree of similarity between the HQ image and the PH image compared to the LQ and PH comparison. This approach allows for evaluating the GQA method’s ability to accurately discern and quantify image quality differences.

Therefore, this work proposes novel approaches specifically designed to address the Quality Assessment of graph-based ECT reconstructions. We leveraged existing QA methods while introducing new ideas to define several GQA methods. Simple methods that compare corresponding node values, such as Mean Squared Error (MSE), are straightforward to adapt because graph-based images are always compared with the same structure, differing only in node values. Thus, graph-based MSE simply compares the values of corresponding nodes in the compared images. A greater challenge arises in adapting methods like the Structural Similarity Index Measure (SSIM), which relies on local pixel structures defined as fixed-size rectangles used to partition the image into sub-regions. Since graphs lack a regular structure, a different approach is necessary.

This paper’s approaches aim to overcome the traditional methods’ limitations, as mentioned above, and to provide a robust framework for evaluating the fidelity of reconstructed images in this emerging domain of graph-based ECT. Specifically, we propose to adopt some existing image quality and similarity assessment methods designed for raster images to the graph-based representation of 3D ECT images.

The remainder of this paper is structured as follows. First, Section 2 provides a comprehensive overview of the foundations of Electrical Capacitance Tomography (ECT), including image reconstruction techniques and traditional methods for assessing ECT image quality. Then follows Section 3, which summarizes our experimental data and introduces our novel ECT image Quality Assessment methods. Section 4 presents and analyzes the experimental results demonstrating the performance of our proposed metrics in both High- and Low-quality reconstructed images. Section 5 offers a detailed discussion of the experimental findings. Finally, Section 6 concludes the paper with a summary of our key contributions and future research directions. The overall structure and logic of the paper can be seen in Figure 1:

2. Background

2.1. Electrical Capacitance Tomography Fundamentals

Electrical Capacitance Tomography (ECT) is a non-intrusive and non-invasive imaging technique used to monitor industrial processes in pipelines and reactors, particularly where mixtures of non-conductive dielectric materials are involved. Common applications include monitoring gas–oil flows, solid particle flows, and reservoirs [5]. An ECT system typically measures changes in mutual capacitance between pairs of electrodes arranged around the circumference of a process container or pipe. The amount of measurement data, usually ranging from 66 to 496 data points, depends on the hardware used, and the acquisition rate can vary from a few to hundreds of images per second. The collected data are then processed by a high-performance PC system using mathematical modeling and specialized algorithms for image reconstruction, raw data analysis, and making effective and robust diagnostic decisions for process control and automation [6]. Industrial processes generally exhibit a three-dimensional nature, making it necessary to sense and measure phenomena in the 3D space occupied by these processes [7,8]. Consequently, ECT is evolving from Z-axis averaged measurements recorded in a cross-sectional plane to 3D scanning of the entire volume. Conventional (cross-sectional, planar) ECT systems utilize a single layer of evenly spaced electrodes to exclusively capture cross-sectional data. This limitation restricts image reconstruction to a two-dimensional space, averaging static or dynamic data along the Z-axis. Conversely, 3D ECT sensors extend the capabilities of planar cross-sectional imaging by using multiplane electrode arrangement and enabling full volumetric reconstruction of material distribution within the entire sensing space. This advancement provides a more comprehensive and detailed understanding of both static and dynamic objects under examination. The 3D ECT sensor and sensing system concept, with the main components, are illustrated in Figure 2.

2.2. 3D ECT Image Reconstruction

3D Electrical Capacitance Tomography reconstruction models generally exploit the relationship between the spatial distribution of the electric field within a capacitance sensor and the spatial distribution of the electric permittivity within the sensor volume, to visualize the contents of a pipe. This concept is illustrated in Figure 3:

The spatial distribution of an ECT tomography electric field can also be defined from a mathematical and physical perspective. When spatial electric charge is negligible, the electric field behavior in a 3D ECT sensor is governed by the Laplace equation (see Equation (1)):

$$\nabla [ϵ(x,y,z)\nabla \varphi (x,y,z)]=0x,y,z\in \mathrm{\Omega }$$

(1)

In this context, $ϵ(x,y,z)$ signifies the electric permittivity distribution in the volume $\mathrm{\Omega }$, while $\varphi (x,y,z)$ represents the electric potential distribution in space. The coordinates $x,y$, and z precisely locate a specific simulated point within the discrete space of the 3D ECT sensor. Knowing the electric potential $\varphi$ at a given point $(x,y,z)$, the induced charge $Q_{E_g}$ on a grounded electrode g can be determined, using Gauss’s law (see Equation (2)):

$$Q_{E_g}={\int }_{E_g}ϵ{\displaystyle \frac{\delta \varphi (x,y,z)}{\delta \mathrm{n}}}{\mathrm{d}}^{3}x,$$

(2)

where $\mathrm{n}$ is the unit normal vector to the electrode surface. The initial boundary conditions ${\varphi }_0$ must be specified for all electrodes. At any given time, a single electrode, ($E_e$), is designated as the excited electrode and a positive electric potential, (i.e., ${\varphi }_0={\varphi }_e$), is applied to it. Concurrently, the remaining electrodes are grounded (i.e., ${\varphi }_0=0$). The capacitance $C_{eg}$ between the electrode pair $E_e$ and $E_g$ can be defined as follows:

$$C_{eg}={\displaystyle \frac{{\int \int \int }_{\mathrm{\Omega }}ϵ(x,y,z)\nabla \varphi (x,y,x)\mathrm{d}\mathrm{\Omega }}{{\varphi }_e-{\varphi }_g}}$$

(3)

where ${\varphi }_e-{\varphi }_g$ denotes the electrical voltage difference between the excited electrode $E_e$ and the grounded electrode $E_g$. As 3D ECT tomography is a relative, contrast-based tomography method, the capacitance data for a given object, ${ϵ}_{obj}$, must be normalized to obtain $C_m$ by using pre-measurements for two electric permittivity distributions, ${ϵ}_{high}$ and ${ϵ}_{low}$, that significantly differ in dielectric constant. To calculate the normalized capacitance data $C_m$, the following formula can be used:

$$C_m={\displaystyle \frac{C\left({ϵ}_{obj}\right)-C\left({ϵ}_{low}\right)}{C\left({ϵ}_{high}\right)-C\left({ϵ}_{low}\right)}},$$

(4)

In the simplest scenario, image reconstruction for a 3D ECT system can be viewed as an inverse problem that necessitates the estimation of the approximate spatial distribution of electric permittivity, $G_{3D}$, inside an ECT sensor based on normalized capacitance measurements, $C_m$, according to formula [5],

$$G_{3D}=F\left(C_m\right)$$

(5)

where F can be defined as a direct transformation function or an iterative optimization function.

For 3D ECT systems, various representations of Jacobian matrices, which serve as sensitivity models, can be utilized [9]. The internal structure of the Jacobian matrix consists of M weight vectors that operate as sensitivity maps. Each map determines the sensitivity of a given point $j(x,y,z)$ (from a set of N points—typically 3D grid nodes or elements) in the measurement space to a non-linear change in capacitance resulting from a perturbation in permittivity at that point. The Jacobian matrix is used to simplify and linearize the inherently non-linear image reconstruction process. The sensitivity map $J_j$ for a given 3D point $j(x,y,z)$ can be computed using the electrical field response to changes in capacitance C (and electric potential $\varphi$), according to the following formula:

$$J_j=-{\int }_{A_j}{\displaystyle \frac{E_{e_1}}{{\varphi }_{e_1}}}{\displaystyle \frac{E_{e_2}}{{\varphi }_{e_2}}}\mathrm{d}j$$

(6)

where $E_{e_1}$ and $E_{e_2}$ are the electrical field intensity values at point j, while the electrodes $e_1$ and $e_2$ store potentials ${\varphi }_{e_1}$ and ${\varphi }_{e_3}$, respectively, and $A_j$ denotes the volume of point j in the imaging region.

Calculating sensitivity across all 3D grid points or elements for all unique measurement electrode combinations yields a sensitivity matrix, J. Directly employing matrix J in the image reconstruction process is mathematically problematic, due to its non-rectangular nature. Additionally, the number of unknowns, N, corresponding to image elements is typically significantly larger than the number of equations, M (the number of capacitance measurements). As a result, multiple solutions are anticipated, leading to various reconstructed image quality issues, such as artifacts, blurred regions, limited axial or spatial resolution, heterogeneity, and poor close objects separation capabilities, especially in 3D mode. To address these numerical challenges, numerous research efforts have focused on optimizing the reconstruction process and minimizing the final image errors during image construction [10]. One straightforward solution involves replacing the non-invertible matrix $J^{-1}$ with its transposed approximation, $J^T$. Equation (5) with Equation (6) can then be reformulated in a discrete matrix form:

$$G_{3D}=J^T{C}_m.$$

(7)

To address the inverse nature of the ECT reconstruction process defined by Equation (7), numerous algorithms have been developed to mitigate the ill-posed and ill-conditioned nature of this problem in Electrical Capacitance Tomography. These algorithms can be broadly classified into three main groups: one-step, fast-but-less-precise image reconstruction algorithms, statistical and stochastic methods, and accurate-but-computationally expensive iterative algorithms.

The one-step method family includes Linear Back Projection (LBP), Singular Value Decomposition, Tikhonov Regularization, Multiple Linear Regression with Regularization, and other algorithms investigated by various researchers [6,10,11]. One-step approaches are known to significantly limit 3D ECT imaging, resulting in low spatial and axial resolution, numerous image artifacts, and poor image quality in specific regions of a 3D ECT scan. However, these limitations can sometimes be acceptable for specific applications, such as emergency state detection or tracking fast-moving objects with high permittivity. Linear direct reconstruction methods offer near-real-time computation using modern PC hardware, such as GPU CUDA parallel computing [12], which is often required in industrial applications. Statistical and stochastic methods, including Bayesian inference, Monte Carlo Markov chain sampling, and the Metropolis–Hastings algorithm, have also been developed to improve the ECT imaging process [13,14].

Another group of reconstruction algorithms involves iterative optimization methods such as the Gauss–Newton algorithm [15], Landweber iterations, the Simultaneous Iterative Reconstruction Technique (SIRT) [6], and non-linear (FEM)-based algorithms. These methods are especially crucial for the research presented here. They generate a finite element grid where the nodes represent the electric permittivity at the corresponding spatial locations within the pipe. This finite element grid enables visualization of the pipe’s contents, using specialized tools.

Non-linear inversion approaches, driven by numerical algorithms and a comprehensive numerical sensor model, significantly enhance the efficiency of 3D ECT inversion [9]. Known for their superior quality, these methods excel in both 2D and 3D ECT image reconstruction. They involve an iterative non-linear reconstruction loop, where the numerical forward problem and the 3D sensitivity map are optimized and converged through incremental adjustments to the image’s dielectric properties distribution. These techniques commonly integrate Finite or Boundary Element Methods, employing precise 3D grid discretization coupled with numerical simulations of ECT sensor structures [9].

The non-linear reconstruction method with a complete 3D ECT sensor model was the starting point for the enhancement of the 3D image resolution of GCN-based ECT [3], as it has been successfully validated for high-quality 3D ECT image reconstructions of challenging dielectric objects in the past [9]. This algorithm defines a process for finding an approximate 3D inverse solution using a fully non-linear scheme, a complete simulated 3D ECT sensor model, and direct FEM-based iterative back-projection image optimization. The flowchart below (Figure 4) shows the main blocks and data flow in this method:

Artificial Neural Networks (ANNs) have recently been used to improve the reconstruction of ECT images and improve the 3D ECT image quality. Most applications of ANNs in Electrical Capacitance Tomography involve integration with direct or iterative image reconstruction methods. These algorithms have been adapted to refine the electrical tomography image reconstruction process and to enable faster extraction of industrial process parameters. Examples include auto-encoder-based image reconstruction methods [16], deep learning [17], Convolutional Neural Networks (CNNs) [18], multiscale CNNs [19], DBAR methods [20], LSTM methods and their optimizations [21,22], and Graph Convolutional Networks (GCNs) [3].

As Graph Convolutional Networks (GCNs) become increasingly prevalent in 3D ECT image reconstruction, the need for reliable image quality metrics has become increasingly apparent. This paper seeks to establish a new benchmark for evaluating the fidelity of GNN-based 3D ECT images.

2.3. Existing Quality Assessment Methods

The problem of assessing image reconstruction quality in Electrical Capacitance Tomography (ECT) has received little attention. The researchers commonly use Mean Squared Error (MSE), Peak Signal-to-Noise Ratio (PSNR), and the Correlation Coefficient (CC) to evaluate the accuracy and reliability of the reconstructed images [3,23].

Other less frequent measures include the relative Image Error (IE) between ground truths and reconstructed permittivity distributions [24] or the entropy of the reconstructed image [25].

While PSNR, MSE, the Correlation Coefficient (CC), entropy, and relative error are valuable metrics for evaluating ECT image quality, they have several limitations that must be carefully considered. Unlike some other metrics, PSNR, MSE, and entropy are not normalized, which can hinder cross-dataset comparisons and potentially lead to misleading interpretations. Furthermore, entropy calculations are sensitive to small changes in the reconstructed image, potentially resulting in disproportionately large changes in entropy values, due to their non-linear nature. Similarly, while relative Image Error is normalized, variations in signal strength across different ECT imaging conditions can introduce bias in the assessment. This sensitivity can complicate the interpretation of results, and it may not accurately reflect meaningful differences in quality. Additionally, inconsistencies in calculation methods or reporting practices can introduce variability in results across studies. Moreover, PSNR, MSE, CC, entropy, and relative error may not always align with human visual perception. Images with high PSNR or CC values might still exhibit significant artifacts that are not adequately reflected by these metrics, potentially overstating the reconstruction quality. Ultimately, these metrics offer numerical values but lack contextual insights into the structural integrity or functional relevance of the reconstructed ECT images. They often fail to capture the structural information present in ECT images that extends beyond pixel-wise differences.

While traditional image Quality Assessment metrics considering structural information tailored for raster images, such as the Structural Similarity Index Measure (SSIM), are not used for ECT image reconstruction evaluation, due to their reliance on pixel-based comparisons, our research aimed to explore the potential of graph-based methods for addressing these challenges. By leveraging the graph representation of ECT data and considering structural similarities between nodes and edges, these methods can provide more accurate and informative assessments of reconstruction quality. The SSIM, a metric designed to capture perceptual similarities between images, has demonstrated effectiveness in various image-processing applications. By adapting the SSIM to graph-based representation of ECT images and incorporating node neighborhoods via node grouping into PSNR and MSE calculations, we aim to address the structural information in ECT images, leading to more meaningful and accurate quality assessments

Finally, the SSIM is less sensitive to image shifts than PSNR or MSE, due to its focus on structural similarity. The SSIM’s local analysis, which considers luminance, contrast, and structural information, is more robust against small shifts than global metrics. By mimicking the SSIM’s local analysis through node grouping, our proposed Group SSIM and Group PSNR metrics are also resilient against image shifts.

3. Materials and Methods

3.1. Input Data

3.1.1. 3D ECT Image Representation

The 3D ECT image constitutes a finite element grid represented as an undirected graph $\mathcal{G}=(\mathcal{V},\mathcal{E},\mathbf{U})$, where $\mathcal{V}$ is a vertex set, $E\subseteq \mathcal{V}\times \mathcal{V}$ is an edge set, and $\mathbf{U}$ is an adjacency matrix. Nodes $v_i\in \mathcal{V}$ represent 3D grid vertices, and they store a feature representing electric permittivity in the corresponding spatial location. All nodes in a graph $\mathcal{G}$ compose a vector $\mathbf{v}=(v_1,v_2,\dots ,v_N)$ of size $N\times 1$, where $N=\left|\mathcal{V}\right|$ is the cardinality of the vertex set. Edges $e_{ij}=(v_i,v_j)\in \mathcal{E}$, where $i\ne j$ connects neighboring nodes $v_i$ and $v_j$, with all $M=\left|\mathcal{E}\right|$ connections in the graph described by an adjacency matrix $\mathbf{U}$ with d-dimensional normalized entries $\mathbf{u}(i,j)\in {[0,1]}^d$ if $e_{ij}\in \mathcal{E}$, and 0 otherwise. This idea is sketched in Figure 5:

3.1.2. 3D ECT Image Dataset

A dataset of 1042 3D ECT data samples was considered in this study. Each data sample consisted of a phantom and corresponding phantom 3D ECT reconstructions obtained for 4 × 4 and 4 × 8 electrode sets. The phantoms were generated automatically and represented four equally distributed types of object shapes, namely, balls, rods, and H-shape, and L-shape. The parameters of the shapes, including size, number, position, and orientation, were selected randomly (cf.

Table 1. Summary of data used in this study.

Shape	Number	Random Parameters
multiple balls	259	number (2–5), radius, position
H-shape	262	size, position, orientation
L-shape	262	size, position, orientation
rod	259	number (1–5), diameter, length, position

). Understanding the interplay between the object parameters for a 3D ECT image dataset and non-uniform electric fields in 3D ECT models is crucial for image quality evaluation. There are also 3D ECT sensor model specific regions within the sensing volume, where changes in object properties have a minimal effect on the measured capacitance. These zones are due to factors such as non-uniform electric fields, electrode geometry, and object properties. The presence of sensitivity dead zones can limit the accuracy of object reconstruction in certain regions and can influence the Quality Assessment. The size of an object significantly affects the magnitude and distribution of the electric field and dead zones as well. Larger objects tend to induce greater capacitance changes, while smaller objects may have limited impact. Multiple objects within the sensing volume can interact and influence each other’s electric fields, leading to complex capacitance patterns. The relative positions of the objects within the sensing volume determine the spatial distribution of the electric field. Objects closer to the electrodes will generally have a stronger influence on the measured capacitance. The orientation of objects can affect the direction and magnitude of capacitance changes. For example, a long, thin object oriented parallel to the electric field lines will induce a different capacitance change than one oriented perpendicularly.

The non-linear reconstruction method (NLECTCM) was applied for both electrode sets, to reconstruct the phantom data. In the phantom data, nodes representing an object were assigned an electric permittivity of 3, while nodes representing air stored an electric permittivity of 1.

The FEM-based graphs representing phantom data ${\mathcal{G}}_p$ and the corresponding 3D ECT reconstructions obtained with different electrode sets (i.e., ${\mathcal{G}}_{4\times 4}^{\prime }$ and ${\mathcal{G}}_{4\times 8}^{\prime }$) exhibited the same structure with N = 25,350 vertices and M = 67,974 edges. Thus, each phantom in the dataset had two reconstructions, a High-Quality (HQ) one ${\mathcal{G}}_{4\times 8}^{\prime }$, and a Low-Quality (LQ) one ${\mathcal{G}}_{4\times 4}^{\prime }$. This variation was necessary, to assess the relationship between image quality and the similarity calculated by GQA methods between the image and its corresponding phantom. Sample Low-Quality and High-Quality reconstructions of the H-shape phantom are shown in Figure 6.

3.2. The Proposed ECT Image Quality Assessment Methods

3.2.1. The Main Idea

To bridge the gap between established raster image Quality Assessment (QA) methods and the unique challenges of graph-based 3D ECT, we introduce node grouping as a novel approach to addressing local node structure issues. This method creates a group of nodes for each node in the graph based on a user-defined depth parameter. The rationale behind node grouping is that each node in the graph has a set of neighboring nodes contributing to its overall information. For a given node, a group with depth 1 includes its immediate neighbors. Increasing the depth to 2 incorporates the immediate neighbors and their respective neighbors, capturing a broader local context.

The schema of node grouping is illustrated in Figure 7, where “1” indicates a node group at depth 1 and “2” represents a node group at depth 2:

3.2.2. Group MSE and Group PSNR

Group Mean Square Error (GMSE) and Group Peak Signal-to-Noise Ratio (GPSNR) address the object-shifting problem via leveraging node grouping, thus focusing on group-level comparisons instead of individual node values. We first determine an average value for each node group (depth 1 or 2, as defined above). Then, we calculate the MSE or the PSNR for these average group values, treating them as pixel values in a traditional image.

Both the GPSNR and the GMSE rely on node grouping. When determining the Group PSNR for each node, an average value is calculated from its assigned group (similar to calculating the mean for a pixel neighborhood). The PSNR is then computed for these average group values, using a maximum node value of 3. In the case of the Group MSE, the squared differences between the corresponding average group values (from reference and other images) are calculated and averaged.

3.2.3. Graph Structural Similarity Index

The Structural Similarity Index Measure (SSIM) [26] is a perceptual metric that measures the similarity between two images. It is designed to more accurately predict perceived image quality compared to traditional metrics like PSNR and MSE.

The SSIM incorporates three factors: (i) luminance, i.e, the difference in brightness between the two images; (ii) contrast, i.e., the difference in contrast between the two images; and (iii) structure, i.e., the similarity in the patterns and textures of the two images. It can be determined from the following formula:

$$SSIM(x,y)={\left[l(x,y)\right]}^{\alpha }\ast {\left[c(x,y)\right]}^{\beta }\ast {\left[s(x,y)\right]}^{\gamma }$$

(8)

where $l(x,y)$, $c(x,y)$, and $s(x,y)$ are luminance, contrast, and structural similarity comparison functions given by Equations (9)–(11), respectively:

$$l(x,y)=(2\ast {\mu }_x\ast {\mu }_y+C_1)/({\mu }_x^2+{\mu }_y^2+C_1)$$

(9)

$$c(x,y)=(2\ast {\sigma }_x\ast {\sigma }_y+C_2)/({\sigma }_x^2+{\sigma }_y^2+C_2)$$

(10)

$$s(x,y)=({\sigma }_{xy}+C_3)/({\sigma }_x\ast {\sigma }_y+C_3)$$

(11)

where ${\mu }_x$ and ${\mu }_y$ are the average luminance, ${\sigma }_x^2$ and ${\sigma }_y^2$ are the variances, and ${\sigma }_{xy}$ is the covariance of images x and y, respectively. Typically, $\alpha =\beta =\gamma =1$ and $C_1$, $C_2$, and $C_3$ are small constants that stabilize the division with small denominators.

The Graph Structural Similarity Index Measure (G-SSIM) adapts the original SSIM for graph-based images by leveraging node grouping to capture local node structures. The G-SSIM calculates local structural similarities for each node group in the reference and target images. Local node values refer to each group’s average value and standard deviation (contrast).

The overall G-SSIM score is the mean of all the local structural similarity results for all the node groups. The local structural similarity is calculated based on the simplified SSIM equation (Equation (12)), where luminance is the average node value (i.e., the electrical permittivity), and contrast is the standard deviation within a group. The combined contrast ($c_{ro}$) is calculated using Equation (13) for each node group:

$$SSIM=\left[(2l_r{l}_o+C_1)(2c_{ro}+C_2)\right]/\left[(l_r^2+l_o^2+C_1)(c_r^2+c_o^2+C_2)\right]$$

(12)

where $l_r$ is local luminance of reference image, $l_o$ is local luminance of other image, $c_r$ is local contrast of reference image, $c_o$ is local contrast of other image, $c_{ro}$ is combined local contrasts of a reference and another image, and $C_1$ and $C_1$ are constant values:

$$c_{ro}={\displaystyle \frac{1}{N-1}}\sum _{i=1}^N(r_i-l_r)(o_i-l_o)$$

(13)

where $l_r$ is luminance of reference image, $l_o$ is luminance of other image, $c_{ro}$ is combined contrast of reference and other image, N is number of nodes values in the node group, $r_i$ is reference image node value, and $o_i$ is other image node value.

3.2.4. Node Histogram Comparison

The node Histogram Comparison (HC) compares the distribution of electrical permittivity values in two graph images. Histograms are constructed for both the reference and target image, with each level representing a possible permittivity value (within a specified precision). The comparison calculates the average difference between the number of nodes at each corresponding histogram level.

A user-defined precision parameter determines the resolution of the histogram. Higher precision creates more levels for finer comparisons. Each node’s value is rounded to its corresponding level in the histogram, and the count for that level is incremented.

After generating histograms, the absolute difference between corresponding levels in both images is calculated. These differences represent the overall dissimilarity between the images’ permittivity distributions (see Equation (14)):

$$H=1-{\displaystyle \frac{▵h}{M+N}}$$

(14)

where $▵h$ is the sum of the absolute differences of two compared images, M is the number of reference image nodes, and N is the number of other image nodes.

3.3. Benchmarking Procedure

A benchmarking process involving paired image comparisons was conducted, to evaluate the performance of the Generalized Quality Assessment (GQA) method. Two corresponding reconstructions were generated for each phantom image in the dataset: a Low-Quality (LQ) reconstruction using 4 × 4 electrodes and a High-Quality (HQ) reconstruction using 4 × 8 electrodes.

The GQA method then calculated similarity measures between each reconstruction and the phantom image. An ideal GQA method should consistently assign a higher similarity score to the HQ reconstruction than to the LQ reconstruction. Discrepancies, where the LQ reconstruction receives a higher score, indicate errors in the GQA method’s assessment. Thus, to quantify the method’s accuracy and efficiency, the following metrics were collected:

• Error Rate: The percentage of phantom images where the LQ reconstruction received a higher similarity score than the HQ reconstruction.
• Execution Time: The average time the GQA method takes to calculate similarity measures for a single pair of images.

Furthermore, a G plot was generated (see Figure 8), to visualize the method’s performance across the dataset. This plot compared the similarity scores assigned to the LQ and HQ reconstructions for all the phantom images with the phantom ID mentioned on the horizontal axis. A clear separation between the two groups of points would indicate a strong correlation between image quality and similarity scores.

3.4. Experimental Setup

To simulate the acquisition process of Electrical Capacitance Tomography (ECT) data, a 3D ECT tomography model simulator was developed in MATLAB 2023a, leveraging the EIDORS 3.8 open-source library. A tetrahedral mesh with 5789 vertices and 25350 simplices was employed to represent the 3D ECT sensor model. Simulated capacitance electrodes were differentiated for both 4 × 4 and 8 × 4 configurations.

A diverse set of electrical permittivity phantoms was computed using the volume subtraction technique. Subsequently, Finite Element Method (FEM) solutions were obtained, to simulate the electrical field distribution within the phantoms under various Neumann and Dirichlet boundary conditions. The NLECTCM 3D iterative reconstruction algorithm was then applied, to reconstruct the 3D permittivity distribution from the simulated ECT data.

For visualization purposes, a custom Matlab code was developed, utilizing the Visualization ToolKit (VTK) library. This code facilitated the 3D rendering and analysis of both the original phantoms and their reconstructed counterparts.

To enhance the quality and interpretability of the 3D ECT reconstructions, a Graph Convolutional Network (GCN) pipeline was integrated. This pipeline operated on graph-based representations of the 3D images, leveraging the inherent connectivity and spatial relationships between the voxels.

The computational demands of the NLECTCM algorithm and the AI routines were addressed by using the Supermicro SYS-6029P HPC Server equiped with ten Nvidia A6000 graphic cards, dual Xeon Platinium 8368 CPUs and 1TB RAM (Supermicro, San Jose, CA, USA). This hardware enabled efficient parallelization of the calculations, accelerating both the raster-type and graph-based 3D ECT reconstruction processes.

4. Results

4.1. Numerical Results

To comprehensively assess the performance of the proposed Graph Quality Assessment (GQA) methods, we evaluated both the standard methods and their customized variants, including those with adjustable parameters. The complete set of evaluated methods was as follows:

• Peak Signal-to-Noise Ratio (PSNR)
• Group PSNR (depth 1 and 2)
• Group Structural Similarity Index Measure (G-SSIM) (depth 1, 2, and 3)
• Histogram Comparison (HC) (precision levels: 0.1, 0.01, and 0.001)

Our evaluation process consisted of two primary phases. Firstly, we systematically assessed each GQA method, using a provided dataset. The results included comparative charts of similarity values across different image qualities and a detailed analysis of benchmark errors. Secondly, we selected phantoms with diverse shapes from the dataset and visualized their corresponding 3D ECT scans at varying quality levels. In conjunction with the calculated similarity values from different the GQA methods, these visualizations allowed for a qualitative assessment of the methods’ performance.

4.1.1. Peak Signal-to-Noise Ratio

As depicted in Figure 8a, the PSNR method exhibited four distinct patterns:

• Period 1 → ball shapes: The initial 259 phantoms representing a ball shape showed a wide range of similarity values.
• Period 2 → H shapes: Phantoms 260–521, representing an H shape, demonstrated consistent similarity within each quality level, with clear differences between qualities.
• Period 3 → L shapes: Phantoms 522–783, representing an L shape, exhibited a tight clustering of Low-Quality values and a wider, evenly distributed spread for High-Quality values.
• Period 4 → rod shapes: Phantoms 784–end, representing a rod shape, showed a similar lack of consistency as Period 1, but with a narrower spread.

These patterns were observed in all the evaluated methods. The differences between the periods likely stemmed from variations in the phantom shapes within the dataset.

The PSNR method produced 43 errors, indicating that in 43 cases it incorrectly assigned a higher quality score to the Low-Quality image compared to the High-Quality image. The average execution time for the PSNR was 0.0034 s.

4.1.2. Group Peak Signal-to-Noise Ratio

For the Group PSNR with depth 1 (see Figure 8b), the results were similar to the PSNR, with an error rate of 59 and an average execution time of 0.0977 s. Using the depth of 2 produced 167 errors and had an average execution time of 0.2899 s. It is also worth noting that in Period 3 this method exhibited a distinct pattern, where a significant number of High-Quality measurements were clustered within the range of Low-Quality measurements (see Figure 8c).

4.1.3. Group Structural Similarity Index Measure

The Group SSIM method significantly improved the error rates compared to the PSNR methods, particularly at depth 1, where only 27 errors were observed. Additionally, the SSIM methods exhibited distinct patterns in Periods 1 and 4, with narrower value spreads. The Group SSIM method with a depth of 1 had an average execution time of 0.2075 s (see Figure 8d).

Increasing the depth to 2 resulted in a further slight reduction in errors to 25, but the execution time increased to 0.6648 s. Similar to the Group PSNR with depth 2, this method showed a clustering of High-Quality measurements within the Low-Quality range in Period 3 (see Figure 8e).

Finally, the Group SSIM with a depth of 3 further exacerbated the pattern observed in Period 3, with a larger number of High-Quality values clustered within the Low-Quality range. This method produced 71 errors and had an average execution time of 1.6794 s (see Figure 8f). Overall, the values appeared less structured than the G-SSIM methods with smaller depths.

4.1.4. Node Histogram Comparison

The Histogram Comparison method demonstrated varying error rates and execution times depending on the precision level. With a precision of 0.1 (see Figure 8g), the method produced 274 errors and had an average execution time of 0.0585 s. While it exhibited a structure similar to the G-SSIM charts, Period 3 showed a more even distribution of similarity values for Low- and High-Quality images.

Increasing the precision to 0.01 resulted in a higher error rate of 590, although the execution time remained comparable at 0.0594 s. Notably, this method consistently assigned higher similarity scores to Low-Quality images (see Figure 8h).

Finally, at a precision of 0.001, the error rate increased to 736 and the average execution time rose to 0.0664 s. This further emphasized the tendency to overestimate the similarity of Low-Quality images (see Figure 8i).

4.2. Benchmark Errors and Average Execution Times of GQA Methods

Table 2. Benchmark errors and average execution times for GQA methods.

GQA Method	Amount of Errors	Average Execution Time [s]
G-SSIM, depth 2	25	0.6648
G-SSIM, depth 1	27	0.2075
PSNR	43	0.0034
G-PSNR, depth 1	59	0.0977
G-SSIM, depth 3	71	1.6794
G-PSNR, depth 2	167	0.2899
HC, prec. 0.1	274	0.0585
HC, prec. 0.01	590	0.0594
HC, prec. 0.001	736	0.0664

presents the error count and average execution time for the evaluated methods in ascending order by their error count:

To better understand the relationship between node grouping depth in a GQA method and its error count (red series) and average execution time (blue), Figure 9 illustrates this correlation for the Group SSIM and Group PSNR methods.

4.3. Visual and Numerical Results—Selected Examples

This section presents results for applying QA measures to High-Quality (HQ) and Low-Quality (LQ) reconstructions of four phantom types: two-balls, H letter, rod, and side-ball. Figure 10 illustrates the phantoms and their corresponding reconstructions: (a) phantom, (b) High-Quality (HQ) reconstruction, and (c) LQ Low-Quality (LQ) reconstruction.

To quantitatively evaluate the performance of the proposed QA methods, Figure 11 presents a comparative analysis of the G-PSNR, G-SSIM, and HC scores for the Low-Quality (LQ) and High-Quality (HQ) reconstructions of the considered phantoms. The horizontal axis of each histogram corresponds to a specific phantom, enabling direct comparison of metric values between the LQ and HQ reconstructions.

5. Discussion

As shown in Table 2, the proposed G-SSIM method with node grouping depths of 1 and 2 demonstrates the most effective error reduction, i.e., the smallest number of LQ images assigned better scores than the HQ images. However, increasing the G-SSIM node grouping depth to 3 significantly increased errors, as depicted in Figure 9a.

While the average execution time for the G-SSIM methods was relatively high across all the evaluated variants, Figure 9a clearly shows a substantial increase in execution time when the node grouping depth was increased to 3.

Figure 9b illustrates a correlation between deeper node grouping and increased errors for the Group PSNR method. This effect might be due to deeper node groups encompassing more nodes, potentially complicating the the similarity assessment. Interestingly, the classic PSNR method outperformed its grouped variant regarding error reduction. For depths 1 and 2, the classic PSNR was the second-most-effective choice after the group SSIM methods.

Regarding average execution time, the PSNR method was the fastest among the examined GQA methods (see Table 2). Despite variations in average execution time observed across different depths of node grouping in Figure 9b, the group PSNR variant maintained a relatively low average execution time overall.

The Histogram Comparison method consistently encountered significant benchmarking errors. Increasing histogram precision directly correlated with increased error occurrences (see Table 2). While the average execution time remained relatively low across all the tested variants, the method’s overall performance was notably sub-par.

When the selected phantom examples are considered, the PSNR family methods and Histogram Comparisons with precisions of 0.01 and 0.001 assigned higher similarity values to the LQ results, while the G-SSIM family methods and Histogram Comparison with a precision of 0.1 assigned higher similarity values to the HQ results (see Figure 10).

As seen in Figure 10, for the two-balls, the rod, and the side-ball phantoms, a visual assessment makes it challenging to definitively determine which reconstruction was closer to the phantom. Consequently, the interpretation of the similarity results is also ambiguous. While the HQ and LQ similarities were close for the two-balls phantom, the HQ slightly exceeded the LQ for most GQA methods, except for the Histogram Comparisons at 0.01 and 0.001 precision. This likely stemmed from the faint connection between the balls in the scan images, absent in the phantom. The Low-Quality image exhibits a more pronounced unwanted connection. The visual similarity aligns with the previous phantom observations. This is to be expected, as both phantoms consisted of ball shapes. For the H phantom, the scan images are noticeably distorted compared to the phantom. The H shape is deformed in both scan images, and the Low-Quality image lacks red spots. Conversely, the High-Quality image has red areas resembling the H’s vertical rods. The scan images for the H phantom are less similar to the phantom than those of the ball phantoms. However, there is a greater similarity between the High-Quality and Low-Quality images.

The numerical results generally align with the visual observations, except for the Histogram Comparisons at 0.01 and 0.001 precision. The HQ results are significantly higher than the LQ results. For the rod phantom, while the scan images differ, it is challenging to determine which is closer to the phantom. The rod shape is well preserved in both, but the red spots are distributed differently. The High-Quality image has a denser red center, while the Low-Quality image has red spots that are more evenly spread.

The GQA method results for the different quality scan images vary. The PSNR family methods and the Histogram Comparisons at 0.01 and 0.001 assigned higher values to the LQ results, while the G-SSIM family methods and the Histogram Comparison at 0.1 assigned higher values to the HQ results. For the side-ball phantom, the visual similarity is ambiguous. However, the Low-Quality image contains more red spots than the High-Quality image. Figure 11 compares the similarity results for the side-ball phantom. This phantom scored benchmark errors for all the examined GQA methods, with LQ similarity consistently higher than HQ similarity. Since the visual determination of the closest reconstruction is challenging, interpreting the similarity of the results is also ambiguous.

The uncertain performance of the benchmark error-prone reconstructions for the last two phantoms (rod and side-ball) suggests that some benchmark errors in the dataset originated from instances where certain High-Quality images were, in fact, less similar to their phantoms than the corresponding Low-Quality images.

Electrical Capacitance Tomography systems often produce images that can be blurry, distorted, and lack clarity. These image quality issues can hinder accurate interpretation and decision making in real-time monitoring and diagnosis of industrial processes. By analyzing the graph structure of the 2D/3D ECT image, GQA can identify and flag artifacts like ringing, blurring, or ghosting. This early detection can trigger offline and online corrective actions, such as adjusting ECT sensor calibration or re-acquiring broken or noisy data. GQA modules can be integrated into existing ECT systems as standalone components or as part of a larger image-processing pipeline. GQA metrics can also be used to support and prepare online ECT industrial process monitoring by optimizing the regularization parameter in image reconstruction algorithms. They can help in selecting the most appropriate image reconstruction model. By comparing the quality of images produced by different models, the best-performing one can be chosen for the online scheme. By assessing the quality of reconstructed images for different parameter values, the optimal setting can be determined. GQA can also guide 3D image enhancement techniques, such as contrast enhancement, sharpening, or noise reduction, by identifying areas that require improvement, which can be added to the 3D image-processing pipeline. GQA techniques can be computationally intensive, especially for real-time applications, so efficient algorithms and GPU-based hardware acceleration techniques can be used to reduce processing time and achieve an acceptable image frame rate.

6. Conclusions

This study presents a novel approach to the accurate Quality Assessment of graph-based images in 3D ECT. By adopting state-of-the-art GQA methods and developing a 3D ECT tomography-based dedicated benchmarking system, we established a robust framework for evaluating the quality of graph-based 3D ECT reconstructions. Specifically, we proposed tailored Graph Quality Assessment (GQA)-based methods, and we demonstrated their effectiveness in evaluating the quality of graph-based 3D Electrical Capacitance Tomography (ECT) image reconstructions. By adapting various image Quality Assessment methods to the graph-based image representation, resulting in our proposed Group Peak Signal-to-Noise Ratio (G-PSNR), Group Structural Similarity Index Measure (G-SSIM), and Histogram Comparison (HC), we have shown that these methods can accurately assess the similarity between reconstructed and reference images. Specifically, our results indicate that the G-SSIM method with node grouping depths of 1 and 2 achieves the most effective error reduction, while the PSNR method is the fastest among the examined GQA methods.

The experimental results presented in this paper highlight the robustness of the proposed GQA methods in evaluating the quality of graph-based 3D ECT image reconstructions. Specifically, the visual and numerical results for the selected examples of various-shaped phantoms demonstrate the effectiveness of these methods in accurately assessing the similarity between reconstructed and reference images. These findings could contribute to advancing 3D ECT imaging by providing a reliable and effective means for evaluating the quality of graph-based image reconstructions.

Future work could build upon this study by exploring the development of methods that can directly compare graph structures, rather than relying on image-based Quality Assessment metrics. This could involve adapting graph-comparison algorithms, such as graph edit distance or graph kernel methods, to the specific requirements of 3D ECT image reconstruction. The development of new GQA methods that can handle complex graph structures and varying node densities could further enhance the accuracy of Quality Assessment in 3D ECT. Furthermore, the integration of GQA methods with machine learning algorithms could enable the development of automated quality control systems for 3D ECT imaging. By continuing to advance the field of Quality Assessment for 3D ECT, researchers and practitioners can work towards improving the reliability and accuracy of this imaging modality, ultimately leading to better process monitoring and control in industrial settings.