Next Article in Journal
A Novel Systematic Oxidative Stress Score Predicts the Survival of Patients with Early-Stage Lung Adenocarcinoma
Next Article in Special Issue
Multimodal Prehabilitation in Patients Undergoing Complex Colorectal Surgery, Liver Resection, and Hyperthermic Intraperitoneal Chemotherapy (HIPEC): A Pilot Study on Feasibility and Potential Efficacy
Previous Article in Journal
Histology and PSMA Expression on Immunohistochemistry in High-Risk Prostate Cancer Patients: Comparison with 68Ga-PSMA PET/CT Features in Primary Staging
Previous Article in Special Issue
Design and Assessment of a Novel Biconical Human-Sized Alternating Magnetic Field Coil for MNP Hyperthermia Treatment of Deep-Seated Cancer
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Hyperthermia Treatment Monitoring via Deep Learning Enhanced Microwave Imaging: A Numerical Assessment

by
Álvaro Yago Ruiz
1,2,*,
Marta Cavagnaro
2,† and
Lorenzo Crocco
1,*
1
CNR-IREA National Research Council of Italy, Institute for Electromagnetic Sensing of the Environment, 80124 Naples, Italy
2
Department of Information Engineering, Electronics, and Telecommunications, University of Rome “La Sapienza”, 00184 Rome, Italy
*
Authors to whom correspondence should be addressed.
Prof. Cavagnaro is also with CNR-IREA as an associate researcher.
Cancers 2023, 15(6), 1717; https://doi.org/10.3390/cancers15061717
Submission received: 15 February 2023 / Revised: 8 March 2023 / Accepted: 9 March 2023 / Published: 11 March 2023
(This article belongs to the Collection Hyperthermia in Cancer Therapy)

Abstract

:

Simple Summary

Non-invasive temperature monitoring during hyperthermia cancer treatment is of paramount importance. It allows physicians to verify the therapeutic temperature is reached in the treated area. Currently, only superficial or invasive thermometry is performed on a clinical level. Magnetic resonance thermometry has been proposed as a a non-invasive alternative but its applicability is limited. Conversely, microwave imaging based thermometry is a potential low cost candidate for non-invasive temperature monitoring. This works presents a computational study in which the use of deep learning is proposed to face the challenges related to the use of microwave imaging in hyperthermia monitoring.

Abstract

The paper deals with the problem of monitoring temperature during hyperthermia treatments in the whole domain of interest. In particular, a physics-assisted deep learning computational framework is proposed to provide an objective assessment of the temperature in the target tissue to be treated and in the healthy one to be preserved, based on the measurements performed by a microwave imaging device. The proposed concept is assessed in-silico for the case of neck tumors achieving an accuracy above 90%. The paper results show the potential of the proposed approach and support further studies aimed at its experimental validation.

1. Introduction

Hyperthermia treatment (HT) is an adjunctive cancer therapy in which cancerous tissues are heated up to 40–43 °C to help damage malignant cells. This is achieved by sensitizing the cells to radiation in radiotherapy (RT) or increasing the permeability of the cell membrane to drugs in chemotherapy [1]. The HT synergistic effect inhibits DNA damage repair and increases blood flow (thereby enhancing oxygenation). In particular, RT is more cytotoxic in normal oxygenation conditions than in hypoxic environments. In this regard, HT may act as a complementary treatment of RT in cases of tumor hypoxia by sensitizing the cells to heat in low-oxygen conditions [2]. Additionally, HT induces the activation of heat shock proteins and a tumor-specific immune response.
Local, regional, and whole-body HT may be considered depending on the size and location of the treatment area. HT consists of a heating applicator that is able to supply heat to the treatment area. However, heat transfer is not easily applied in a localized manner. Tracking the temperature during the HT is needed to verify that the therapeutic temperature is reached in the target tissue and that the surrounding healthy tissue is not heated (and, thus, not damaged) giving rise to the so-called hot spots. In clinical practice, this task is performed by means of superficial thermometry or by directly inserting thermal sensors into the tissue, such as thermocouples or optical fibers. However, these approaches are limited to superficial HT or are invasive (due to the need for inserting the probes), respectively. Furthermore, since probes are sensitive to temperature in the insertion location, invasive thermometry is limited to local measurements. Patient feedback is also taken into consideration during HT, but it is subject to individual patient perception.
Considering the above reasons, there is increasing interest in the development of fully non-invasive temperature monitoring approaches capable of overcoming the above issues. To this end, magnetic resonance (MR) thermometry [3] has been proposed based on temperature-sensitive parameters involved in MR, such as proton resonance frequency, diffusion coefficient, T 1 and T 2 relaxation times, magnetization transfer, and proton density. However, MR thermometry is only feasible in 15% of the treatments, mainly when the treatment area is in the limbs or the HT device fits inside the bore of the MR system [4,5]. Moreover, HT-MR has a high cost and its outcomes are affected by the inaccuracies introduced by movements (e.g., breathing, bowels, blood vessels). Finally, standard HT applicators may not be suitable for operation in the presence of electromagnetic (EM) fields generated by an MR machine due to EM compatibility issues, and thus, they must be redesigned accordingly.
Microwave imaging (MWI) [6] is an emerging medical imaging modality, which may represent a potential candidate technology for non-invasive temperature monitoring in HT. MWI is based on the scattering phenomenon that occurs when an EM field interacts with an object. Such an interaction perturbs the field according to the morphological properties of the target and its EM properties, i.e., dielectric permittivity and electric conductivity. By processing the perturbed field, known as the interrogating one, it is possible to build an image of the investigated scenario in terms of a map of the spatial distribution of the EM properties in the region under test. In the field of medical imaging, MWI is applicable thanks to the different EM properties that characterize human tissues, as well as the different statuses (healthy vs. pathological) of certain tissues, due to the non-ionizing nature and low power of the EM fields involved, as well as the low cost of the underlying components, MWI has garnered significant research attention in recent years, particularly in the development of devices for the early diagnosis of breast cancer [7] and cerebrovascular diseases [8].
As there is a known relationship between temperature and tissue EM properties [9], MWI can also be used to monitor temperature during HT. This was suggested by Bolomey and colleagues many years ago [10]. Notably, unlike MR thermometry, MWI-based thermometry would rely on portable, cost-effective devices, and would not pose significant issues in terms of EM compatibility when integrated with HT systems. On the other hand, the mathematical nature of the problem underlying MWI poses a non-trivial challenge in the practical implementation of the technique. MWI involves solving an inverse scattering problem (ISP) [6,11], which is known to be nonlinear and ill-posed, leading to instabilities in the solution process. As such, inaccurate estimations of the EM properties (and, hence, their temperature) may negatively impact the reliability of the treatment monitoring technique. For this reason, while the use of MWI to monitor the temperature in HT has been investigated by several authors [12,13,14,15], the possibility of quantitatively sensing and tracking the changes in the EM properties occurring due to thermal treatment via MWI is still an open issue.
Recently, there has been increasing attention given to the use of machine learning tools to address the difficulties associated with solving the ISP in MWI [16]. In particular, the ability of deep learning (DL) algorithms to solve complex nonlinear regression problems has been explored, especially in the development of physics-driven solution frameworks. In the latter, DL architectures are combined with standard MWI approaches that supply the neural network with domain knowledge that either makes the training of the network easier or its predictions more efficient [17,18]. However, to the best of our knowledge, the use of these powerful tools to provide a robust solution to ISP in the framework of temperature monitoring via MWI has never been explored.
According to the above, this paper describes a physics-driven DL framework to perform reliable, quantitative, and robust MWI-based monitoring and tracking of the temperature evolution during HT. The basic assumption of the proposed approach is that accurate knowledge of the treated region (in terms of tissue segmentation) is available thanks to the treatment planning stage routinely performed before the actual treatment. Prior knowledge about the changes in EM properties within the temperature range of HT allows for the ISP to be formulated as a linear inverse problem, with remarkable simplification. However, the problem remains ill-posed and may result in inaccurate or unstable estimations, which can be addressed by exploiting DL. In particular, the proposed approach consists of a first stage in which the MWI raw data are processed by means of a linearized inversion algorithm based on the distorted Born approximation [6], in which the stability of the solution is supplied by adopting the truncated singular value decomposition [19]. This step focuses on an (approximated) estimation of the EM properties in the regions of interest, i.e., the tumor to be treated and some surrounding regions to be protected. In the second step, the MWI imaging results are supplied to a convolutional neural network (CNN), trained in such a way as to classify the incoming results according to their temperature. The output of the CNN is a set of classification labels, which provides immediate information on the heated or unheated status of the region of interest.
In the following sections, we provide a detailed description of the proposed approach and present a simulated example of monitoring an HT treatment of neck tumors to provide an initial proof of concept.

2. Material and Methods

2.1. Electromagnetic Properties of Tissue

The goal of a general MWI system consists of the characterization of the target in terms of its morphological features, such as shape and position, and its electromagnetic properties, namely, dielectric permittivity, electric conductivity, and magnetic permeability.
Some materials, including biological tissues, are non-magnetic, meaning they have free space permeability μ = μ 0 = 4 π × 10 7 H/m or relative permeability μ r = μ / μ 0 = 1 . Consequently, permittivity and conductivity are usually the quantities of interest. Exploiting the formulation for time-harmonic waves [20], they can be conveniently incorporated into the relative complex permittivity, defined as
ε r = ε r j ε r ,
where j stands for the imaginary unit.
In (1), the real and imaginary parts of ε r encode dielectric and conductive phenomena depending on the specific features of the media at hand. Biological tissues present dispersive behaviors, meaning that their EM properties depend on the operating frequency f. Moreover, and more important to our purposes here, their EM properties exhibit a dependence on temperature [9], which lays the basis for the feasibility of MWI-based monitoring of temperature in HT treatments.
Accordingly, the EM behavior of tissues may be modeled using a temperature dependent Cole–Cole model [9]:
ε r ω , T = ε T + Δ ε 1 T 1 + j ω τ 1 T 1 α 1 + Δ ε 2 T 1 + j ω τ 2 T 1 α 2 + σ T j ω ε 0
where ω = 2 π f is the angular frequency of the time-harmonic field, T is the temperature, ε is the permittivity at very high frequencies, Δ ε 1 and Δ ε 2 are the dispersion amplitudes with their corresponding relaxation times τ 1 , τ 2 , and σ is the conductivity, measured in [ S / m ] . α 1 and α 2 are dispersion-broadening parameters and ε 0 = 8.858 × 10 12 F/m is the dielectric permittivity in the vacuum.

2.2. The Microwave Imaging Problem

The main challenge in MWI is solving the ISP, which is addressed in this paper as a 2D problem. Specifically, the MWI procedure is applied to a single slice of the body, as depicted in Figure 1. In a typical MWI setting, let Ω denote the selected imaging domain (i.e., the slice under treatment) enclosing a cross-section Σ of a collection of tissues, which are assumed to be elongated bodies that extend to infinity. Such a circumstance implies that the (unknown) properties of the tissues do not change across the longitudinal z-axis, / z = 0 (2D problem). Thus, changes only happen in the transverse plane = t r a n s = x x + y y . Additionally, such tissues are probed with time-harmonic incident fields E i n c radiating in a transverse magnetic (TM) mode under the time convention e j ω t . In TM-polarization, the magnetic field is transverse to the z-axis, ( H x , H y ), while the electric field only has the longitudinal component, E z . As the magnetic and electric fields are uniquely linked through the Maxwell equations for any fixed scenario, in these conditions the ISP can be cast as a scalar problem by considering only the electric field.
The interrogating process begins with the incident fields impinging the imaging domain. Such fields are radiated from a set v = 1 , , N v of transmitting antennas T x [ v ] , located outside Ω , on a curve Γ enclosing it. For the generic transmitter positioned in r v Γ , the interaction between E i n c and the tissues gives rise to a scattered field E S , which can be measured by a set m = 1 , , N m of receiving antennas R x [ m ] , positioned in r m Γ . More precisely, the actual field measurements at R x [ m ] correspond to a total field E t o t , which is a superposition of both incident and scattered fields E t o t = E i n c + E S . The interactions between the interrogating fields and the tissues are governed by the state and data equations:
E t o t ( r , r v ) = E i n c ( r , r v ) + k b 2 Ω g ( r , r ) χ ( r ) E t o t ( r , r v ) d r
E S ( r m , r v ) = k b 2 Ω g ( r m , r ) χ ( r ) E t o t ( r , r v ) d r
where for a given angular frequency ω , k b = ω μ 0 ε 0 ε r b is the wavenumber of the background medium in which the probing antennas are located, whose relative complex permittivity is ε r b . g denotes the scalar Green’s function, i.e., the impulsive response in the background medium, and χ = ε r / ε r b 1 is the contrast function, which encodes the EM properties of the targeted tissues. The retrieval of χ from (3) and (4) is the goal of the ISP underlying MWI.

2.3. Deep Learning Microwave Imaging Framework for HT Temperature Monitoring

During HT, physicians aim to monitor the temperature to determine whether certain areas are exceeding a defined maximum temperature safety threshold (in addition to patient feedback), while ensuring that the temperature in the tumor is within the therapeutic range and not too hot, which could lead to ablation, and, thus, other protocols should be used [1].
In this respect, the devised DL-MWI monitoring system could support the physician acting as a cooperative safety system which confirms the achievement of the treatment endpoint in the tumor and points out hazardous heating in temperature sensitive regions.
This approach can be naturally turned into a classification problem in which a categorical label is assigned to each region of interest (ROI), i.e., the tumor region or/and the surrounding healthy tissues, according to the temperature. In particular, in the tumor region, the expected outcome reaches the therapeutic temperature, whereas, in the healthy tissue, the goal is to avoid hot spots.
As illustrated in Figure 2, such an application can be conveniently cast in terms of a physics-assisted DL-MWI framework involving the processing of the raw data measured by the MWI device through a suitable MWI algorithm, in order to obtain an image of the treatment area, followed by a deep learning architecture in charge of classifying the incoming images into their corresponding temperature labels. Obviously, some surrounding tissues of the tumor tolerate heat better than others. In fact, hot spots in nervous tissue may be more dangerous to the patient than those happening in other areas given the known heat sensitivity of this type of tissue [1]. For this reason, the proposed framework is conceived not as a general hot spot detection framework but rather as a monitoring framework of the tumor as well as temperature-sensitive ROIs.

2.3.1. Microwave Imaging Processing

The proposed DL-MWI framework uses a MWI algorithm to process the measured raw data and provide a 2D image of the treated region. From this image, the DL architecture derives the temperature status of the tumor and the healthy tissue to be protected. Due to the small perturbations of the EM properties in the treated tissue corresponding to temperature increases Δ T during HT [9], a linear approximation of the EM scattering phenomenon, namely the distorted-wave Born approximation (DWBA) [21], can be utilized.
To introduce the DWBA formulation, let us denote with t 0 the starting time of the treatment and with t s the generic time instant during the treatment, and with χ [ 0 ] and χ [ s ] the corresponding contrast functions. The differential contrast
δ χ = χ [ s ] χ [ 0 ]
provides information on the variation of the tissue EM properties during the treatment, which in turn reflects the temperature changes within the tissue. As these changes are small, it follows that | δ χ | < <   | χ [ s ] | , | δ χ | < <   | χ [ 0 ] | . Accordingly, E t o t [ s ] E t o t [ 0 ] , where E t o t [ 0 ] is the field in the ROI at the starting stage of the treatment, i.e., when all of the tissues are at physiological temperatures and E t o t [ s ] is the field at t s . Notably, considering that HT treatments are preceded by a treatment planning stage that involves using MR followed by accurate segmentation of the tissues in the treatment area [1], it is possible to compute the field E t o t [ 0 ] via EM simulations.
For the small perturbation regime underlying the DWBA, and assuming δ χ as the unknown of the MWI problem, (3)–(4) reduce to a single linear equation, which reads:
Δ E S ( r m , r v ) = k b 2 Ω E t o t [ 0 ] ( r , r m ) E t o t [ 0 ] ( r , r v ) δ χ ( r ) d r = L Ω δ χ
where L Ω is a short notation for the linear and compact integral operator appearing in the intermediate term and Δ E S = E S [ s ] E S [ 0 ] is the differential scattered field obtained by subtracting the scattered field measured by the MWI system in the unheated conditions, E S [ 0 ] , from the one at the time t s of the treatment E S [ s ] . Note that the role of Green’s function in Equation (6) is played by E t o t [ 0 ] ( r , r m ) since the unheated condition is considered as the reference scenario.
The linear ill-posed problem cast via the DWBA Equation (6) can then be solved in a regularized form via the truncated singular value decomposition (TSVD) [6] to provide a stable reconstruction. The TSVD inversion of Equation (6) reads:
δ χ = p = 1 P c u t 1 λ p ν p u p H Δ E S
where λ p stands for the p-th singular value of L Ω and u p , ν p for its left and right p-th singular vectors, respectively. P c u t is the regularization parameter that truncates the summation in (7). It is worth noting that the estimation of the differential contrast provided by (7) can be achieved in real time since its computationally intensive part (the evaluation of the SVD of L Ω ) is done offline before the processing of the measured data.
The above DWBA inverse formula refers to the evaluation of the differential contrast over the entire treated region Ω . However, the dimensions and positions of the tumor and the healthy tissue to be preserved are a priori known from the treatment planning stage. Hence, it can be convenient to restrict the ISP to the sole identification of the contrast perturbations at those locations, say Ω 1 and Ω 2 , which are the ones of actual clinical interest. In doing so, a further simplification is introduced by the fact that, since the DWBA holds, the scattering phenomenon is localized and the mutual interaction between the two ROIs are negligible. Hence, the imaging of the two ROIs can be handled separately by computing the TSVD images for the operators L Ω 1 and L Ω 2 , obtained by restricting L Ω to the relevant ROIs. The resulting estimated differential contrasts δ χ 1 and δ χ 2 represent the input of the DL architecture.

2.3.2. Deep Learning Architecture for Classification

Once the MWI processing has generated the two images of the tumor and the healthy tissue to be protected, two independent DL architectures can be trained—one in charge of monitoring the temperature in Ω 1 and one in charge of monitoring the temperature in Ω 2 . Among the possible choices, here, two convolutional neural networks (CNNs), both with identical structures, are used. CNNs are broadly applied in classification tasks, given their superior performances as compared to traditional algorithms. Their most important characteristic involves the use of concatenated convolutional layers, where K kernels are convoluted over incoming images to provide a solution [22]. The internal structure of one of the two CNNs is shown in Figure 3.
The heating statuses of the two ROIs can be presented as categorical labels derived from CNN classifiers. However, there is a difference between the expected outcomes of the two cases. The tumor is expected to stay in the therapeutic regime between two thresholds; therefore, the tumor CNN has to work with N c = 3 classes: {unheated, therapeutic, hot}. The healthy tissue must remain unheated during the whole treatment. Therefore, N c = 2 classes are required to handle its case: {unheated, hot}.
The parameters W of the two CNNs must be optimized based on a loss function. For categorical labels, the categorical cross-entropy is typically used:
L e W = 1 N N c n = 1 N c = 1 N c C c [ n ] · l o g C ^ c [ n ] W
where N corresponds to the dataset size, C is the ground truth label that should be retrieved and only supplied during the training, and C ^ is the output prediction.
Since hot spots may appear in any of the tissues surrounding the tumor, monitoring more than two regions could be needed. Notably, this does not require substantial changes to the proposed approach. More precisely, the overall DL-MWI framework would be the same, while only the number of TSVD images and CNNs would have to be changed. Accordingly, without loss of generality, the following proof of concept is discussed for the case of two regions. However, the proposed approach is not limited to such regions.

2.4. Assessment of the DL-MWI Framework for Temperature Monitoring in Neck Tumor Hyperthermia

In the following, the above-described DL-MWI framework is developed and numerically assessed for the case of neck tumors. More precisely, in this case, the objective is the simultaneous monitoring of the tumor and the spinal cord heating status. A scheme of the whole framework is illustrated in Figure 4.

2.4.1. Anatomical Model

To perform the simulated study aimed at assessing the proposed approach, a numerical phantom of the neck was built. According to the assumed 2D geometry, a neck slice was extracted from the Ella anatomical model [23], which has been broadly adopted for numerical studies involving interactions between EM waves and the human body. Then, a target with a circular cross-section and a radius of 4.5 mm (with the EM properties of a nodular tumor [24]) was added to the thyroid gland to simulate the pathological condition. The tissue segmentation and electromagnetic properties of the resulting phantom are shown in Figure 5. The EM behaviors of all tissues were modeled using the Cole–Cole model Equation (2) with the parameters obtained from a publicly available database [25]. However, for the pathological tissues, the parameters were taken as those of a nodular goiter tumor [26].

2.4.2. MWI Simulations

To simulate the measurements generating the MWI raw data, the interactions between the phantom described above and the MWI antennas were computed by numerically solving the scattering Equations (3) and (4). To this end, proprietary software based on the method of moments (MoM) [27,28] and the conjugate gradient fast Fourier transform (CG-FFT) [29] was exploited.
In the simulations, the antennas used to generate the MWI data are modeled as point sources and their arrangements around the neck are depicted on the left side of Figure 4. As can be noted, the N v = N m = 12 antennas are not evenly spaced, so there are gaps to accommodate the antennas of the HT applicator and those of the MWI system. In particular, assuming that the HT applicator is made of three antennas, one at the front of the neck and the other two at its sides, the MWI antennas are positioned in groups of three at each side of the HT antenna placed in front of the tumor, whereas the remaining six were positioned behind the neck. In order to compensate for the limited number of measurements, frequency diversity was exploited by simulating the measurements of the data at N f = 10 frequencies evenly spaced in the f = [ 0.9 , 1.0 , , 1.8 ] GHz range. Such range of frequencies, as well as the EM properties of the water mixture used as the background medium were chosen in accordance with previous studies [30].
To simulate the changes in the tissue EM properties during the treatment, the MWI simulations were repeated for several conditions. For the tumor, 10 different values of the EM properties were considered, corresponding to the temperatures in the T = [37–46] °C range, whereas for the spinal cord, the selected temperature range was narrower, T = [37–40] °C, as the goal was to detect the switch between two conditions (heated–unheated). For both tissues, the Cole–Cole parameters were modified according to the model in [9]. The differential data Δ E S and the MWI raw data were computed for each of the considered temperature conditions, and the scattered fields resulting from the simulations were corrupted with additive white Gaussian noise ( S N R = 30 dB) to mimic the unavoidable presence of the measurement noise.

2.4.3. MWI Imaging Results

For each set of multi-frequency MWI raw data, two TSVD images were computed—one for the tumor and one for the spinal cord. To this end, the two operators, L Ω 1 and L Ω 2 , were built from the knowledge of the total field inside the ROIs at the beginning of the treatment, E t o t [ 0 ] , and of the morphology of the tumor and the spinal cord, respectively. Note that, for the sake of simplicity but without loss of generality, in this study, Ω 1 and Ω 2 were taken as square domains enclosing the anatomical site of interest. The whole imaging domain had N x = N y = 128 pixel dimensions, from which the two ROIs were cropped around the pixels belonging to the tumor and the spinal cord, leaving N x = N y = 5 and N x = N y = 6 , respectively. Finally, the TSVD images were computed using Equation (7) with P c u t = 1 for both cases.

2.4.4. CNN Implementation: Categorical Labels

For the considered temperature ranges, the ground truth classes in C were set as:
C 1 Δ T = 0 if   Δ T < 3   ° C 1 if   3 Δ T < 7   ° C 2 if   7 Δ T   ° C
corresponding to { 0 , 1 , 2 } = {unheated, therapeutic, hot}. Note that the typical HT treatment therapeutic regime was T = [ 40 , 43 ] °C, i.e., Δ T = [ 3 , 6 ] °C.
Similarly, for the spinal cord,
C 2 Δ T = 0 if   Δ T < 2   ° C 1 if   2 Δ T   ° C
with { 0 , 1 } = {unheated, hot}. Here, Δ T = 1 °C was considered unheated for the purposes of these experiments. In clinical practice, a more stringent threshold could be used.

2.4.5. CNN Implementation: Training

The necessary datasets for training the two CNNs were built by performing a number of simulations in the conditions described above. Each simulation provides an instance of the training set made by a pair δ χ , C c , i.e., the differential contrast estimated via TSVD in the relevant ROI and the corresponding categorical label.
In total, N 1 = 3000 samples for the tumor CNN and N 2 = 2000 samples for the spinal cord CNN were simulated. Each sample was built by randomly varying the EM properties of all tissues within a ± 0.2 -wide interval around their average values. For each tissue, this corresponds to modeling the real and imaginary parts of the complex dielectric permittivity as two normally distributed variables, ε r N ( ε r , 0 . 1 2 ) and ε r N ( ε r , 0 . 1 2 ) . Modeling tissues with higher standard deviations than 0.1 is possible, but would possibly require increasing the number of samples N 1 and N 2 to obtain performances comparable to those reported in Section 3.
For implementation purposes, images had to be fed into the CNNs as real-valued quantities; therefore, the real and imaginary parts of the TSVD images had to be split and fed as stacks of images with N c h a n = 2 channels, such as δ χ R N x × N y × N c h a n . In Figure 4, two pairs of TSVD images (one for each CNN) are shown as examples of the input data.
The training was carried out using a K-fold cross-validation scheme [22], where some instances from the two datasets N 1 and N 2 were removed for each training iteration. In K-fold cross-validation, a dataset was split into N f o l d sub-datasets of equal sizes N v a l to carry out N f o l d training. In each iteration, one of the sub-datasets was removed from the dataset, leaving N N v a l instances for training and N v a l instances for validation of the model’s performance. Once the N f o l d training was concluded, the performance could be assessed by averaging the results across folds. In this work, N f o l d = 10 was for both datasets (tumor, spinal cord). The validation split was N v a l = 300 for the CNN whose duty was to monitor the temperature status of the tumor, whereas N v a l = 200 for the CNN in charge of monitoring the temperature status of the spinal cord. The Adam optimizer [31] was used with a learning rate η = 10 4 and a batch size of N b = 16 . The trainings were run for a maximum of N t = 300 epochs with an automatic stopping criterion of 10 consecutive epochs without improving the validation loss. Note that an epoch of t corresponds to a complete pass of all the instances in a given fold.

2.4.6. Performance Assessment Metrics

Metrics to evaluate the performances of the classification problems involving categorical labels are based on the positive (P)/negative (N) count, such as the Dice similarity coefficient ( D S C ) [32]:
D S C = 2 · T P 2 · T P + F P + F N
where T P , F P , and F N correspond to the true positive, false positive, and false negative count, respectively. More precisely, T P corresponds to the statistical count of correctly labeled instances into one of the temperature status categories. In contrast, F P and F N correspond to the mistakes incurred by the CNNs when labeling the instances. For example, a T P would be a variation of Δ T = 4 °C, correctly labeled as therapeutic when considering the tumor CNN. Additionally, a second metric called the Matthews correlation coefficient ( M C C ) [33] was calculated as well:
M C C = T P · T N F P · F N D M
where D M = ( T P + F P ) ( T P + F N ) ( T N + F P ) ( T N + F N ) . As opposed to D S C , M C C employs a true negative count, T N , i.e., the statistical count of the correctly labeled instances as not belonging to a category. In general, M C C tends to show lower values than D S C and, therefore, provides a more conservative performance evaluation. Both metrics achieve their highest values at 1.00 .

3. Results

After the CNN training is carried out on all the folds, the performance are assessed using the selected metrics ( D S C and M C C ). The metrics are computed class-wise on the validation split of each fold. Then, the final result for each class is obtained by averaging among all the obtained values. Table 1 and Table 2 report the metrics for the tumor CNN and for the spinal cord CNN, respectively.

4. Discussion

Concerning the tumor CNN, in agreement with the expectations, M C C tends to report lower values than D S C (see Table 1). Nevertheless, the achieved values are comparable between the two metrics for all classes. Moreover, there are no major differences in the performance of any specific class over the rest, only a slightly superior performance of the unheated class over the other two. More in detail, the overall number of misclassified samples is 26, which corresponds to about 8.5 % of the whole validation split. In particular, for 2 samples ( 0.6 % ) the CNN predicted that the temperature was above the therapeutic one of 42 °C, while it was not. Of these, only 1 ( 0.3 % ) was above the therapeutic range. For another 15 samples ( 5.0 % ), the CNN predicted that the therapeutic regime was not reached, while instead it was. For 9 ( 3.0 % ) overheated samples, the CNN erroneously attributed the therapeutic label. These results are summarized in the confusion matrix reported in Figure 6.
For the spinal cord CNN, the performance scores are on average slightly worse than those of the previous case (see Table 2). This suggests that the task of classifying the spinal cord temperature status is more challenging, even though less classes were considered in this case. Such an outcome may be explained by two aspects. The first one is the further distance of the spinal cord from the antennas in the selected setup configuration (see Figure 4) as compared to the rather superficial location of the tumor. The second aspect is the smaller temperature range which is considered. Both these aspects result in scattered fields which are possibly more corrupted by noise and thus in less accurate TSVD images. Going more into detail, the reported metrics correspond to 16 misclassified samples, i.e., 8 % of the validation set, evenly distributed between the two classes. Interestingly, 6 out of the 8 cases in which the spinal cord was erroneously labeled as unheated correspond to cases in which the temperature of the sample was 39 °C. From a practical perspective, this suggests that the network performs better when the spinal cord temperature is above 39 °C and thus that, even with some delay with respect to actual moment in which the hot spot occurs, it would eventually reveal it.
To figure out how to further improve the performance of the proposed framework, it is worth to analyze the behaviour of the categorical cross-entropy in the training and in the validation. Figure 7, reports the categorical cross-entropy of the fold whose performance was closest to the average in each dataset for the tumor CNN and the spinal cord one. Both plots confirm that the training is not affected by overfitting and hence that the adopted training is suitable to optimize the CNNs. In addition, the validation losses are on average always below the corresponding training losses across epochs, which is consistent. On the other hand, the cross-entropy for the validation splits exhibits an increasingly oscillating behavior with the epochs, especially for the spinal cord CNN. This suggests that the considered number of epochs is sufficient (if not redundant) and that there is room for performance improvements if the datasets get increased beyond N 1 = 3000 and N 2 = 2000 .

5. Conclusions

The exploitation of a DL-MWI approach for HT monitoring was proposed and initially validated in silico for the case of neck tumors. The proposed approach is a physics-assisted DL-MWI framework, in which an image generated via MWI is input to a CNN that classifies it into one of several predefined categories. In particular, since tissue EM properties exhibit very small variations in the temperature range of HT, the ISP can be cast in terms of a linear inverse problem, whose solution can be reliably and efficiently built using the TSVD algorithm. In addition, this allows for achieving two separate images, i.e., one for the region of the tumor to be treated and one for the healthy tissue to be protected. Accordingly, two CNNs are exploited to provide a classification of hyperthermia measurements into labels reporting the temperature status in the different ROIs. The approach assumes prior knowledge of the morphology and tissue segmentation of the treated region, based on the fact that such information can be obtained from the treatment planning stage routinely performed in clinical HT.
The compelling results shown by the proposed approach on the validation samples provide a good basis to progress toward the experimental demonstration. To this end, a more realistic numerical scenario could be used to validate the approach, which includes representing realistic antennas, evaluating the temperature distribution using the bio-heat equation or other models proposed in the literature [34], taking into account additional information on the dependence of EM properties on the physiological changes of the tissue during the hyperthermia treatment, and extending the framework to the 3D geometry. Notably, all of these aspects only imply increased computational complexity but do not require changing the overall architecture of the approach. In addition, to better integrate the proposed approach in the clinical flow, an interesting development could be that of linking the classification labels to the usual HT monitoring parameter. For instance, the way in which the classification labels are designed has a strong connection with T 90 since the therapeutic regime is only assigned when all of the pixels belonging to the tumor ROI reach the lower boundary of the relevant permittivity range.
Our final comment concerns the extent to which motion artifacts, which are critical for MR thermometry, impact the performance of the proposed framework. In MWI, inaccuracies introduced by movements can be modeled as additional sources of noise on the measured data. Since MW data acquisition can be performed very fast (faster than the breath rate), several measurements can be acquired and then averaged in such a way as to remove noise and achieve reliable imaging results. On the other hand, the movements of the ROI affect the co-registration of the treatment-planning image needed to implement the kernel of the inversion algorithm. However, MWI has a lower resolution than MR, which in this case turns into a sort of advantage as “small” enough motion shifts are basically not seen by the MWI device.

Author Contributions

Conceptualization, Á.Y.R., M.C., and L.C.; methodology, Á.Y.R., M.C., and L.C.; software, Á.Y.R.; validation, Á.Y.R., M.C., and L.C.; formal analysis, Á.Y.R., M.C., and L.C.; investigation, Á.Y.R.; resources, M.C. and L.C.; data curation, Á.Y.R.; writing—original draft preparation, Á.Y.R., M.C., and L.C.; writing—review and editing, Á.Y.R., M.C., and L.C.; visualization, Á.Y.R.; supervision, M.C. and L.C.; project administration, M.C. and L.C.; funding acquisition, M.C. and L.C. All authors have read and agreed to the published version of the manuscript.

Funding

This project has received funding from the European Union Horizon 2020 Research and Innovation Programme under the Marie Sklodowska-Curie Actions, Grant Agreement No. 764479.

Data Availability Statement

The data generated for the numerical simulations presented in this study are available on request from the corresponding author.

Acknowledgments

The authors would like to thank Alexandra Prokhorova and Marko Helbig for the useful interactions in the initial part of this research work.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Hahn, G.M. Hyperthermia and Cancer; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2012. [Google Scholar]
  2. Suit, H.D.; Gerweck, L.E. Potential for Hyperthermia and Radiation Therapy. Cancer Res. 1979, 39, 2290–2298. [Google Scholar] [PubMed]
  3. Rieke, V.; Butts Pauly, K. MR thermometry. J. Magn. Reson. Imaging 2008, 27, 376–390. [Google Scholar] [CrossRef] [PubMed]
  4. Winter, L.; Oberacker, E.; Paul, K.; Ji, Y.; Oezerdem, C.; Ghadjar, P.; Thieme, A.; Budach, V.; Wust, P.; Niendorf, T. Magnetic resonance thermometry: Methodology, pitfalls and practical solutions. Int. J. Hyperth. 2016, 32, 63–75. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Kok, H.P.; Cressman, E.N.; Ceelen, W.; Brace, C.L.; Ivkov, R.; Grüll, H.; Ter Haar, G.; Wust, P.; Crezee, J. Heating technology for malignant tumors: A review. Int. J. Hyperth. 2020, 37, 711–741. [Google Scholar] [CrossRef] [PubMed]
  6. Pastorino, M. Microwave Imaging; John Wiley & Sons: Hoboken, NJ, USA, 2010; Volume 208. [Google Scholar]
  7. An Introduction to Microwave Imaging for Breast Cancer Detection; Conceiç ao, R.C.; Mohr, J.J.; O’Halloran, M. (Eds.) Springer: Berlin/Heidelberg, Germany, 2016. [Google Scholar]
  8. Crocco, L.; Karanasiou, I.; James, M.L.; Conceiç ao, R.C. Emerging Electromagnetic Technologies for Brain Diseases Diagnostics, Monitoring and Therapy; Springer: Berlin/Heidelberg, Germany, 2018. [Google Scholar]
  9. Ley, S.; Schilling, S.; Fiser, O.; Vrba, J.; Sachs, J.; Helbig, M. Ultra-wideband temperature dependent dielectric spectroscopy of porcine tissue and blood in the microwave frequency range. Sensors 2019, 19, 1707. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  10. Bolomey, J.; Jofre, L.; Peronnet, G. On the Possible Use of Microwave-Active Imaging for Remote Thermal Sensing. IEEE Trans. Microw. Theory Tech. 1983, 31, 777–781. [Google Scholar] [CrossRef]
  11. Colton, D.L.; Kress, R.; Kress, R. Inverse Acoustic and Electromagnetic Scattering Theory; Springer: Berlin/Heidelberg, Germany, 1998; Volume 93. [Google Scholar]
  12. Meaney, P.M.; Fanning, M.W.; Paulsen, K.D.; Li, D.; Pendergrass, S.A.; Fang, Q.; Moodie, K.L. Microwave thermal imaging: Initial in vivo experience with a single heating zone. Int. J. Hyperth. 2003, 19, 617–641. [Google Scholar] [CrossRef]
  13. Fiser, O.; Helbig, M.; Sachs, J.; Ley, S.; Merunka, I.; Vrba, J. Microwave Non-Invasive Temperature Monitoring Using UWB Radar for Cancer Treatment by Hyperthermia. Prog. Electromagn. Res. 2018, 162, 1–14. [Google Scholar] [CrossRef] [Green Version]
  14. Mozerova, H.; Scapaticci, R.; Vrba, J.; Crocco, L. Monitoring regional hyperthermia via microwave imaging: A feasibility study. In Proceedings of the 2020 IEEE MTT-S International Microwave Biomedical Conference (IMBioC), Toulouse, France, 14–17 December 2020; pp. 1–3. [Google Scholar] [CrossRef]
  15. Altintas, G.; Akduman, I.; Janjic, A.; Yilmaz, T. A Novel Approach on Microwave Hyperthermia. Diagnostics 2021, 11, 493. [Google Scholar] [CrossRef]
  16. Chen, X.; Wei, Z.; Li, M.; Rocca, P. A review of deep learning approaches for inverse scattering problems (invited review). Prog. Electromagn. Res. 2020, 167, 67–81. [Google Scholar] [CrossRef]
  17. Dash, T.; Chitlangia, S.; Ahuja, A.; Srinivasan, A. A review of some techniques for inclusion of domain-knowledge into deep neural networks. Sci. Rep. 2022, 12, 1040. [Google Scholar] [CrossRef] [PubMed]
  18. von Rueden, L.; Mayer, S.; Beckh, K.; Georgiev, B.; Giesselbach, S.; Heese, R.; Kirsch, B.; Walczak, M.; Pfrommer, J.; Pick, A.; et al. Informed Machine Learning—A Taxonomy and Survey of Integrating Prior Knowledge into Learning Systems. IEEE Trans. Knowl. Data Eng. 2021, 35, 614–633. [Google Scholar] [CrossRef]
  19. Bertero, M. Linear Inverse and III-Posed Problems. In Advances in Electronics and Electron Physics; Academic Press: Cambridge, MA, USA, 1989; Volume 75, pp. 1–120. [Google Scholar] [CrossRef]
  20. Balanis, C.A. Advanced Engineering Electromagnetics; John Wiley & Sons: Hoboken, NJ, USA, 2012. [Google Scholar]
  21. Devaney, A.J.; Dennison, M. Inverse scattering in inhomogeneous background media. Inverse Probl. 2003, 19, 855–870. [Google Scholar] [CrossRef]
  22. Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning; MIT Press: Cambridge, MA, USA, 2016; Available online: http://www.deeplearningbook.org (accessed on 16 January 2023).
  23. Christ, A.; Kainz, W.; Hahn, E.G.; Honegger, K.; Zefferer, M.; Neufeld, E.; Rascher, W.; Janka, R.; Bautz, W.; Chen, J.; et al. The Virtual Family—Development of surface-based anatomical models of two adults and two children for dosimetric simulations. Phys. Med. Biol. 2009, 55, N23. [Google Scholar] [CrossRef] [Green Version]
  24. Ruggeri, A. Development and Multiphysic Analysis of a Neck Phantom for Microwave Hyperthermia. In Proceedings of the 2020 28th Telecommunications Forum (TELFOR), Belgrade, Serbia, 24–25 November 2020; pp. 1–4. [Google Scholar] [CrossRef]
  25. Hasgall, P.; Di Gennaro, F.; Baumgartner, C.; Neufeld, E.; Lloyd, B.; Gosselin, M.; Payne, D.; Klingenböck, A.; Kuster, N. IT’IS Database for Thermal and Electromagnetic Parameters of Biological Tissues, Version 4.0. IT’IS 2018. Available online: https://itis.swiss/virtual-population/tissue-properties/database/dielectric-properties/ (accessed on 20 January 2023).
  26. Huang, S.; Cai, W.; Han, S.; Lin, Y.; Wang, Y.; Chen, F.; Shao, G.; Liu, Y.; Yu, X.; Cai, Z.; et al. Differences in the dielectric properties of various benign and malignant thyroid nodules. Med. Phys. 2021, 48, 760–769. [Google Scholar] [CrossRef]
  27. Harrington, R. Origin and development of the method of moments for field computation. IEEE Antennas Propag. Mag. 1990, 32, 31–35. [Google Scholar] [CrossRef]
  28. Richmond, J. Scattering by a dielectric cylinder of arbitrary cross section shape. IEEE Trans. Antennas Propag. 1965, 13, 334–341. [Google Scholar] [CrossRef]
  29. Catedra, M.F.; Gago, E.; Nuno, L. A numerical scheme to obtain the RCS of three-dimensional bodies of resonant size using the conjugate gradient method and the fast Fourier transform. IEEE Trans. Antennas Propag. 1989, 37, 528–537. [Google Scholar] [CrossRef]
  30. Dachena, C.; Fedeli, A.; Fanti, A.; Lodi, M.B.; Pastorino, M.; Randazzo, A. Microwave Imaging for the Diagnosis of Cervical Diseases: A Feasibility Analysis. IEEE J. Electromagn. RF Microw. Med. Biol. 2021, 5, 277–285. [Google Scholar] [CrossRef]
  31. Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
  32. Taha, A.A.; Hanbury, A. Metrics for evaluating 3D medical image segmentation: Analysis, selection, and tool. BMC Med. Imaging 2015, 15, 29. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  33. Matthews, B.W. Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochim. Biophys. Acta (BBA)-Protein Struct. 1975, 405, 442–451. [Google Scholar] [CrossRef]
  34. Pennes, H.H. Analysis of Tissue and Arterial Blood Temperatures in the Resting Human Forearm. J. Appl. Physiol. 1948, 1, 93–122. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Typical MWI setup configuration. A number of transmitting T x [ v ] and receiving R x [ m ] antennas surround the imaging domain Ω .
Figure 1. Typical MWI setup configuration. A number of transmitting T x [ v ] and receiving R x [ m ] antennas surround the imaging domain Ω .
Cancers 15 01717 g001
Figure 2. The basic ingredients of the proposed DL-MWI framework for temperature monitoring during HT treatments.
Figure 2. The basic ingredients of the proposed DL-MWI framework for temperature monitoring during HT treatments.
Cancers 15 01717 g002
Figure 3. Convolutional neural network diagram for classification problems.
Figure 3. Convolutional neural network diagram for classification problems.
Cancers 15 01717 g003
Figure 4. DL-MWI framework for monitoring temperature in neck tumor HT. Two separate CNNs simultaneously check the temperature status of the treated tumor and the spinal cord. In the figure, Ω 1 denotes the tumor region and Ω 2 is the spinal cord to be preserved.
Figure 4. DL-MWI framework for monitoring temperature in neck tumor HT. Two separate CNNs simultaneously check the temperature status of the treated tumor and the spinal cord. In the figure, Ω 1 denotes the tumor region and Ω 2 is the spinal cord to be preserved.
Cancers 15 01717 g004
Figure 5. The phantom adopted for the numerical study. Tissue segmentation (left) and electromagnetic properties (right).
Figure 5. The phantom adopted for the numerical study. Tissue segmentation (left) and electromagnetic properties (right).
Cancers 15 01717 g005
Figure 6. Confusion matrix for the validation samples of the tumor CNN.
Figure 6. Confusion matrix for the validation samples of the tumor CNN.
Cancers 15 01717 g006
Figure 7. Categorical cross-entropy optimization of the considered CNNs versus the number of epochs.
Figure 7. Categorical cross-entropy optimization of the considered CNNs versus the number of epochs.
Cancers 15 01717 g007
Table 1. DL-MWI Tumor CNN performance assessment.
Table 1. DL-MWI Tumor CNN performance assessment.
ClassDSCMCC
Unheated0.9530.928
Therapeutic0.9320.887
Hot0.9120.878
Table 2. DL-MWI Spinal cord CNN performance assessment.
Table 2. DL-MWI Spinal cord CNN performance assessment.
ClassDSCMCC
Unheated0.9200.855
Hot0.9070.855
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Yago Ruiz, Á.; Cavagnaro, M.; Crocco, L. Hyperthermia Treatment Monitoring via Deep Learning Enhanced Microwave Imaging: A Numerical Assessment. Cancers 2023, 15, 1717. https://doi.org/10.3390/cancers15061717

AMA Style

Yago Ruiz Á, Cavagnaro M, Crocco L. Hyperthermia Treatment Monitoring via Deep Learning Enhanced Microwave Imaging: A Numerical Assessment. Cancers. 2023; 15(6):1717. https://doi.org/10.3390/cancers15061717

Chicago/Turabian Style

Yago Ruiz, Álvaro, Marta Cavagnaro, and Lorenzo Crocco. 2023. "Hyperthermia Treatment Monitoring via Deep Learning Enhanced Microwave Imaging: A Numerical Assessment" Cancers 15, no. 6: 1717. https://doi.org/10.3390/cancers15061717

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop