Toward Precision Radiotherapy: A Nonlinear Optimization Framework and an Accelerated Machine Learning Algorithm for the Deconvolution of Tumor-Infiltrating Immune Cells

Abstract

Tumor-infiltrating immune cells (TIICs) form a critical part of the ecosystem surrounding a cancerous tumor. Recent advances in radiobiology have shown that, in addition to damaging cancerous cells, radiotherapy drives the upregulation of immunosuppressive and immunostimulatory TIICs, which in turn impacts treatment response. Quantifying TIICs in tumor samples could form an important predictive biomarker guiding patient stratification and the design of radiotherapy regimens and combined immune-radiation treatments. As a result of several limitations associated with experimental methods for quantifying TIICs and the availability of extensive gene sequencing data, deconvolution-based computational methods have appeared as a suitable alternative for quantifying TIICs. Accordingly, we introduce and discuss a nonlinear regression approach (remarkably different from the traditional linear modeling approach of current deconvolution-based methods) and a machine learning algorithm for approximating the solution of the resulting constrained optimization problem. This way, the deconvolution problem is treated naturally, given that the gene expression levels of pure and heterogenous samples do not have a strictly linear relationship. When applied across transcriptomics datasets, our approach, which also allows the coupling of different loss functions, yields results that closely match ground-truth values from experimental methods and exhibits superior performance over popular deconvolution-based methods.

1. Introduction

In recent times, the continuous rise in the global cancer burden has emphasized the need for increased efforts in cancer treatment strategies. In 2020 alone, there were 19.3 million new cases and almost 10 million cancer deaths worldwide, with the number of new cases projected to climb to 28.4 million by 2040 [1]. This projection is not far-fetched, given the 36.9% increase in the number of new cases from 2012 to 2020 [2]. With up to half of all these cancer cases receiving radiotherapy during their treatment, radiotherapy is still a vital cancer treatment strategy [3,4,5].

In simple terms, radiotherapy involves using ionizing radiation, usually high-energy X-rays, to kill cancer cells or, at least, limit their proliferation by damaging their genetic code of life, known as deoxyribonucleic acid (DNA). In doing this, the goal is to achieve tumor control without introducing severe damage to surrounding normal tissues, enhancing treatment outcomes and minimizing adverse effects. Precision radiotherapy aims to reach this goal by stratifying and precisely treating “each individual cancer patient, using state-of-the-art new radiotherapy technology and biomarkers” [6].

Biomarkers are objectively evaluated and measured characteristics indicative of normal (or abnormal) biological processes, pathogenesis, or therapeutic response [7]. Their roles could be prognostic, diagnostic, treatment response monitoring, or predictive [8]. In their predictive role, biomarkers indicate the likelihood of a therapeutic benefit from a specific treatment for a given patient. Thus, in the case of precision radiotherapy, the complementary role of biomarkers for predictive purposes evolved from recent findings, showing that the effects of radiotherapy on the tumor microenvironment (TME) may be a significant determinant of the efficacy of a radiotherapy regimen [8,9,10]. For instance, in addition to damaging the malignant part of the TME, radiotherapy has been found to trigger immunomodulatory effects and alterations to critical components of the TME, such as tumor-infiltrating immune cells (TIICs) [9,11,12,13]. The latter is of particular interest because, as radiotherapy drives the upregulation of immunostimulatory TIICs such as cytotoxic CD8+ T cells, and immunosuppressive TIICs such as regulatory T cells (Treg), its impact is felt on differing cell subsets [13]. Consequently, quantifying TIICs in pretreatment and treatment of tumor samples is crucial in identifying predictive biomarkers guiding patient stratification and designing suitable radiotherapy regimens, including combined immune-radiation treatments [14].

Traditionally, experimental methods such as immunohistochemistry (IHC), cytometry, and recently, single-cell RNA sequencing (scRNA-seq) have been the gold standard for quantifying TIICs in samples. Although these methods precisely quantify TIICs in samples, there are limitations in terms of the technicality and range of applicability associated with each method. On the one hand, scRNA-seq is not just expensive and laborious for routine use but also highly prone to bias due to variations in the dissociation efficiencies of single cells [15]. On the other hand, IHC and cytometry rely on a small number of phenotypic markers, exhibit low to medium throughput, have little or no public datasets available, and are difficult to apply in large tumor series [16]. This situation necessitates the search for suitable alternatives to quantify TIICs in tumor samples.

Recently, the sharp reduction in the cost of next-generation sequencing (NGS) technologies has encouraged its routine application in clinical settings, resulting in the availability of large amounts of transcriptomics datasets from patients’ tumor samples, such as The Cancer Genome Atlas (TCGA) [17]. Although these datasets represent the bulk tumor sample, they provide a suitable alternative for quantifying the sample TIICs using computational techniques [18]. Computational techniques serving this purpose are broadly categorized into two. The first broad category is marker gene-based methods [14,19]. Methods under this category utilize a list of genes characteristic of a cell type (called marker genes), to quantify every cell type independently from the expression levels of the marker genes in the heterogenous tumor sample. As a result, marker gene-based methods can only generate “a semi-quantitative score describing the enrichment of a cell type in a sample” [14], thus effectively making the comparison between cell types impractical.

The second broad category, deconvolution-based methods, considers the gene expression profile of the heterogeneous tumor sample as a convolution of the gene expression levels of the different cell components [20]; as a result, they can quantitatively estimate the fractions of cell types of interest (in this case TIICs). This consideration allows the problem to be formulated mathematically as a function of the gene expression profiles of the cell-type admixture. Thus, given the bulk gene expression of a tumor sample and a known cell-type specific expression profile, solving an inverse problem can estimate the cell-type fractions in the heterogeneous tumor sample.

To date, most deconvolution-based methods, including those specific to quantifying TIICs, such as CIBERSORT [21], CIBERSORTx [22], EPIC [23], ECIS [24], quanTIseq [25], and TIMER [26], assume that function to be linear. However, they yield different results for different cell types and use cases despite utilizing the gradient algorithm or its variants to approximate solutions to the inverse problem. Interestingly, this is because each method is conceptually different according to the choice of loss function (or objective functions) and the setting of the optimization problem (constrained or unconstrained). It is against this backdrop that packages such as Immunedeconv [27], TIMER2.0 [28], and TumorDecon [29] have sought to provide a unified platform that allows each of the different methods to be applied on the same dataset to compare or complement results. Accordingly, the strengths of each method can be harnessed to gain more robust and comprehensive estimates. Nevertheless, this approach is still susceptible to the potential issues associated with traditional linear modeling [30], given that “the relationship between the expression levels of pure and heterogeneous samples is not strictly linear” [14]. Moreover, dealing with large transcriptomics datasets calls for computationally efficient methods with fast rates of convergence and runtime [30].

Therefore, the main aim of this paper is to introduce and discuss a mathematical formulation that permits the TIIC deconvolution inverse problem to be handled in its natural state alongside an accelerated machine learning algorithm for approximating its solution. Through rigorous mathematical analysis, we show that the algorithm converges to an optimal solution of the inverse problem for various loss functions. More specifically, a globally optimal solution is guaranteed for convex loss functions. Furthermore, we use numerical experiments to show that the algorithm exhibits faster convergence rates and runtime than other traditionally used algorithms. When applied across transcriptomics datasets, our results closely match values from experimental methods and show superior performance over popular TIIC deconvolution-based methods. We end with a note on the detailed science behind these observations and an explanation of how this framework can be applied across similar inverse problems in biology, medical physics, and oncology.

2. Materials and Methods

2.1. Formulation and Discussion of the Deconvolution Problem

Let $N$ denote the number of different cell types forming a mixture sample, and $M$ be the number of genes whose expressions are measured in the sample. Let $B=\left(b_1, b_2\dots ,b_M\right)\in {ℝ}^M$ be the measurements of gene expression in the sample. Let $S\in {ℝ}^{M\times N}$ be the corresponding reference expressions matrix of the $M$ genes from the $N$ constituent cell types, and $P=\left(p_1, p_2\dots ,p_N\right)\in {ℝ}^N$ be the unknown proportions of mix of the different cell types. An operator $r$ can model the relationship of $B, S$, and $P$ as

$$B=r\left(S,P\right).$$

(1)

The deconvolution problem is concerned with the inverse problem of estimating $P$, given $B$ and $S$. This inverse problem can be formulated mathematically as the following equivalent constrained optimization problem:

$$\underset{P\in C}{min}\mathcal{L}\left(B,r\left(S,P\right)\right),$$

(2)

where $\mathcal{L}$ is a loss function measuring model fitness, and $C$ is the set of constraints on $P$ arising naturally from its definition as proportions, i.e.,

$$C=\left\{P\in {ℝ}^N: p_i\ge 0 \forall i=1,\dots ,N, {\displaystyle \sum }_{i=1}^N{p}_i=1 \right\}.$$

Many of the existing deconvolution methods (see, for example, [21,22,23,31,32,33,34,35] and references therein) consider $B, S$, and $P$ to be linearly related in the form

$$B=SP+e,$$

(3)

where $e$ is a random error. Several issues with the linear framework have been identified [30]. More recently, the authors of [36] showed problems associated with different scales of gene expression within the linear framework and then proposed the following hybrid model:

$$\mathit{log}\left(b_i\right)=d+\mathit{log}\left({\displaystyle \sum }_{j=1}^N{S}_{ij}{p}_j\right)+e_i,$$

(4)

where $d$ accounts for systemic technical variation. Equivalently,

$$B\approx exp\left(d\right)SP.$$

(5)

We note that $\mathit{exp}\left(d\right)$ in Equation (5) is a constant factor adjustment across all genes. However, more than this constant factor adjustment may be required, because such systemic technical variations affect genes differently [37]. Thus, one may consider a more general model of the form

$$B\approx DSP,$$

(6)

where $D$ is a diagonal matrix with diagonal entries as gene-specific factor adjustment generated from some known distribution. Accordingly, Equations (3) and (5) become special cases of Equation (6). Nevertheless, such a linear transformation may not efficiently describe nonlinear patterns.

Consequently, we formulate a generic nonlinear framework for the deconvolution problem by considering the operator $r$ in Equation (1) as a nonlinear transformation involving$S$ and P. This is because generalized nonlinear regression methods have been shown to yield better prediction accuracy for complex nonlinear patterns, where traditional linear regression models may fail [38]. Specifically, our nonlinear operator $r$ is given as

$$r_i={\left(\delta +{\displaystyle \sum }_{j=1}^N{S}_{ij}{p}_j\right)}^{\theta }, \delta ,\theta >0.$$

(7)

The nonlinearity of Equation (7) depends on the value of $\theta$. Note that Equation (7) reduces to the linear framework for $\theta =1$. Therefore, for an arbitrary sequencing dataset, it is worthwhile to determine what values of $\theta$ (different from one) closely describe the not strictly linear relationship of the gene expression profiles of pure and heterogeneous samples.

Rigorous data assimilation techniques are useful in making such determinations from arbitrary datasets. However, we refrain from such nontrivial rigorous analytical examinations as they are beyond the aims of this work. As a result, we choose our $\theta$ values using an empirical approach for the purpose of demonstrating the proof of principle which is the subject of this work. The empirical approach is based on our hypothesis that, for some $\theta \in \left(0,1\right)\cup \left(1,2\right)$, we may be able to get suitable values satisfying the description of the relationship between the gene expression profiles of pure and heterogeneous samples. More specifically, we hypothesize that such a value might be slightly less than one or slightly greater than one on the order of a few decimal places. This hypothesis is guided by our preferred interpretation of the expression “not strictly linear”.

2.2. Review of Some Commonly Used Loss Functions

Several research surveys of cell-type deconvolution methods (see, for example, [14,19,39]) have identified the quadratic/squared error loss and the $\epsilon$-insensitive loss, as the most commonly used loss functions for reference-based cell-type deconvolution within the linear framework. On the one hand, the squared error loss (SEL) is formulated on the basis of squared deviations as follows:

$$\mathcal{L}\left(B,r\left(S,P\right)\right)={\displaystyle \sum }_{i=1}^M{\left(b_i-S_{i}P\right)}^2=B-S{P}_2^2,$$

(8)

where $S_i$ denotes the i-th row of $S$. In fact, it is the most common choice of loss function due to its simplicity. Notably, the SEL is highly susceptible to outliers. This feature can be especially beneficial when the outliers originate naturally from variations within the process and, as such, contain useful systemic information. Conversely, it can be a drawback when the outliers arise from noise (errors).

On the other hand, the $\epsilon$-insensitive loss (also referred to as the support vector method) [40] and other robust techniques, such as the Huber and Laplacian losses, are employed to reduce the drawback of the SEL. A unified version of these robust techniques was introduced in [41] as a soft insensitive loss function (SILF), expressed mathematically as

$$\mathcal{L}\left(B,r\left(S,P\right)\right)={\displaystyle \sum }_{i=1}^M{\mathcal{l}}_i\left(b_i-S_{i}P\right),$$

(9)

where

$${\mathcal{l}}_i\left(a\right)=\left\{\begin{array}{c}-a-\epsilon , if a\in \left(-\infty ,-\left(1+\rho \right)\epsilon \right)\\ \begin{array}{c}\frac{{\left(a+\left(1-\rho \right)\epsilon \right)}^2}{4\rho \epsilon }, if a\in \left[-\left(1+\rho \right)\epsilon ,-\left(1-\rho \right)\epsilon \right] \\ 0, if a\in \left(-\left(1-\rho \right)\epsilon ,\left(1-\rho \right)\epsilon \right)\end{array}\\ \begin{array}{c}\frac{{\left(a-\left(1-\rho \right)\epsilon \right)}^2}{4\rho \epsilon }, if a\in \left[\left(1-\rho \right)\epsilon ,\left(1+\rho \right)\epsilon \right]\\ a-\epsilon , if a\in \left(\left(1+\rho \right)\epsilon ,+\infty \right)\end{array}\end{array},\right.$$

with $0<\rho \le 1$ and $\epsilon >0$. In [41], the authors remarked that this function (Equation (9)) is smooth and inherits most of the desirable characteristics of several robust techniques, including insensitivity to outliers. They further demonstrated in great detail the computational efficiency and competitiveness of SILF compared to other well-respected techniques. Remarkably, these loss functions have been a dominating paradigm in the deconvolution problem, mainly due to their convexity.

Nevertheless, it has been shown in recent times that nonconvex loss functions improve the generic applicability and robustness of learning, especially in situations where the data and noise distributions are unknown [42]. One such loss function is given in Equation (10) as a Cauchy kernel risk-sensitive loss (CKRSL), derived using a Gaussian kernel-adapted operator and following methods similar to those in [43,44].

$$\mathcal{L}\left(B,r\left(S,P\right)\right)=\frac{1}{M}{\displaystyle \sum }_{i=1}^M\beta \mathit{log}\left[1+2\left(\frac{1-\exp \left(-{\left(\frac{{{\displaystyle \sum }}_{j=1}^N{S}_{ij}{p}_j-b_i}{2\sigma }\right)}^2\right)}{\beta }\right)\right], \sigma >0.$$

(10)

We remark that the abstract formulation presented in Section 2.1 has the advantage of accommodating several loss functions including those reviewed in this subsection.

2.3. Specification of the Loss Function for this Study

For the present study, we use a weighted squared error loss (WSEL) on the relationship operator $r$ defined by Equation (7), expressed as

$$\mathcal{L}\left(B,r\left(S,P\right)\right)=\frac{1}{M}{\displaystyle \sum }_{i=1}^M{\left({\left(\delta +{\displaystyle \sum }_{j=1}^N{S}_{ij}{p}_j\right)}^{\theta }-b_i\right)}^2.$$

(11)

2.4. Accelerated Machine Learning Algorithm (AMLA)

An optimal solution to the constrained optimization problem of Equation (2) can be approximated by the following accelerated machine learning algorithm (AMLA):

$$\left\{\begin{array}{c}{v}_0,v_1 \in C\\ w_j=v_j+{\alpha }_j\left(v_{j-1}-v_j\right)\\ v_{j+1}={\wp }_C\left(w_j-\lambda A{v}_j\right)\end{array},\right.$$

(12)

for some predefined values of the adaptive momentum parameter ${\alpha }_j$ and step size (or learning rate) $\lambda$. $A$ denotes the gradient of the desired loss function $\mathcal{L}$, and ${\wp }_C$ is the projection operator on $C$. The projection operator ${\wp }_C$ locates a point in $C$ having the least distance to a given point, while $v_0,v_1$are initialization points.

AMLA converges to a solution of the deconvolution problem (Equation (2)) for various loss functions, including those reviewed in Section 2.2. A detailed mathematical analysis establishing this convergence is presented in Appendix A, starting with the preliminary mathematical tools, as well as the lemmas, theorems, and their proofs.

For our choice of loss function (Equation (11)), the gradient $A$ is the vector defined by

$$A_j=\frac{2\theta }{M}{\displaystyle \sum }_{i=1}^M{S}_{ij}\left({\left(\delta +{\displaystyle \sum }_{k=1}^N{S}_{ik}{p}_k\right)}^{\theta }-b_i\right){\left(\delta +{\displaystyle \sum }_{k=1}^N{S}_{ik}{p}_k\right)}^{\theta -1}.$$

(13)

Furthermore, the control parameters ${\alpha }_j$ and $\lambda$ are determined by a Lipschitz constant ($K)$ of $A$(See Theorem 12 in Appendix A). For $A$ defined by Equation (13), a suitable $K$ can be given by

$$K=\left\{\begin{array}{c}N\left[\frac{2{\theta }^2{s}_{\mathit{max}}^2}{{\delta }^{2(1-\theta )}}+\frac{2\theta |\theta -1|s_{\mathit{max}}^2\left[{\left(N{s}_{\mathit{max}}+\delta \right)}^{\theta }+b_{\mathit{max}}\right]}{{\delta }^{2-\theta }}\right],\mathit{for}\theta \le 1\\ N\left[2{\theta }^2{s}_{\mathit{max}}^2{\left(N{s}_{\mathit{max}}+\delta \right)}^{2(\theta -1)}+\frac{2\theta |\theta -1|s_{\mathit{max}}^2\left[{\left(N{s}_{\mathit{max}}+\delta \right)}^{\theta }+b_{\mathit{max}}\right]}{{\delta }^{2-\theta }}\right]\mathit{for}1<\theta \le 2\\ N\left[2{\theta }^2{s}_{\mathit{max}}^2{\left(N{s}_{\mathit{max}}+\delta \right)}^{2(\theta -1)}+2\theta |\theta -1|s_{\mathit{max}}^2\left[{\left(N{s}_{\mathit{max}}+\delta \right)}^{\theta }+b_{\mathit{max}}\right]{\left(N{s}_{\mathit{max}}+\delta \right)}^{\theta -2}\right],\mathit{for}\theta >2\end{array},\right.$$

(14)

where $s_{max}=\underset{\begin{array}{c}1\le i\le M\\ 1\le j\le N\end{array}}{\mathit{max}}\left|S_{ij}\right|, b_{max}=\underset{1\le i\le M}{\mathit{max}}{b}_i.$

Lastly, the projection operator on $C$ denoted by ${\wp }_C$ is computed using the alternating projections method [45].

2.5. Validation Datasets

Our validation datasets, which come from published findings in [22,23], consisted of experimentally measured immune cell-type proportions from tumor samples, bulk RNA sequencing data of tumor samples, and a gene expression reference profile. The gene expression reference profile was a signature matrix of eight Melanoma subsets derived from scRNA-seq (SMART-Seq2) (see Supplementary Table S2e in [22]). These melanoma subsets are listed across 3121 genes. They include five TIICs (B cells, CD8 T cells, CD4 T cells, NK cells, and macrophages) and other cell subsets, including endothelial cells, malignant cells, and cancer-associated fibroblasts (CAFs).

Experimentally measured immune cell-type proportions from tumor samples and bulk RNA sequencing data of tumor samples were obtained from [23]. In [23], the authors divided the single-cell suspensions collected from the lymph nodes of four patients with metastatic melanoma into two portions. For one portion, flow cytometry was used to measure the percentages of live cells, including four TIICs (B cells, CD4 T cells, CD8 T cells, and NK cells), malignant cells, and other cells made up of primarily stromal and endothelial cells (see Supplementary Table S3A in [23]). The fractions of each of these cell types for each patient are presented in

Table 1. Fractions of cell types measured using flow cytometry for the lymph nodes of metastatic melanoma patients (modified from Supplementary Table S3A of [23]).

Patient ID	B Cells	CD4 T Cells	CD8 T Cells	NK Cells	Malignant Cells	Other Cells 1
LAU125	0.1812	0.0082	0.0035	0.0050	0.6803	0.1218
LAU355	0.3248	0.2315	0.0582	0.0017	0.0006	0.3832
LAU1255	0.0579	0.0276	0.0376	0.0017	0.3756	0.4997
LAU1314	0.4667	0.1815	0.0454	0.0025	0.0007	0.3031

¹ These consist mostly of stromal (for example, cancer-associated fibroblasts (CAFs)) and endothelial cells.

below. We refer to these cell-type fractions as the ground-truth values from the experiment (GTVEs).

The other portion of the single-cell suspensions was used for bulk RNA sequencing (RNA-seq). We downloaded this RNA-seq data for the four Melanoma patients from the “example data from EPIC” link on the EPIC web application (http://epic.gfellerlab.org) accessed on 28 September 2022. It can also be accessed from the Gene Expression Omnibus (GEO) repository [46] through the accession number GSE93722. This RNA-seq data consists of 49,902 genes quantified in transcripts per million (TPM) for each of the four melanoma samples.

2.6. Deconvolution Workflow

Our deconvolution workflow consists of partly sequential steps necessary to achieve efficient and accurate estimation of TIICs from bulk RNA-seq data. As illustrated in Figure 1, the input data comprise a tab-delimited text file of bulk RNA-seq samples and gene reference profiles. These inputs are first processed using a simple data filter algorithm, which identifies the genes common to both inputs and passes values of these genes in each input data across the filter. These values are then fed into the respective variables of our nonlinear framework. After that, the cell fractions per sample are computed using AMLA. We remark that suitable values of the parameters θ and δ can be estimated using a rigorous non-trivial pattern analysis of the bulk RNA-Seq data, as indicated in Figure 1, although no attempt was made in this direction in the present work.

2.7. Software Used

We implement the deconvolution workflow described in Figure 1 as a Python package. The package was developed using the IDE (Integrated Development Environment) PyCharm Community Edition 2021.2.1 version 212.5080.64 created by JetBrains s.r.o, Prague, Czech Republic, running Python 3.9.7 (64 bit) version 3.9.7150.0. The package contains three custom-made modules: the filtering algorithm, our nonlinear framework, and AMLA. The Pandas library was used to manipulate the import of sequencing data and reference profiles and export of estimated fractions for visualization.

To compare our method with two popular cell-type deconvolution methods, we also generated results from CIBERSORTx [22] and EPIC [23] using the web application versions of their software available at https://cibersortx.stanford.edu/ accessed on 30 September 2022 and http://epic.gfellerlab.org accessed on 28 September 2022, respectively. We performed all these software activities on an Intel^® Core™ i5-6300U CPU @at 2.40GHz with 8 GB RAM on a 64 bit Windows 10 Pro operating system.

3. Results

3.1. Estimating Cell-Type Fractions in Four Melanoma Samples Using Our Nonlinear Framework, EPIC, and CIBERSORTx

We considered the bulk RNA-Seq dataset of four melanoma patients and the gene reference profile containing eight cell subsets, as described in Section 2.5. By inputting tab-delimited text files of these two datasets into the filtering algorithm, we obtained 2928 genes common to both datasets. The values for these specific genes in the respective datasets were passed across the filtering algorithm into the nonlinear framework. We estimated eight cell-type fractions present in each of the four melanoma samples using four different versions of our nonlinear framework. These versions, named according to the value of the hyperparameter $\theta$ in Equation (11) and the procedure for applying AMLA, include those described below.

1.

Equivalent linear model (ELM), expressed for $\theta =1$, such that Equation (11) becomes

$$\mathcal{L}\left(B,r\left(S,P\right)\right)=\frac{1}{M}{\displaystyle \sum }_{i=1}^M{\left(\left(\delta +{\displaystyle \sum }_{j=1}^N{S}_{ij}{p}_j\right)-b_i\right)}^2.$$

(15)

AMLA is then applied to approximate the solution.

2.

Linearized nonlinear model (LNM), expressed for $\theta =0.92$, such that Equation (11) becomes

$$\mathcal{L}\left(B,r\left(S,P\right)\right)=\frac{1}{M}{\displaystyle \sum }_{i=1}^M{\left({\left(\delta +{\displaystyle \sum }_{j=1}^N{S}_{ij}{p}_j\right)}^{0.92}-b_i\right)}^2 .$$

(16)

However, we do not apply AMLA directly to Equation (16). Rather, we linearize it to obtain the form

$$\mathcal{L}\left(B,r\left(S,P\right)\right)=\frac{1}{M}{\displaystyle \sum }_{i=1}^M{\left(\left(\delta +{\displaystyle \sum }_{j=1}^N{S}_{ij}{p}_j\right)-{\left(b_i\right)}^{\frac{1}{0.92}}\right)}^2,$$

(17)

and thereafter apply AMLA to approximate the solution.

3.

Nonlinear model one (NM1), expressed for $\theta =0.92$, such that we obtain Equation (16) above, and then apply AMLA to approximate the solution.

4.

Nonlinear model two (NM2), expressed for $\theta =1.08$, such that Equation (11) becomes

$$\mathcal{L}\left(B,r\left(S,P\right)\right)=\frac{1}{M}{\displaystyle \sum }_{i=1}^M{\left({\left(\delta +{\displaystyle \sum }_{j=1}^N{S}_{ij}{p}_j\right)}^{1.08}-b_i\right)}^2.$$

(18)

AMLA is then applied to approximate the solution.

For all four versions enumerated above, we set the variable $\mathsf{\delta }$, such that $\mathsf{\delta }=1$. Moreover, we initialized AMLA with distinct set values of $v_0$and $v_1$, creating a cartesian product for the patient series.

Table 2. Hyperparameter values for named versions of our nonlinear framework (Equation (11)).

Version	$\mathsf{\theta }$	$\mathsf{\delta }$
ELM	1.00	1.00
LNM	0.92	1.00
NM1	0.92	1.00
NM2	1.08	1.00

summarizes the hyperparameter values $\mathsf{\delta }$ and $\theta$ chosen for the named versions of our nonlinear framework.

We have already emphasized that the nonlinearity of our model (Equation (11)) strictly depends on the value of $\theta$, which must be different from one. Furthermore, we remarked that the selection and tuning procedure for this parameter can be achieved analytically on the basis of the input gene sequencing datasets using rigorous data assimilation techniques. However, our primary goal in this work was to demonstrate that a nonlinear regression approach for the TIIC deconvolution problem could yield significantly more accurate estimates of the fractions of cell types, including TIICs from the bulk gene expression data of tumor samples. Thus, we favored an empirical approach for selecting the hyperparameter $\theta$, in line with this goal.

Our empirical approach relies on the observation that the relationship between the gene expression profiles of pure and heterogeneous samples is not strictly linear. Guided by this, we interpret the expression “not strictly linear” as slightly different from one, and this difference can be either side of one, i.e., greater than or less than one. Because $\theta$ is strictly positive, we looked at $\theta \in \left(0,1\right)\cup \left(1,+\infty \right)$. Guided by our hypothesis presented in Section 2.1, we randomly selected 0.92 from the interval $\left(0,1\right)$. We also selected 1.08 from the interval $\left(1,+\infty \right)$ by considering the symmetric distance of the previous selection from one. The hyperparameter $\mathsf{\delta }$ is a smoothing parameter. The literature is filled with several rigorous techniques for smoothing parameter estimation. Again, for the same reasons as in $\theta$, we chose to assume a default value of one.

We present two results when estimating the cell-type fractions in the four melanoma samples using EPIC. The first result which we denote as “EPIC1” was obtained from the default setting of the EPIC web application (http://epic.gfellerlab.org) accessed on 30 September 2022. The default setting comprises tab-delimited inputs of bulk RNA-seq dataset as described in Section 2.5. Furthermore, it includes a reference profile of seven cell subsets (B cells, CAFs, CD4 T cells, CD8 T cells, endothelial, macrophages, and NK cells), built from tumor-infiltrating cells from TPM normalized scRNA-seq (see Supplementary Table S2A in [23]). This reference profile contains 23,684 genes. The second result, which we denote as “EPIC2”, was obtained using the datasets as described in Section 2.5.

Furthermore, we estimated the cell-type fractions in the four melanoma samples using the CIBERSORTx web application (https://cibersortx.stanford.edu/) accessed on 30 September 2022. The gene expression reference profile used was as described in Section 2.5. Here, we present two results for our cell-type fractions estimation using CIBERSORTx. The first result, which we denote as “CIBERSORTx1”, was obtained by checking the batch correction box, selecting B-mode, and then running CIBERSORTx, after the tab-delimited bulk RNA-seq and reference profile files were uploaded. The second result, which we denote as “CIBERSORTx2”, was obtained following the same procedure as in the first result, with the only exception being to uncheck the batch correction box. In both estimations using CIBERSORTx, quantile normalization was disabled, and the permutation for significance analysis was set at 100. We present these results in

Table 3. Fractions of cell types estimated using deconvolution methods for the lymph nodes of metastatic melanoma patients.

Patient ID	Deconvolution Method	B Cells	CD8 T Cells	CD4 T Cells	NK Cells	Macrophages	Endothelial Cells	CAF	Malignant Cells
LAU125	ELM	0.0139	0.0450	0.0714	0.0376	0.0389	0.0628	0.1175	0.6129
	LNM	0.0009	0.0170	0.0283	0.0120	0.0872	0.0468	0.1165	0.6834
	NM1	0.1884	0.0063	0.0000	0.0158	0.0684	0.0000	0.0616	0.6594
	NM2	0.1884	0.0063	0.0000	0.0158	0.0684	0.0000	0.0616	0.6594
	EPIC1 *	0.0101	0.0095	0.0303	0.0000	0.0120	0.0253	0.0003	-
	EPIC2 **	0.0000	0.0000	0.0000	0.0000	0.0258	0.0161	0.1087	0.2987
	CIBERSORTx1	0.0015	0.0000	0.0250	0.0000	0.0175	0.0002	0.0063	0.9494
	CIBERSORTx2	0.0000	0.0000	0.0107	0.0000	0.0120	0.0000	0.0135	0.9638
LAU355	ELM	0.1596	0.1154	0.3119	0.0264	0.0865	0.1008	0.0970	0.1023
	LNM	0.2558	0.1085	0.2828	0.0049	0.2287	0.0680	0.0331	0.0182
	NM1	0.3221	0.0836	0.2299	0.0084	0.0821	0.2122	0.0616	0.0000
	NM2	0.3220	0.0837	0.2300	0.0084	0.0821	0.2122	0.0616	0.0000
	EPIC1 *	0.4540	0.0182	0.2672	0.0000	0.0086	0.0000	0.0001	-
	EPIC2 **	0.1834	0.0000	0.4528	0.0000	0.1058	0.0034	0.0000	0.0000
	CIBERSORTx1	0.5550	0.0000	0.3536	0.0104	0.0794	0.0000	0.0000	0.0017
	CIBERSORTx2	0.5896	0.0000	0.3297	0.0065	0.0741	0.0000	0.0000	0.0000
LAU1255	ELM	0.0383	0.1102	0.1270	0.0390	0.0510	0.0981	0.1070	0.4292
	LNM	0.0346	0.0981	0.0985	0.0108	0.1111	0.0429	0.0610	0.5431
	NM1	0.0589	0.0521	0.0342	0.0068	0.1042	0.2974	0.1089	0.3373
	NM2	0.0589	0.0521	0.0342	0.0069	0.1042	0.2974	0.1089	0.3374
	EPIC1*	0.0411	0.1299	0.0583	0.0000	0.0197	0.0000	0.0001	-
	EPIC2**	0.0148	0.0563	0.1094	0.0000	0.0487	0.0138	0.0216	0.4628
	CIBERSORTx1	0.0493	0.0987	0.1059	0.0000	0.1360	0.0003	0.0035	0.6062
	CIBERSORTx2	0.0329	0.0993	0.0707	0.0000	0.1490	0.0002	0.0018	0.6462
LAU1314	ELM	0.1773	0.1040	0.2823	0.0343	0.0898	0.0890	0.0978	0.1257
	LNM	0.2872	0.1026	0.2519	0.0089	0.2433	0.0515	0.0284	0.0261
	NM1	0.4436	0.0452	0.1695	0.0032	0.0095	0.2506	0.0658	0.0126
	NM2	0.4436	0.0453	0.1695	0.0032	0.0094	0.2506	0.0658	0.0126
	EPIC1 *	0.6760	0.0181	0.0790	0.0042	0.0015	0.0000	0.0000	-
	EPIC2 **	0.2040	0.0001	0.4244	0.0000	0.1057	0.0045	0.0000	0.0000
	CIBERSORTx1	0.6183	0.0000	0.3229	0.0062	0.0207	0.0000	0.0000	0.0318
	CIBERSORTx2	0.6593	0.0000	0.3082	0.0099	0.0109	0.0000	0.0000	0.0115

* The reference profile does not contain malignant cells. A column labeled as “other cells” is included in the results with the values 0.9127, 0.2519, 0.7510, and 0.2212 recorded for LAU125, LAU355, LAU1255, and LAU1314, respectively. ** The results also include a column for “other cells” with values 0.5507, 0.2546, 0.2726, and 0.2613 recorded for LAU125, LAU355, LAU1255, and LAU1314, respectively.

.

It is easy to notice that many values for the deconvolution methods NM1 and NM2 were identical across Table 3. Although NM1 and NM2 had different $\theta$ values of 0.92 and 1.08, respectively, both $\theta$ values had a symmetric distance of 0.08 about 1.00. This observation directly suggests that the scalar $\theta$ (for $\theta \ne 1$) in our nonlinear framework exhibits some symmetry about one, such that we can expect identical results for two choices of $\theta$ with the same symmetrical distance about 1.00.

3.2. Estimated vs. Experimentally Measured Cell-Type Fractions in the Four Melanoma Samples

We directly compare the estimated cell-type fractions in the four Melanoma samples presented in Table 3 with the ground-truth values from experiment (GTVE) presented in Table 1. To be able to do this, we ignored the EPIC1 results (due to the absence of malignant fractions) and NM2 results (as they are almost identical to NM1 results). Then, we aggregated the fractions—macrophages, endothelial cells, and CAFs—together as other cells since they satisfied the definition of other cells, as in Table 1. For EPIC2, we also added the values of “other cells” specified in the footnote below Table 3. The comparison is readily visualized in Figure 2.

It is clear from Figure 2 that estimates from NM1 most closely matched the GTVE across all four samples, as indicated by the close resemblance of NM1 and GTVE stacks in all four samples. Even so, stacks from other versions of our nonlinear framework (ELM and LNM) appeared to more closely resemble the GTVE stacks in all four samples than the stacks from EPIC and CIBERSORTx.

However, to quantitatively describe the extent of these resemblances, we considered a general-purpose error metric known as the root-mean-squared error (RMSE). RMSE is excellent for comparing the prediction error of different models for a specific variable. As a result, it is an incredibly good measure of model accuracy. The most accurate model would have an RMSE of zero, which is far from possible. Therefore, the model accuracy is determined by how close the RMSE is to zero, although this determination is made relative to the values of the observations or predictions.

Here, we calculated the RMSE for NM1, LNM, ELM, CIBERSORTx1, CIBERSORTx2, and EPIC2 for all cell types of the four melanoma samples and then for TIICs only. In the latter, we also included the RMSE calculation for EPIC2. We present these results in Figure 3. We calculated the RMSE values using the formula

$$\mathrm{RMSE}=\sqrt{\frac{{{\displaystyle \sum }}_{i=1}^N{\left(x_i-{\widehat{x}}_i\right)}^2}{\mathrm{N}}},$$

(19)

where $N$ is the total number of analyzed cell-type subsets, $x_i$ is the GTVE of the analyzed cell-type subset, and ${\widehat{x}}_i$ is the estimated value of the analyzed cell-type subset from a deconvolution method. Values from Table 1 and Table 3 were utilized in these calculations.

NM1 can be seen to have the lowest RMSE in both charts of Figure 3. Remarkably, its RMSE value was significantly lower than that of all the other deconvolution methods compared and was extremely close to zero, being in the range of 0–0.02. This indicates that NM1 outperformed the other deconvolution methods and gave more accurate estimates that closely matched GTVE. The accuracy of NM1 can be directly attributed to the nonlinear framework used and the choice of the value of $\theta$. Thus, we validated our hypothesis that a selection of $\theta$ slightly greater than or less than one on the order of a few decimal places accurately captures the not strictly linear sense of the relationship between the bulk gene expression and the reference profile. By using the same reference profile in the direct comparison of the deconvolution methods, we successful limited any chance that the results are a direct consequence of a factor other than the setting or framework of the deconvolution problem.

4. Discussion

The linear framework has been the dominating paradigm in the computational deconvolution of TIICs and other tumor cell subsets from bulk gene expression data of tumor samples. Efforts toward improving the accuracy of cell fraction estimates from the computational deconvolution of bulk gene expression data have been centered on modifications still built around the basic linear framework. For instance, in both charts of Figure 3, CIBERSORTx1 had a slightly lower RMSE value when compared to CIBERSORTx2, indicating improved accuracy. Similar to the conclusion in [47], we attribute this improved accuracy to the batch correction effect in CIBERSORTx1, which aims to reduce data variability resulting from technical differences between samples. In another instance, in Figure 3b, EPIC1 had a lower RMSE value when compared to EPIC2, CIBERSORTx1, CIBERSORTx2, and ELM, possibly because of the use of a specific TIIC reference profile. On the other hand, in Figure 3a, ELM, a linear version of our nonlinear framework, had a lower RMSE value (more accuracy) when compared to EPIC2, CIBERSORTx1, and CIBERSORTx2. A likely reason is that the computational algorithm AMLA used in ELM projects directly onto the natural constraint set, which is much different from the computation in CIBERSORTx and EPIC, applied on some broadly defined constraints set, followed by a renormalization of the obtained values.

It is clear from Figure 3 that these modifications would only yield minor improvements in accuracy since they are all based on the linear framework. Notably, the RMSE values from these linear framework-based models revolved around a narrow range of 0.11–0.18 and 0.08–0.11 in Figure 3a,3b, respectively. We may not expect any result different from these ranges if we were to analyze these datasets using other deconvolution methods, for example [24,25,26,31,32,33], because they are all based on the linear model with variations being in the choice of loss function or other technical modifications.

As shown in Figure 3, the RMSE of NM1 is significantly low, and there is a remarkable difference between its RMSE range and that of the linear models. This observation demonstrates the enormous positive gains in terms of accuracy associated with modeling the deconvolution problem within a nonlinear framework, as it truly represents the natural state of the problem. We also emphasize through LNM results that any attempts to linearize the nonlinear framework before applying the solution algorithm (in this case, AMLA) would vastly diminish these positive gains. However, the outcome may still be slightly better than those from the traditional linear modeling. As evident from Figure 3, LNM RMSE values range from 0.07 to 0.10, which is significantly distant from the NM1 RMSE range (0–0.02) but much closer to the linear models’ RMSE ranges. For this reason, AMLA was especially designed to approximate solutions directly from the nonlinear framework without the need to linearize first.

Furthermore, AMLA’s design allows it to exhibit faster rates of convergence and runtime in comparison to other traditionally used algorithms in machine learning, which is highly advantageous in the event of enormous amounts of bulk gene expression data of many tumor series. Verifying these with simple numerical experiments on $ℝ$ (set of real numbers) using known loss functions employed in regression analysis is straightforward. We consider the log-hyperbolic loss, squared error loss, Cauchy loss, and$\epsilon$-insensitive loss given, for example, by Equations (20)–(23), respectively.

$$q\left(x\right)=\log \left(\cosh \left(6x-2\right)\right).$$

(20)

$$q\left(x\right)={\left(5x-4\right)}^2.$$

(21)

$$q\left(x\right)=5\log \left[1+\frac{{\left(3x-2\right)}^2}{5}\right].$$

(22)

$$q\left(x\right)=\left\{\begin{array}{c}-x-\epsilon , if x\in \left(-\infty ,-\left(1+\rho \right)\epsilon \right)\\ \begin{array}{c}\frac{{\left(x+\left(1-\rho \right)\epsilon \right)}^2}{4\rho \epsilon }, if x\in \left[-\left(1+\rho \right)\epsilon ,-\left(1-\rho \right)\epsilon \right] \\ 0, if x\in \left(-\left(1-\rho \right)\epsilon ,\left(1-\rho \right)\epsilon \right)\end{array}\\ \begin{array}{c}\frac{{\left(x-\left(1-\rho \right)\epsilon \right)}^2}{4\rho \epsilon }, if x\in \left[\left(1-\rho \right)\epsilon ,\left(1+\rho \right)\epsilon \right]\\ x-\epsilon , if x\in \left(\left(1+\rho \right)\epsilon ,+\infty \right)\end{array}\end{array}.\right.$$

(23)

For each of the Equations (20)–(23), we approximated their solutions using AMLA, the well-known classical gradient descent algorithm (CGDA), and Nesterov’s accelerated gradient (NAG) widely used in machine learning. Since the optimal solutions of the equations are known, we measured the convergence to the solutions from AMLA, CGDA, and NAG using variations of ${\left(x_n-\overline{x}\right)}_{n\in \mathbb{N}}$, where $x_n$ is the labeling obtained at the n-th iteration, and $\overline{x}$ is the optimal solution. We plot this measure of convergence against the number of iterations in Figure 4.

From Figure 4, we can observe that AMLA converged in a significantly fewer number of iterations than CGDA and NAG, for all four loss functions considered. The implication of this observation is that AMLA has a higher order of convergence and, consequently, a faster rate of convergence since the “order of convergence defines the rate of convergence” [48]. Furthermore, Figure 4 affirms the robustness of AMLA. A robust algorithm is one that is theoretically guaranteed to converge, “starting from any initial design estimate” [49], “such that their correctness is not destroyed by round-off errors” [50]. We show in Appendix A, using rigorous mathematical analysis, that AMLA is guaranteed to converge to a solution of the constrained optimization problem (Equation (2)) for a variety of loss functions satisfying the stated conditions. The four loss functions presented in Figure 4 satisfy the stated conditions, thus leading to their convergence in Figure 4, even when CGDA and NAG did not converge (see Figure 4a,c). Moreover, as shown in Figure 4a–d, once AMLA converged, it maintained the flat zero line, which highlights its gross insensitivity to round-off errors, thus affirming its high robustness.

A very noteworthy affirmation of the high robustness of AMLA can be seen in Figure 4d. The $\epsilon$-insensitive loss introduces extra parameters of $\rho$ and $\epsilon$ whose selection can critically affect the convergence and robustness of any machine learning algorithm. As shown in Figure 4d, when we randomly selected $\rho =0.5$ and $\epsilon =0.2$, AMLA converged in fewer than 20 iterations while CGDA was yet to converge beyond 20 iterations. Furthermore, when we randomly selected $\rho =0.8$ and $\epsilon =0.4$, AMLA converged in about seven iterations, while CGDA converged in about nine iterations. These observations from Figure 4d clearly indicate that the values of the parameters $\rho$ and $\epsilon$ are significant determinants of the rate of convergence of AMLA. In fact, from Figure 4d, we can see that the selection of $\rho$ and $\epsilon$ can either increase or decrease the rate of convergence of AMLA. However, it does not affect the robustness of AMLA, as AMLA is guaranteed to converge regardless of the values of $\rho$ and $\epsilon$. It is important to point out that, as seen in Figure 4d, AMLA still outperformed CGDA for the chosen values of $\rho$ and $\epsilon$.

Overall, this proof of principle demonstrates that fully accurate and efficient computational deconvolution of tumor bulk gene expression data for estimation of the proportions of cell-type fractions, including TIICs, is best achievable using a nonlinear optimization framework, whose solution can be approximated by an accelerated machine learning algorithm (AMLA). This nonlinear optimization framework truly captures the natural state of the deconvolution problem. Consequently, in the future, we will implement the entire deconvolution workflow, described in Figure 1, as a cloud-based tool with a user-friendly graphical interface, which we shall call NECSTGEP (Naturally Estimating Cell-type Subsets from Tumor Gene Expression Profiles). NECSTGEP will be equipped with an additional rigorous machine learning algorithm that will be able to automatically fix the model parameters and the initialization values for AMLA, using pattern analysis of the input bulk gene expression profiles, as well as the reference profile. This is very essential as this proof of principle has shown a crucial role in yielding highly accurate estimation results. Similar problems in biology, oncology, and medical physics, such as the optimal scheduling of combined cancer therapies and reconstruction of gene regulatory networks, parade similar levels of complexity. Thus, they can benefit from an application of the technique described thus far, to yield highly accurate results.

5. Conclusions

We introduced and discussed a nonlinear constrained optimization framework for the computational deconvolution of TIICs and other tumor cell-type subsets from tumor bulk gene expression profiles, in addition to an accelerated machine learning algorithm (AMLA) for directly approximating its solution. Our analysis using real tumor transcriptomics datasets concluded that this nonlinear approach yields values closely matching ground-truth values from experiment, because it treats the problem in its natural state. Models NM1 and NM2 produced the “best” values for the estimated cell-type fractions in this study and were significantly different from those obtained using the traditional linear modeling approach. However, one main limitation of the study is the empirical choice of model hyperparameters, which will be addressed in future studies. This study, therefore, heralds a paradigm shift away from the traditional linear modeling of the TIIC deconvolution problem.