A GPU-CUDA Numerical Algorithm for Solving a Biological Model

Abstract

Tumor angiogenesis models based on coupled nonlinear parabolic partial differential equations require solving stiff systems where explicit time-stepping methods impose severe stability constraints on the time step size. Implicit–Explicit (IMEX) schemes relax this constraint by treating diffusion terms implicitly and reaction–chemotaxis terms explicitly, reducing each time step to a single linear system solution. However, standard Gaussian elimination with partial pivoting exhibits cubic complexity in the number of spatial grid points, dominating computational cost for realistic discretizations in the range of 400–800 grid points. This work presents a CUDA-based parallel algorithm that accelerates the IMEX scheme through GPU implementation of three core computational kernels: pivot finding via atomic operations on double-precision floating-point values, row swapping with coalesced memory access patterns, and elimination updates using optimized two-dimensional thread grids. Performance measurements on an NVIDIA H100 GPU demonstrate speedup factors, achieving speedup factors from 3.5× to 113× across spatial discretizations spanning $M\in [25,800]$ grid points relative to sequential CPU execution, approaching 94.2% of the theoretical maximum speedup predicted by Amdahl’s law. Numerical validation confirms that GPU and CPU solutions agree to within twelve digits of precision over extended time integration, with conservation properties preserved to machine precision. Performance analysis reveals that the elimination kernel accounts for nearly 90% of total execution time, justifying the focus on GPU parallelization of this component. The method enables parameter studies requiring $\sim {10}^4$ PDE solves, previously computationally prohibitive, facilitating model-driven investigation of anti-angiogenic therapy design.

1. Introduction

Tumor angiogenesis, the formation of new blood vessels from pre-existing vasculature in response to tumor-secreted growth factors, constitutes a critical mechanism enabling solid tumor progression beyond the diffusion-limited threshold of a 1–2 mm diameter. The Anderson–Chaplain model [1] describes angiogenesis via four coupled nonlinear parabolic PDEs. This model, which forms the foundation of our numerical investigation, couples four state variables through nonlinear chemotactic and haptotactic flux terms, yielding a system whose stiffness parameter scales as $\mathcal{O}\left(h^{-2}\right)$ where h denotes the spatial mesh spacing.

Numerical solution of stiff reaction-diffusion systems presents computational challenges that manifest in two distinct but interconnected dimensions. First, explicit time-stepping methods impose a stability constraint $\tau \le C{h}^2$, where $\tau$ represents the temporal discretization parameter and C is a problem-dependent constant typically on the order of unity. The spatial resolutions required to capture sharp gradient features at the angiogenic front yield prohibitively small time steps under this restriction. Second, fully implicit methods eliminate this constraint but necessitate solving nonlinear systems at each time level, requiring iterative procedures with multiple linear solves per time step. Operator splitting techniques, specifically Implicit–Explicit (IMEX) methods, provide an intermediate approach that treats diffusion terms implicitly while advancing reaction and chemotaxis terms explicitly. The IMEX trapezoidal scheme proposed in [2] reduces each time step to a single linear system solution, avoiding nonlinear iterations while maintaining favorable stability properties. However, solving the resulting linear systems via Gaussian elimination with partial pivoting requires cubic computational complexity in the number of spatial grid points, rendering sequential CPU implementations prohibitively expensive for realistic discretizations. GPU acceleration has demonstrated substantial performance gains across scientific computing applications, with vendor-optimized libraries such as cuBLAS, cuSOLVER [3], MAGMA [4,5], and AmgX [6] achieving speedup factors of 10–100-fold relative to CPU implementations. While iterative solvers with algebraic multigrid preconditioning show excellent performance for large-scale stiff PDEs [7,8], the moderate system dimensions and block-structured coupling inherent in the tumor angiogenesis model suggest that GPU-accelerated direct methods justify investigation.

This work presents a CUDA-based parallel implementation of Gaussian elimination with partial pivoting for the IMEX-discretized tumor angiogenesis model from [2]. We develop three specialized computational kernels addressing the algorithmic challenges of parallel pivot selection, row interchange, and elimination updates on GPU architectures. Performance measurements demonstrate speedup factors ranging from 3.47-fold to 113.08-fold across spatial discretization levels spanning two orders of magnitude, with numerical accuracy preserved to machine precision. Detailed performance analysis quantifies the contributions of memory bandwidth, load balancing, and synchronization overhead to overall efficiency, revealing that forward elimination dominates execution time at 93.75% and identifying specific bottlenecks limiting further acceleration.

The rest of this paper is organized as follows: Section 2 reviews related work on GPU-based acceleration of stiff ODE/PDE solvers and examines vendor-optimized libraries for linear algebra operations. Section 3 presents the mathematical model of tumor angiogenesis, spatial discretization using finite differences, and the IMEX trapezoidal time-stepping scheme from [2]. Section 4 develops the CUDA-based parallel algorithm, including detailed descriptions of pivot finding, row swapping, and elimination kernels with complexity analysis. Section 5 reports performance measurements comparing CPU and GPU implementations across different spatial discretization levels, validates numerical accuracy, and analyzes scaling behavior. Finally, Section 6 closes the paper with the conclusions.

2. Related Works

In recent years, GPU-based acceleration of stiff ODE/PDE solvers has attracted growing attention, particularly in applications such as chemical systems, fluid dynamics, and biological modeling [9,10,11]. In the following, we review key works that deal with CUDA implementations of implicit or semi-implicit time integration schemes, solvers for linear systems, and hybrid or advanced parallelization strategies. Several vendor-optimized libraries, e.g., cuBLAS, cuSOLVER, MAGMA, AmgX, have been implemented to offload dense or sparse linear algebra operations to GPUs [4,12]. The development of these libraries shows that delegating the computationally intensive parts (factorizations, triangular solves, sparse matrix–vector products, multigrid preconditioning) to GPU kernels can yield substantial speedups over CPU-based solutions. For instance, Al Farhan et al. [5] demonstrate the scalability of MAGMA for dense systems up to several thousand unknowns.

For implicit or IMEX methods, the choice between direct (LU, QR) and iterative (GMRES, BiCGStab) solvers is crucial. GPU-based direct solvers implemented in MAGMA or cuSOLVER [3,5] scale well for dense or moderately sized matrices but become inefficient for large sparse systems because of pivoting and memory transfer overhead. On the contrary, iterative solvers with algebraic multigrid (AMG) or ILU preconditioners, such as those in AmgX [13,14] or Ginkgo [15], show good performance for stiff reaction–diffusion problems. Benchmark analyses in [4,7,8] confirm that AMG-preconditioned GMRES on GPUs provides superior performance per watt for stiff PDEs compared to CPU-based solvers. More recent works explore batched and mixed-precision solvers to exploit fine-grained GPU parallelism when solving many small to medium systems in parallel [16,17]. For example, Haidar et al. [16] present mixed-precision LU factorization with iterative refinement on GPUs, achieving up to 4× speedup over full double precision while maintaining numerical stability. Batched sparse operations have been successfully employed in ensemble simulations of stiff chemical kinetics [17].

Several custom CUDA implementations of Gaussian elimination and LU factorization have been proposed to address the limitations of general-purpose libraries [18,19]. Although these methods achieve impressive speedups for very large dense matrices, they face challenges such as pivoting serialization, thread divergence, and diminishing parallelism toward the final elimination stages. Studies like [18,19] show that hybrid CPU–GPU approaches can alleviate these bottlenecks and outperform pure GPU strategies for moderately sized systems. Beyond spatial parallelization, parallel-in-time methods such as Parareal have been adapted in settings that use GPU backends alongside CPU coarse components [20,21]. For example, Arteaga et al. [21] backend for the spatial components and use coarse-fine steps in a time-parallel fashion. More recently, RandNet–Parareal extends this by using neural networks for the coarse propagator over PDEs, demonstrating speedups and applicability to diffusion-reaction problems [22]. These methods decompose the time domain into parallel subintervals, combining coarse propagators (often computed on CPUs or approximated) with fine GPU-accelerated solves. For stiff problems, time-parallel integration offers a complementary path to acceleration, though communication overhead and synchronization remain major challenges.

Semi-implicit IMEX schemes treat stiff PDEs by separating their linear and nonlinear components after spatial discretization. Since each time step requires solving a linear system for the updated solution, the literature provides several practical guidelines, such as the following:

• Dense or moderately sized systems: use vendor libraries like MAGMA or cuSOLVER for optimized dense algebra.
• Large sparse systems: prefer iterative GPU solvers with AMG or ILU preconditioning (e.g., AmgX, Ginkgo).
• Many small systems: leverage batched and mixed-precision GPU routines.
• Hybrid strategies: combine CPU and GPU roles to balance pivoting and throughput.
• Time-parallel approaches: consider multi-GPU Parareal frameworks for extreme-scale stiff systems.

3. Mathematical and Numerical Model

The tumor angiogenesis process is governed by a system of coupled nonlinear parabolic partial differential equations defined on the one-dimensional spatial domain $\mathsf{\Omega }=[0,L_f]\subset \mathbb{R}$ and temporal domain $\mathcal{T}=[0,T_f]\subset {\mathbb{R}}^{+}$. Following the model presented in [2], we consider four state variables: the endothelial cell density $C:\mathsf{\Omega }\times \mathcal{T}\to {\mathbb{R}}^{+}$, the protease concentration $P:\mathsf{\Omega }\times \mathcal{T}\to {\mathbb{R}}^{+}$, the inhibitor concentration $I:\mathsf{\Omega }\times \mathcal{T}\to {\mathbb{R}}^{+}$, and the extracellular matrix (ECM) density $F:\mathsf{\Omega }\times \mathcal{T}\to {\mathbb{R}}^{+}$. The evolution of these quantities is described by the nonlinear system:

$$\left\{\begin{array}{c}{\displaystyle \frac{\partial C}{\partial t}=d_C\frac{{\partial }^{2}C}{\partial x^2}+\frac{\partial }{\partial x}\left({\alpha }_{2}C\frac{\partial I}{\partial x}\right)-\frac{\partial }{\partial x}\left({\alpha }_{1}C\frac{\partial F}{\partial x}\right)-\frac{\partial }{\partial x}\left(\frac{{\alpha }_{3}C}{1+{\alpha }_{4}T}\frac{\partial T}{\partial x}\right)+k_{1}C(1-C),}\hfill \\ {\displaystyle \frac{\partial P}{\partial t}=d_P\frac{{\partial }^{2}P}{\partial x^2}-k_{3}PI+k_{4}TC+k_{5}T-k_{6}P,}\hfill \\ {\displaystyle \frac{\partial I}{\partial t}=d_I\frac{{\partial }^{2}I}{\partial x^2}-k_{3}PI,}\hfill \\ {\displaystyle \frac{\partial F}{\partial t}=-k_{2}PF,}\hfill \end{array}\right.$$

(1)

where the tumor angiogenic factor (TAF) distribution is given by

$$T\left(x\right)=exp\left(-{ϵ}^{-1}{(L_f-x)}^2\right),$$

(2)

which models the spatial gradient of vascular endothelial growth factor (VEGF-A) established by hypoxic tumor cells localized at $x=L_f$. The scaling parameter $ϵ=0.1$ ${\mathrm{mm}}^{-2}$ yields a decay length ${\lambda }_{\mathrm{TAF}}=\sqrt{ϵ}\approx 100 \mathsf{\mu }\mathrm{m},$ consistent with measured VEGF-A gradients in tumor microenvironments. This gradient provides the primary chemotactic signal directing endothelial cell migration from pre-existing vessels at $x=0$ toward the tumor boundary.

In the endothelial cell equation C, the diffusion term $d_C{\displaystyle \frac{{\partial }^{2}C}{\partial x^2}}$ captures random motility via Brownian motion with intrinsic cell motility of approximately 1 $\mathsf{\mu }$${\mathrm{m}}^2$/s. The term ${\displaystyle \frac{\partial }{\partial x}}\left({\alpha }_{2}C{\displaystyle \frac{\partial I}{\partial x}}\right)$ models chemotaxis toward inhibitor gradients, where, counter-intuitively, angiostatin acts as an attractant at low concentrations. Haptotaxis along ECM density gradients is captured by $-{\displaystyle \frac{\partial }{\partial x}}\left({\alpha }_{1}C{\displaystyle \frac{\partial F}{\partial x}}\right)$, representing fibronectin binding via ${\alpha }_5{\beta }_1$ integrin. The term $-{\displaystyle \frac{\partial }{\partial x}}\left({\displaystyle \frac{{\alpha }_{3}C}{1+{\alpha }_{4}T}}{\displaystyle \frac{\partial T}{\partial x}}\right)$ describes saturable chemotaxis toward VEGF-A with Michaelis–Menten receptor saturation ($K_m\sim {\alpha }_4^{-1}$), while $k_{1}C(1-C)$ represents logistic proliferation with unit carrying capacity due to contact inhibition at confluence. The protease equation (P) couples production through $k_{4}TC$ and $k_{5}T$ (basal secretion), with decay via $k_{6}P$ (natural degradation) and inhibition through $k_{3}PI$. The inhibitor equation (I) exhibits decay solely through protease binding via mass-action kinetics ($-k_{3}PI$), while the ECM equation (F) describes irreversible degradation via proteolytic cleavage ($-k_{2}PF$: MMP-2-mediated collagen type IV breakdown). The system is subject to homogeneous Neumann boundary conditions ${\left.{\displaystyle \frac{\partial u}{\partial x}}\right|}_{x=0}={\left.{\displaystyle \frac{\partial u}{\partial x}}\right|}_{x=L_f}=0$ for $u\in \{C,P,I,F\}$, representing no-flux conditions at the domain boundaries. The diffusion coefficients $d_C,d_P,d_I\in {\mathbb{R}}^{+}$ quantify molecular diffusivity, the chemotactic and haptotactic sensitivity parameters ${\alpha }_1,{\alpha }_2,{\alpha }_3,{\alpha }_4\in {\mathbb{R}}^{+}$ govern directed cell migration, and the kinetic rate constants $k_1,k_2,k_3,k_4,k_5,k_6\in {\mathbb{R}}^{+}$ characterize proliferation, degradation, binding, and production processes. Parameters are dimensionless, scaled by reference length $L_{\mathrm{ref}}=1$ mm and time $t_{\mathrm{ref}}=10$ h. Diffusion coefficients match VEGF-A and MMP-2 diffusivities measured via fluorescence recovery after photobleaching in collagen gels. Kinetic rates $k_1,\dots ,k_6$ are fitted via least-squares minimization to match experimentally observed endothelial cell migration speeds $v_{exp}=40$–50 $\mathsf{\mu }$m/h in Boyden chamber work [23]. Chemotactic parameters ${\alpha }_1,\dots ,{\alpha }_4$ are calibrated via Morris sensitivity screening against capillary density time series from Matrigel plug assays, selecting values with $R^2>0.85$ correlation.

In order to obtain a computationally tractable discrete approximation, we introduce a uniform spatial mesh ${\mathsf{\Omega }}_h=\{x_i=(i-1)h:i=1,2,\dots ,M\}$ with grid spacing $h=L_f/(M-1)$. For any sufficiently smooth function $v:\mathsf{\Omega }\to \mathbb{R}$, we denote its grid function by $v_i=v\left(x_i\right)$ and introduce the standard centered finite difference operators. The first derivative is approximated by

$${\mathcal{D}}_x{v}_i\approx {\displaystyle \frac{v_{i+1}-v_{i-1}}{2h}},i=2,3,\dots ,M-1,$$

while the second derivative employs

$${\mathcal{D}}_{xx}{v}_i\approx {\displaystyle \frac{v_{i+1}-2v_i+v_{i-1}}{h^2}},i=2,3,\dots ,M-1,$$

for the same index range. The no-flux boundary conditions are enforced through ghost point relations—$v_0=v_2$ and $v_{M+1}=v_{M-1}$—which ensure ${\mathcal{D}}_x{v}_1={\mathcal{D}}_x{v}_M=0$ to machine precision. These approximations exhibit second-order truncation error $\mathcal{O}\left(h^2\right)$ for functions with bounded fourth derivatives, as verified through Taylor series expansion. The discrete spatial operators admit a natural matrix representation. The operator takes the form in the following matrix:

$${\mathbf{D}}_{i,j}=\left\{\begin{array}{cc}-2h^{-2},\hfill & \mathrm{if}i=j\mathrm{and}1\le i\le M\hfill \\ h^{-2},\hfill & \mathrm{if}|i-j|=1\mathrm{and}2\le i\le M-1\hfill \\ 2h^{-2},\hfill & \mathrm{if}(i,j)\in \left\{\right(1,2),(M,M-1\left)\right\}\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(3)

Concatenating the state variables into a single vector $\mathbf{U}={[{\mathbf{C}}^T,{\mathbf{P}}^T,{\mathbf{I}}^T,{\mathbf{F}}^T]}^T\in {\mathbb{R}}^{4M}$, the spatially semi-discretized system assumes the compact form

$${\displaystyle \frac{d\mathbf{U}}{dt}}=\mathbf{L}\mathbf{U}+\mathbf{N}\left(\mathbf{U}\right),$$

where $\mathbf{L}\in {\mathbb{R}}^{4M\times 4M}$ represents the linear differential operators and $\mathbf{N}:{\mathbb{R}}^{4M}\to {\mathbb{R}}^{4M}$ encapsulates the nonlinear reaction and chemotaxis terms. Considering endothelial cells, the only linear term is diffusion $d_C\mathbf{D}\mathbf{C}$, while chemotaxis terms ${\displaystyle \frac{\partial }{\partial x}}\left(f_{I}C{\displaystyle \frac{\partial I}{\partial x}}\right)$ involve products $C·\nabla I$ and, thus, belong to $\mathbf{N}\left(\mathbf{U}\right)$, yielding ${\left[\mathbf{L}\right]}_{1,1}=d_C\mathbf{D}$ and ${\left[\mathbf{L}\right]}_{1,j}={\mathbf{0}}_M$ for $j\ne 1$. For proteases, the linear terms comprise diffusion $d_P\mathbf{D}\mathbf{P}$, production $k_{4}TC$ coupling cells to proteases, and decay $-k_{6}P$. The production term is linear in C with spatially varying coefficient $k_4{T}_i$, giving ${\left[\mathbf{L}\right]}_{2,1}={\mathbf{K}}_C:=\mathrm{diag}(k_4{T}_1,\dots ,k_4{T}_M)$ and ${\left[\mathbf{L}\right]}_{2,2}=d_P\mathbf{D}-k_6{\mathbf{I}}_M$, while the term $-k_{3}PI$ is bilinear and belongs to $\mathbf{N}\left(\mathbf{U}\right)$. For inhibitors, only diffusion $d_I\mathbf{D}\mathbf{I}$ is linear, with the binding term $-k_{3}PI$ being bilinear, thus ${\left[\mathbf{L}\right]}_{3,3}=d_I\mathbf{D}$ and ${\left[\mathbf{L}\right]}_{3,j}={\mathbf{0}}_M$ for $j\ne 3$. About ECM, the degradation $-k_{2}PF$ is entirely bilinear, rendering the fourth row block identically zero: ${\left[\mathbf{L}\right]}_{4,j}={\mathbf{0}}_M$ for all j.

The special treatment of the first and last rows incorporates the ghost point boundary conditions directly into the matrix structure, eliminating the need for explicit boundary handling in the computational kernels. Similarly, we define the discrete gradient operator $\mathbf{G}\in {\mathbb{R}}^{M\times M}$ through ${\mathbf{G}}_{i,i-1}=-{\left(2h\right)}^{-1}$ and ${\mathbf{G}}_{i,i+1}={\left(2h\right)}^{-1}$ for $i=2,3,\dots ,M-1$, with all other entries vanishing. The boundary rows $i\in \{1,M\}$ remain identically zero, consistent with the no-flux conditions.

Assembling these blocks, the linear operator exhibits a $4\times 4$ block structure:

$$\mathbf{L}=\left[\begin{array}{cccc}{d}_C\mathbf{D}& {\mathbf{0}}_M& {\mathbf{0}}_M& {\mathbf{0}}_M\\ \mathrm{diag}(k_4{T}_1,\dots ,k_4{T}_M)& d_P\mathbf{D}-k_6{\mathbf{I}}_M& {\mathbf{0}}_M& {\mathbf{0}}_M\\ {\mathbf{0}}_M& {\mathbf{0}}_M& d_I\mathbf{D}& {\mathbf{0}}_M\\ {\mathbf{0}}_M& {\mathbf{0}}_M& {\mathbf{0}}_M& {\mathbf{0}}_M\end{array}\right],$$

(4)

where ${\mathbf{0}}_M\in {\mathbb{R}}^{M\times M}$ denotes the zero matrix and ${\mathbf{I}}_M\in {\mathbb{R}}^{M\times M}$ the identity matrix. The TAF-dependent coupling in the $(2,1)$ block arises from the protease production term $k_{4}TC$ in system (1). The eigenvalue spectrum of $\mathbf{L}$ is dominated by the diffusion operators, with the smallest eigenvalue satisfying ${\lambda }_{min}\left(\mathbf{L}\right)=\mathcal{O}(-h^{-2})$, which imposes severe stability restrictions on explicit time-stepping methods.

The nonlinear operator $\mathbf{N}\left(\mathbf{U}\right)$ is constructed from the chemotactic, haptotactic, and reaction terms. For the endothelial cell equation, we have:

$${\left[\mathbf{N}\left(\mathbf{U}\right)\right]}_i={\mathcal{D}}_x\left[\left({\alpha }_2{C}_i{\mathcal{D}}_x{I}_i\right)-\left({\alpha }_1{C}_i{\mathcal{D}}_x{F}_i\right)-\left({\displaystyle \frac{{\alpha }_3{C}_i}{1+{\alpha }_4{T}_i}}{\mathcal{D}}_x{T}_i\right)\right]+k_1{C}_i(1-C_i),$$

(5)

for $i=1,2,\dots ,M$, where the spatial derivatives are evaluated using the centered difference approximations with appropriate boundary modifications. The product rule ${\mathcal{D}}_x\left(fg\right)=f{\mathcal{D}}_{x}g+g{\mathcal{D}}_{x}f+\mathcal{O}\left(h^2\right)$ ensures consistency with the continuous formulation. The remaining components follow similarly:

$${\left[\mathbf{N}\left(\mathbf{U}\right)\right]}_{M+i}=-k_3{P}_i{I}_i+k_5{T}_i,$$

$${\left[\mathbf{N}\left(\mathbf{U}\right)\right]}_{2M+i}=-k_3{P}_i{I}_i,$$

and

$${\left[\mathbf{N}\left(\mathbf{U}\right)\right]}_{3M+i}=-k_2{P}_i{F}_i\mathrm{for}i=1,2,\dots ,M.$$

The Lipschitz constant of $\mathbf{N}$ with respect to the Euclidean norm ${\parallel ·\parallel }_2$ can be bounded uniformly on compact subsets of ${\mathbb{R}}^{4M}$ through careful analysis of the chemotactic terms, yielding

$${\parallel \mathbf{N}\left(\mathbf{U}\right)-\mathbf{N}\left(\mathbf{V}\right)\parallel }_2\le L_N{\parallel \mathbf{U}-\mathbf{V}\parallel }_2,$$

where $L_N=\mathcal{O}\left(h^{-1}\right)$ due to the presence of gradient operators.

3.1. Time Integration Method and Algorithm

The stiffness inherent in the spatially discretized system necessitates an operator splitting approach that treats the linear diffusion terms implicitly while handling the nonlinear reaction–chemotaxis terms explicitly. We partition the time domain $\mathcal{T}=[0,T_f]$ into N uniform intervals with time step $\tau =T_f/N$ and discrete time levels $t^n=n\tau$ for $n=0,1,\dots ,N$. The Implicit–Explicit (IMEX) trapezoidal scheme, proposed in [2], integrates the linear operator $\mathbf{L}\mathbf{U}$ using the second-order Crank–Nicolson method, which provides A-stability and unconditional stability for diffusion-dominated problems, while the nonlinear term $\mathbf{N}\left(\mathbf{U}\right)$ is advanced explicitly via the forward Euler method to avoid the computational expense of solving nonlinear systems at each time step.

Given the solution ${\mathbf{U}}^n\approx \mathbf{U}\left(t^n\right)$ at time $t^n$, the update to ${\mathbf{U}}^{n+1}$ satisfies the discrete evolution equation:

$${\displaystyle \frac{{\mathbf{U}}^{n+1}-{\mathbf{U}}^n}{\tau }}={\displaystyle \frac{1}{2}}\mathbf{L}({\mathbf{U}}^{n+1}+{\mathbf{U}}^n)+\mathbf{N}\left({\mathbf{U}}^n\right).$$

(6)

Rearranging terms and introducing the notation

$$\mathbf{A}:={\mathbf{I}}_{4M}-{\displaystyle \frac{\tau }{2}}\mathbf{L}\mathrm{and}\mathbf{B}:={\mathbf{I}}_{4M}+{\displaystyle \frac{\tau }{2}}\mathbf{L},$$

where ${\mathbf{I}}_{4M}\in {\mathbb{R}}^{4M\times 4M}$ denotes the identity matrix, and we obtain the linear system:

$$\mathbf{A}{\mathbf{U}}^{n+1}=\mathbf{B}{\mathbf{U}}^n+\tau \mathbf{N}\left({\mathbf{U}}^n\right)=:{\mathbf{r}}^n.$$

(7)

The system matrix $\mathbf{A}$ inherits the block structure from $\mathbf{L}$ and, critically, is nonsingular for all $\tau >0$ since the eigenvalues of $\mathbf{L}$ are real and non-positive (due to the discrete Laplacian being negative semi-definite), implying that the eigenvalues of $\mathbf{A}$ satisfy $\mathrm{Re}\left(\lambda \left(\mathbf{A}\right)\right)=1-{\displaystyle \frac{\tau }{2}}\mathrm{Re}\left(\lambda \left(\mathbf{L}\right)\right)\ge 1>0$. The right-hand side vector ${\mathbf{r}}^n\in {\mathbb{R}}^{4M}$ is explicitly computable from the known data at time level n, involving one matrix–vector multiplication $\mathbf{B}{\mathbf{U}}^n$ with computational cost $\mathcal{O}\left(M^2\right)$ due to the banded structure of $\mathbf{B}$, and one evaluation of the nonlinear operator $\mathbf{N}\left({\mathbf{U}}^n\right)$ requiring $\mathcal{O}\left(M\right)$ operations.

The stability of the IMEX scheme (6) is governed by the spectral properties of the amplification operator and the Lipschitz constant of the nonlinear term. For the linear component, the trapezoidal method applied to $d\mathbf{v}/dt=\mathbf{L}\mathbf{v}$ yields the amplification factor

$$\rho \left(\lambda \right)=(1+{\displaystyle \frac{\tau }{2}}\lambda )/(1-{\displaystyle \frac{\tau }{2}}\lambda ),$$

for an eigenvalue $\lambda$ of $\mathbf{L}$. Since $\lambda \le 0$, we have

$$|1+{\displaystyle \frac{\tau }{2}}\lambda |\le 1\mathrm{and}|1-{\displaystyle \frac{\tau }{2}}\lambda |\ge 1,$$

ensuring $\left|\rho \right(\lambda \left)\right|\le 1$ for all $\tau >0$. This unconditional stability for the linear part eliminates the restrictive $\tau =\mathcal{O}\left(h^2\right)$ constraint imposed by fully explicit schemes. The explicit treatment of the nonlinear term introduces a stability condition $\tau <2/L_N$, where $L_N$ is the Lipschitz constant of $\mathbf{N}$. However, since $L_N=\mathcal{O}\left(h^{-1}\right)$, the allowable time step satisfies $\tau =\mathcal{O}\left(h\right)$, which is significantly less restrictive than the $\mathcal{O}\left(h^2\right)$ constraint of fully explicit methods. The local truncation error of the IMEX scheme is $\mathcal{O}\left({\tau }^3\right)$ for the implicit linear part (trapezoidal method) and $\mathcal{O}\left({\tau }^2\right)$ for the explicit nonlinear part (forward Euler), yielding an overall temporal accuracy of $\mathcal{O}\left({\tau }^2\right)$ when combined with the $\mathcal{O}\left(h^2\right)$ spatial accuracy.

The computational bottleneck of the IMEX method is related to the solution of the dense linear system (7) at each time step. While $\mathbf{A}$ inherits some sparsity from the discrete Laplacian operators in its diagonal blocks, the coupling terms and the $4\times 4$ block structure result in a matrix with $\mathcal{O}\left(M^2\right)$ nonzero entries. Standard Gaussian elimination with partial pivoting requires $\mathcal{O}\left({\left(4M\right)}^3\right)=\mathcal{O}\left(M^3\right)$ arithmetic operations per time step, dominating the $\mathcal{O}\left(M\right)$ cost of evaluating $\mathbf{N}\left({\mathbf{U}}^n\right)$ and the $\mathcal{O}\left(M^2\right)$ cost of the matrix–vector product $\mathbf{B}{\mathbf{U}}^n$. Over N time steps, the total complexity becomes $\mathcal{O}\left(N{M}^3\right)$. In case of realistic spatial resolutions $M\in [200,800]$ and long-time integrations $N\in [1000,10000]$, this cubic scaling renders sequential CPU implementations prohibitively expensive, motivating the GPU parallelization strategies developed in the following section.

The computational procedure for advancing the solution from time $t^n$ to $t^{n+1}$ can be formalized algorithmically. Given the current state ${\mathbf{U}}^n\in {\mathbb{R}}^{4M}$, we must evaluate the nonlinear term, construct the right-hand side vector, solve the linear system, and enforce physical constraints. The complete IMEX time-stepping procedure is presented in Algorithm 1.

The algorithm decomposes naturally into four computational kernels per time step. The nonlinear evaluation (Step 1) requires $\mathcal{O}\left(M\right)$ operations and involves evaluating finite difference approximations and pointwise products, which are trivially parallelizable with perfect load balance. The right-hand side construction (Step 2) performs a sparse matrix–vector multiplication with complexity $\mathcal{O}\left(M^2\right)$ due to the banded structure of $\mathbf{B}$, exploiting the fact that $\mathbf{L}$ contains discrete Laplacian operators with at most five nonzero entries per row. The linear system solution (Step 3) dominates the computational cost at $\mathcal{O}\left(M^3\right)$ for the serial Gaussian elimination procedure, making it the primary target for GPU acceleration. Finally, the non-negativity enforcement (Step 4) is an $\mathcal{O}\left(M\right)$ post-processing operation that ensures the numerical solution respects the physical constraints inherent in the continuous model, where all state variables represent concentrations or densities that must remain non-negative. This structure, wherein Steps 1, 2, and 4 exhibit embarrassingly parallel characteristics, while Step 3 presents a dense linear algebra challenge, motivates the CUDA-based parallelization strategies developed in the following section. The main observation is that while the nonlinear term $\mathbf{N}\left({\mathbf{U}}^n\right)$ could potentially be evaluated on the GPU and kept in device memory to avoid host–device transfers, the dominant cost lies in solving the implicit linear system $\mathbf{A}{\mathbf{U}}^{n+1}={\mathbf{r}}^n$, which must be performed at every time step and accounts for over 95% of the total computational time for typical spatial resolutions $M\in [400,800]$.

Algorithm 1 IMEX time-stepping for tumor angiogenesis.
Require: System matrices $\mathbf{D},\mathbf{G}\in {\mathbb{R}}^{M\times M}$, TAF distribution ${\left\{T_i\right\}}_{i=1}^M$, parameters $\{d_C,d_P,d_I,k_1,\dots ,k_6,{\alpha }_1,\dots ,{\alpha }_4\}$, time step $\tau >0$, final time $T_f$, initial condition ${\mathbf{U}}^0\in {\mathbb{R}}^{4M}$
Ensure: Solution trajectory ${\left\{{\mathbf{U}}^n\right\}}_{n=0}^N$ with $N=\lceil T_f/\tau \rceil$
1:	Initialization:
2:	Construct linear operator $\mathbf{L}\in {\mathbb{R}}^{4M\times 4M}$ according to (4)
3:	Form system matrix $\mathbf{A}\leftarrow {\mathbf{I}}_{4M}-{\displaystyle \frac{\tau }{2}}\mathbf{L}$
4:	Form explicit matrix $\mathbf{B}\leftarrow {\mathbf{I}}_{4M}+{\displaystyle \frac{\tau }{2}}\mathbf{L}$
5:	for  $n=0$  to  $N-1$  do
6:	Step 1: Nonlinear Evaluation
7:	Extract components: ${\mathbf{C}}^n\leftarrow {\mathbf{U}}_{1:M}^n$, ${\mathbf{P}}^n\leftarrow {\mathbf{U}}_{M+1:2M}^n$, ${\mathbf{I}}^n\leftarrow {\mathbf{U}}_{2M+1:3M}^n$, ${\mathbf{F}}^n\leftarrow {\mathbf{U}}_{3M+1:4M}^n$
8:	for $i=1$ to M do
9:	Compute chemotactic flux: ${\mathsf{\Phi }}_I^i\leftarrow {\alpha }_2{C}_i^n{\mathcal{D}}_x{I}_i^n$
10:	Compute haptotactic flux: ${\mathsf{\Phi }}_F^i\leftarrow {\alpha }_1{C}_i^n{\mathcal{D}}_x{F}_i^n$
11:	Compute TAF chemotaxis: ${\mathsf{\Phi }}_T^i\leftarrow {\displaystyle \frac{{\alpha }_3{C}_i^n}{1+{\alpha }_4{T}_i}}{\mathcal{D}}_x{T}_i$
12:	${\left[\mathbf{N}\left({\mathbf{U}}^n\right)\right]}_i\leftarrow {\mathcal{D}}_x({\mathsf{\Phi }}_I^i-{\mathsf{\Phi }}_F^i-{\mathsf{\Phi }}_T^i)+k_1{C}_i^n(1-C_i^n)$
13:	${\left[\mathbf{N}\left({\mathbf{U}}^n\right)\right]}_{M+i}\leftarrow -k_3{P}_i^n{I}_i^n+k_5{T}_i$
14:	${\left[\mathbf{N}\left({\mathbf{U}}^n\right)\right]}_{2M+i}\leftarrow -k_3{P}_i^n{I}_i^n$
15:	${\left[\mathbf{N}\left({\mathbf{U}}^n\right)\right]}_{3M+i}\leftarrow -k_2{P}_i^n{F}_i^n$
16:	end for
17:	Step 2: Right-Hand Side Construction
18:	${\mathbf{r}}^n\leftarrow \mathbf{B}{\mathbf{U}}^n+\tau \mathbf{N}\left({\mathbf{U}}^n\right)$	▹$\mathcal{O}\left(M^2\right)$ matrix–vector product
19:	Step 3: Linear System Solution
20:	Solve $\mathbf{A}{\mathbf{U}}^{n+1}={\mathbf{r}}^n$ via Gaussian elimination with partial pivoting
21:	Step 4: Enforce Non-Negativity
22:	for $j=1$ to $4M$ do
23:	if $U_j^{n+1}<0$ then
24:	$U_j^{n+1}\leftarrow 0$	▹ Physical constraint: concentrations/densities $\ge 0$
25:	end if
26:	end for
27:	end for
28:	return  ${\left\{{\mathbf{U}}^n\right\}}_{n=0}^N$

Figure 1 illustrates the computational pipeline for solving the linear system $\mathbf{A}{\mathbf{U}}^{n+1}={\mathbf{r}}^n$ via GPU-accelerated Gaussian elimination with partial pivoting. The workflow consists of five sequential phases: (1) host-to-device transfer of the augmented matrix; (2) iterative execution of three CUDA kernels within a $(4M-1)$-step loop; (3) device-to-host retrieval of the upper-triangular result; (4) CPU-based back substitution exploiting cache locality; and (5) non-negativity enforcement. The decision to perform back substitution on the CPU rather than the GPU reflects the inherent data dependency $U_i=f(U_{i+1},\dots ,U_{4M-1})$, which prevents effective parallelization for systems with $M\le 1000$.

Figure 1. GPU workflow for solving $\mathbf{A}{\mathbf{U}}^{n+1}={\mathbf{r}}^n$ via Gaussian elimination with partial pivoting. The pipeline executes three CUDA kernels iteratively within a $(4M-1)$-step loop.

3.2. Gaussian Elimination Choice Discussion

We employ Gaussian elimination rather than iterative methods (GMRES, BiCGStab with AMG preconditioning) for three interconnected reasons. First, the system size $4M\in [400,3200]$ lies below the crossover point ($\sim {10}^4$ DOF) where iterative solvers with algebraic multigrid preconditioning become competitive [7]. In the case of $M=400$, AmgX requires 847 GMRES iterations versus 1599 guaranteed elimination steps, resulting in 3.5 s versus 1.7 s wall time. Second, the off-diagonal block ${\mathbf{K}}_C=\mathrm{diag}\left(k_4{T}_i\right)$ in Equation (4) couples proteases to endothelial cells, destroying the block-tridiagonal sparsity pattern. This coupling yields an effective matrix bandwidth of $\tilde{b}=M$, substantially reducing sparse solver advantages. Third, clinical decision support systems require worst-case latency guarantees. Direct methods provide deterministic $O\left(M^3\right)$ complexity, whereas iterative methods exhibit iteration count variability (600–1200 iterations observed for angiogenesis parameters in Table 1). This guaranteed deterministic behavior, combined with moderate dimensions and unfavorable coupling structure, justifies our choice of Gaussian elimination with partial pivoting.

Table 1. Model parameters: biological interpretation and typical values.

Parameter	Value	Units	Biological Meaning
$d_C$	0.001	${\mathrm{mm}}^2$/h	Endothelial cell random motility (Brownian diffusion)
$d_P$	0.001	${\mathrm{mm}}^2$/h	Matrix metalloproteinase-2 (MMP-2) diffusivity in tissue
$d_I$	0.001	${\mathrm{mm}}^2$/h	Angiostatin diffusivity
$k_1$	0.1	${\mathrm{h}}^{-1}$	Logistic proliferation rate with contact inhibition
$k_2$	0.15	${\mathrm{h}}^{-1}$	ECM degradation rate
$k_3$	0.5	${\mathrm{h}}^{-1}$	Protease-inhibitor irreversible binding
$k_4$	0.2	${\mathrm{h}}^{-1}$	VEGF-induced protease production
$k_5$	0.1	${\mathrm{h}}^{-1}$	Basal protease secretion
$k_6$	0.1	${\mathrm{h}}^{-1}$	Protease natural decay
${\alpha }_1$	0.6	–	Haptotaxis coefficient
${\alpha }_2$	0.25	–	Chemotaxis to inhibitors
${\alpha }_3$	0.3	–	Maximum chemotaxis to VEGF-A
${\alpha }_4$	0.75	–	Michaelis–Menten constant for VEGFR-2 saturation
$ϵ$	0.1	${\mathrm{mm}}^{-2}$	TAF gradient steepness

4. CUDA-Based Parallel Algorithm

The GPU parallelization of the IMEX scheme focuses on accelerating the solution of the linear system $\mathbf{A}{\mathbf{U}}^{n+1}={\mathbf{r}}^n$ via a CUDA implementation of Gaussian elimination with partial pivoting, alongside parallel construction of the discrete operators $\mathbf{G}$ and $\mathbf{D}$. We adopt the augmented matrix representation

$$\tilde{\mathbf{A}}=[\mathbf{A}\mid {\mathbf{r}}^n]\in {\mathbb{R}}^{4M\times (4M+1)},$$

and store it in row-major order as a contiguous one-dimensional array in GPU device memory. This memory layout ensures coalesced access patterns, wherein threads within a warp (32 consecutive threads) access consecutive memory locations, maximizing memory bandwidth utilization. The flattened index mapping is given by

$${\tilde{A}}_{i,j}\mapsto {\tilde{A}}_{\mathrm{flat}}[i(4M+1)+j]\mathrm{for}i\in \{0,1,\dots ,4M-1\},j\in \{0,1,\dots ,4M\},$$

where the zero-based indexing convention is adopted consistent with CUDA arrays.

The Gaussian elimination algorithm proceeds in $4M-1$ sequential elimination steps. At step $k\in \{0,1,\dots ,4M-2\}$, we first identify the pivot row

$$p_k=\underset{i\ge k}{arg\; max}\left|{\tilde{A}}_{i,k}\right|,$$

to ensure numerical stability by bounding the growth factor. The pivot-finding operation is parallelized over the subcolumn $\{{\tilde{A}}_{k,k},{\tilde{A}}_{k+1,k},\dots ,{\tilde{A}}_{4M-1,k}\}$ using a CUDA kernel that employs atomic operations to determine the global maximum. Since CUDA does not provide a native atomic maximum operation for double-precision floating-point types, we implement it via a compare-and-swap (CAS) loop operating on the bit representation, as detailed in Algorithm 2.

The CAS operation $\mathtt{atomicCAS}(\mathrm{addr},\mathrm{expected},\mathrm{desired})$ atomically compares the value at addr with expected. If equal, it writes desired and returns expected; otherwise, it returns the actual value found at addr without modification. The loop repeats until no other thread has modified the memory location between the read and write operations, ensuring lock-free progress. The bit-level reinterpretation via $\_\_\mathtt{double}\_\mathtt{as}\_\mathtt{longlong}$ and $\_\_\mathtt{longlong}\_\mathtt{as}\_\mathtt{double}$ preserves the IEEE 754 double-precision representation while enabling integer-based atomic operations. Each thread $t\in \{0,1,\dots ,255\}$ processes row index $i=t$ (and potentially $i=t+256,t+512,\dots$ via grid-stride loop if $4M>256$). The shared memory variables $\mu$ (maximum absolute value) and $\iota$ (corresponding row index) are initialized to zero and k, respectively, by thread 0 before a block-wide synchronization barrier $\_\_\mathtt{syncthreads}\left(\right)$ ensures visibility to all threads. Subsequently, each thread computes $v_i=\left|{\tilde{A}}_{i,k}\right|$, performs the atomic maximum update on $\mu$, synchronizes again, and if $|v_i-\mu |<{ϵ}_{\mathrm{tol}},$ updates the index $\iota \leftarrow i$. Finally, thread 0 writes the result $p_k\leftarrow \iota$ to a device memory location accessible to the host. The computational complexity of this kernel is $\mathcal{O}(4M-k)$ work distributed over ${\theta }_{\mathrm{pivot}}$ threads with $\mathcal{O}(log{\theta }_{\mathrm{pivot}})$ atomic contention overhead, yielding an effective parallel time complexity of

$${\mathcal{T}}_{\mathrm{pivot}}^{\left(k\right)}=\mathcal{O}((M-k)/{\theta }_{\mathrm{pivot}}+log{\theta }_{\mathrm{pivot}})=\mathcal{O}\left(M\right),$$

for each step k.

Algorithm 2 Atomic maximum for doubles types.
Require: Memory address $\mu \in \mathbb{R}$ (shared or global), candidate value $v\in \mathbb{R}$
Ensure:$\mu \leftarrow max(\mu ,v)$ atomically
1:	${\mathrm{addr}}_{\mathrm{ull}}\leftarrow \mathtt{reinterpret}\_\mathtt{cast}<\mathrm{unsigned}\mathrm{long}\mathrm{long}\ast >(\&\mu )$
2:	repeat
3:	${\mathrm{old}}_{\mathrm{ull}}\leftarrow \ast {\mathrm{addr}}_{\mathrm{ull}}$	▹ Read current bit pattern
4:	${\mathrm{assumed}}_{\mathrm{ull}}\leftarrow {\mathrm{old}}_{\mathrm{ull}}$
5:	${\mathrm{current}}_{\mathrm{double}}\leftarrow \_\_\mathtt{longlong}\_\mathtt{as}\_\mathtt{double}\left({\mathrm{assumed}}_{\mathrm{ull}}\right)$
6:	${\mathrm{new}}_{\mathrm{double}}\leftarrow max({\mathrm{current}}_{\mathrm{double}},v)$
7:	${\mathrm{new}}_{\mathrm{ull}}\leftarrow \_\_\mathtt{double}\_\mathtt{as}\_\mathtt{longlong}\left({\mathrm{new}}_{\mathrm{double}}\right)$
8:	${\mathrm{old}}_{\mathrm{ull}}\leftarrow \mathtt{atomicCAS}({\mathrm{addr}}_{\mathrm{ull}},{\mathrm{assumed}}_{\mathrm{ull}},{\mathrm{new}}_{\mathrm{ull}})$
9:	until${\mathrm{old}}_{\mathrm{ull}}={\mathrm{assumed}}_{\mathrm{ull}}$	▹ CAS succeeded
10:	return$\_\_\mathtt{longlong}\_\mathtt{as}\_\mathtt{double}\left({\mathrm{old}}_{\mathrm{ull}}\right)$	▹ Previous value

Once the pivot row $p_k$ is determined, if $p_k\ne k$, we perform a row interchange ${\tilde{A}}_{p_k,:}\leftrightarrow {\tilde{A}}_{k,:}$ to move the pivot element to the diagonal position. This operation is parallelized by assigning each column $j\in \{0,1,\dots ,4M\}$ to a separate thread, which simultaneously swaps the elements ${\tilde{A}}_{p_k,j}\leftrightarrow {\tilde{A}}_{k,j}$ using three memory accesses (read–temp–write–write). The kernel is launched with ${\beta }_{\mathrm{swap}}=\lceil (4M+1)/{\theta }_{\mathrm{swap}}\rceil$ thread blocks, each containing ${\theta }_{\mathrm{swap}}=256$ threads, such that thread t in block b handles column $j=b·{\theta }_{\mathrm{swap}}+t$. The memory accesses exhibit adequate coalescing since consecutive threads access consecutive columns in the row-major layout. The parallel time complexity is

$${\mathcal{T}}_{\mathrm{swap}}=\mathcal{O}((4M+1)/\left({\beta }_{\mathrm{swap}}·{\theta }_{\mathrm{swap}}\right))=\mathcal{O}\left(1\right),$$

assuming sufficient parallelism. The elimination step itself, which zeroes out the elements below the pivot in column k, constitutes the most computationally intensive phase. For each row $i\in \{k+1,k+2,\dots ,4M-1\}$, we compute the multiplier $m_i={\tilde{A}}_{i,k}/{\tilde{A}}_{k,k}$ and update the elements

$${\tilde{A}}_{i,j}\leftarrow {\tilde{A}}_{i,j}-m_i{\tilde{A}}_{k,j}\forall j\in \{k+1,k+2,\dots ,4M\}.$$

This operation updates a $(4M-k-1)\times (4M-k)$ submatrix and is parallelized via a two-dimensional thread grid. Each thread $(t_x,t_y)$ in a two-dimensional thread block of dimensions $({\theta }_x,{\theta }_y)=(16,16)$ is assigned to submatrix element $(i,j)=(b_x·{\theta }_x+t_x,b_y·{\theta }_y+t_y)$, where $(b_x,b_y)$ denotes the block index. The grid dimensions are chosen as $({\beta }_x,{\beta }_y)=(\lceil 4M/{\theta }_x\rceil ,\lceil (4M+1)/{\theta }_y\rceil )$ to cover the entire augmented matrix, with threads outside the valid submatrix range performing no operation. Within each thread, we first check the condition $i>k\wedge i<4M\wedge j>k\wedge j\le 4M$, then compute the multiplier via a single read of ${\tilde{A}}_{i,k}$ and ${\tilde{A}}_{k,k}$, followed by a fused multiply–subtract operation updating ${\tilde{A}}_{i,j}$. The memory access pattern for the read of ${\tilde{A}}_{k,j}$ is highly coalesced since threads with consecutive $t_y$ (and, hence, consecutive j) access consecutive elements of the pivot row. Similarly, the write to ${\tilde{A}}_{i,j}$ coalesces across threads with consecutive $t_y$ within the same row i. However, reading the multiplier element ${\tilde{A}}_{i,k}$ involves threads with different $t_x$ (different rows i) accessing the same column k, which does not coalesce perfectly but benefits from L1 cache reuse within the thread block. The computational work at step k is $(4M-k-1)(4M-k)$ multiply–subtract operations distributed over ${\theta }_x{\theta }_y{\beta }_x{\beta }_y\approx 256·4M^2/\left(16·16\right)·(4M+1)/\left(16·16\right)=\mathcal{O}\left(M^2\right)$ threads, yielding a parallel time complexity of

$${\mathcal{T}}_{\mathrm{elim}}^{\left(k\right)}=\mathcal{O}({(4M-k)}^2/\left({\theta }_x{\theta }_y\right))=\mathcal{O}\left(M^2\right),$$

per elimination step. Summing over all steps, the total forward elimination phase requires:

$${\mathcal{T}}_{\mathrm{forward}}=\sum _{k=0}^{4M-2}\left[{\mathcal{T}}_{\mathrm{pivot}}^{\left(k\right)}+{\mathcal{T}}_{\mathrm{swap}}+{\mathcal{T}}_{\mathrm{elim}}^{\left(k\right)}\right]=\sum _{k=0}^{4M-2}\mathcal{O}\left({\displaystyle \frac{{(4M-k)}^2}{{\theta }_x{\theta }_y}}\right)=\mathcal{O}\left({\displaystyle \frac{M^3}{{\theta }_x{\theta }_y}}\right),$$

(8)

where we have used

$$\sum _{j=1}^n{j}^2=n(n+1)(2n+1)/6=\mathcal{O}\left(n^3\right).$$

Setting ${\theta }_x{\theta }_y=256$ and a GPU with $P\approx 5000$ CUDA cores, the effective parallelism reduces the constant factor by approximately two orders of magnitude compared to a sequential CPU implementation. After completing the forward elimination phase, the augmented matrix $\tilde{\mathbf{A}}$ has been transformed into upper triangular form $\tilde{\mathbf{U}}=[\mathbf{U}\mid \mathbf{z}]$, where $\mathbf{U}\in {\mathbb{R}}^{4M\times 4M}$ is upper-triangular and $\mathbf{z}\in {\mathbb{R}}^{4M}$ is the transformed right-hand side. The back substitution phase computes the solution ${\mathbf{U}}^{n+1}$ via:

$$U_i^{n+1}={\displaystyle \frac{1}{{\tilde{U}}_{i,i}}}\left({\tilde{U}}_{i,4M}-\sum _{j=i+1}^{4M-1}{\tilde{U}}_{i,j}{U}_j^{n+1}\right),i=4M-1,4M-2,\dots ,0.$$

(9)

This computation exhibits an inherent data dependency: the value $U_i^{n+1}$ depends on all previously computed values $U_{i+1}^{n+1},\dots ,U_{4M-1}^{n+1}$, preventing direct parallelization across the rows. While parallel scan algorithms and dependency-graph scheduling techniques can mitigate this seriality to some extent, the performance gains are typically modest for linear systems of dimension $4M\in [800,3200]$ due to synchronization overhead. Consequently, our implementation performs back substitution on the CPU host after copying the upper triangular matrix $\tilde{\mathbf{U}}$ from device to host memory. The serial complexity of back substitution is

$$\sum _{i=0}^{4M-1}(4M-i-1)=(4M-1)\left(4M\right)/2=\mathcal{O}\left(M^2\right),$$

which is asymptotically dominated by the $\mathcal{O}\left(M^3\right)$ forward elimination but can represent a non-negligible fraction of the total time for moderate M values.

The construction of discrete operators $\mathbf{G}$ and $\mathbf{D}$ is also parallelized on the GPU. Considering the gradient matrix $\mathbf{G}\in {\mathbb{R}}^{M\times M}$, each thread $i\in \{1,2,\dots ,M-2\}$ sets the sub-diagonal element ${\mathbf{G}}_{i,i-1}=-{\left(2h\right)}^{-1}$ and super-diagonal element ${\mathbf{G}}_{i,i+1}={\left(2h\right)}^{-1}$, with all other entries initialized to zero. The boundary rows $i\in \{0,M-1\}$ remain identically zero to enforce the no-flux conditions. A subsequent scaling pass (which can be fused into the same kernel via a second loop over all entries) applies the factor ${\left(2h\right)}^{-1}$ if not already applied. The kernel is launched with ${\beta }_G=\lceil M/{\theta }_G\rceil$ blocks of ${\theta }_G=256$ threads each, yielding a parallel complexity

$${\mathcal{T}}_G=\mathcal{O}(M/{\theta }_G)=\mathcal{O}\left(M\right),$$

under the assumption that each thread performs $\mathcal{O}\left(1\right)$ work. Similarly, the Laplacian matrix $\mathbf{D}$ construction assigns each row i to a thread, which sets the diagonal element ${\mathbf{D}}_{i,i}=-2h^{-2}$ and off-diagonal elements ${\mathbf{D}}_{i,i\pm 1}=h^{-2}$ with special handling for the boundary rows $i\in \{0,M-1\}$ as specified in Equation (3), again achieving $\mathcal{O}\left(M\right)$ parallel complexity.

Taking into account the memory transfer costs between host and device, at each time step, the augmented matrix $\tilde{\mathbf{A}}\in {\mathbb{R}}^{4M\times (4M+1)}$ requires $4M(4M+1)\times 8=128M^2+32M$ bytes of data transfer from host to device before the elimination kernel launches. After forward elimination, the upper triangular matrix $\tilde{\mathbf{U}}$ of the same size is transferred back to the host for back substitution, doubling the transfer volume per time step. For a typical PCIe Gen3 ×16 link with peak bandwidth ${\mathrm{BW}}_{\mathrm{PCIe}}\approx 16$ GB/s, the transfer time is ${\mathcal{T}}_{\mathrm{transfer}}=2(128M^2+32M)/(16\times {10}^9)=\mathcal{O}\left(M^2\right)$ s. However, this cost is reduced across the N time steps and can be partially overlapped with computation through CUDA streams if multiple systems are solved concurrently. The ratio of computation time to transfer time is proportional to $(M^3/P)/M^2=M/P$, indicating that for $M\gtrsim 10P$, the transfer overhead becomes negligible. In our target regime $M\in [400,800]$ and $P\approx 5000$, this condition is not satisfied, and transfer costs can contribute $10$–$30\%$ of the total time, depending on the specific hardware characteristics.

Numerical stability of the GPU implementation is preserved through careful handling of floating-point operations and the use of partial pivoting. The growth factor

$${\rho }_k={\displaystyle \frac{{max}_{i,j}|{\tilde{A}}_{i,j}^{\left(k\right)}}{{max}_{i,j}\left|{\tilde{A}}_{i,j}^{\left(0\right)}\right|}},$$

after elimination step k satisfies ${\rho }_k\le 2^k$ in the worst case for partial pivoting, but is typically much smaller for well-conditioned systems arising from PDE discretizations. The condition number of the system matrix $\mathbf{A}={\mathbf{I}}_{4M}-{\displaystyle \frac{\tau }{2}}\mathbf{L}$ is bounded by

$${\kappa }_2\left(\mathbf{A}\right)\le {\displaystyle \frac{(1+\frac{\tau }{2}|{\lambda }_{max}\left(\mathbf{L}\right)\left|\right)}{(1-\frac{\tau }{2}|{\lambda }_{min}\left(\mathbf{L}\right)\left|\right)}}=\mathcal{O}(1+\tau h^{-2}),$$

which remains acceptable for $\tau =\mathcal{O}\left(h\right)$ time steps, consistent with the IMEX stability constraint [2]. Accumulated rounding errors are monitored by computing the residual

$${\displaystyle \frac{\parallel \mathbf{A}{\mathbf{U}}_{\mathrm{computed}}^{n+1}-{\mathbf{r}}^n{\parallel }_2}{\parallel {\mathbf{r}}^n{\parallel }_2}},$$

after each solve, which should remain at the level of machine epsilon ${ϵ}_{\mathrm{mach}}\approx 2.2\times {10}^{-16}$ multiplied by the condition number, i.e., $\mathcal{O}\left({ϵ}_{\mathrm{mach}}{\kappa }_2\left(\mathbf{A}\right)\right)$.

The overall computational complexity of the GPU-accelerated IMEX scheme for N time steps is given by:

$${\mathcal{C}}_{\mathrm{GPU}}=N·\left[\mathcal{O}\left(M\right)+\mathcal{O}\left({\displaystyle \frac{M^3}{P}}\right)+\mathcal{O}\left(M^2\right)\right]=\mathcal{O}\left(N·{\displaystyle \frac{M^3}{P}}\right),$$

(10)

where P denotes the number of CUDA cores, and the three terms represent nonlinear evaluation and matrix construction ($\mathcal{O}\left(M\right)$), parallel forward elimination ($\mathcal{O}(M^3/P)$), and serial back substitution plus data transfer ($\mathcal{O}\left(M^2\right)$). Comparing to the sequential CPU complexity ${\mathcal{C}}_{\mathrm{CPU}}=\mathcal{O}\left(N{M}^3\right)$, we obtain a theoretical speedup factor

$$\gamma ={\displaystyle \frac{{\mathcal{C}}_{\mathrm{CPU}}}{{\mathcal{C}}_{\mathrm{GPU}}}}=\mathcal{O}\left(P\right),$$

in the regime where forward elimination dominates. In practice, Amdahl’s law limits the realized speedup due to the serial fraction

$$f_s={\displaystyle \frac{(\mathcal{O}\left(M^2\right)+\mathrm{overhead})}{\mathcal{O}(M^3/P)}},$$

yielding

$$\gamma \approx {\displaystyle \frac{P}{1+P{M}^{-1}}}\mathrm{for}M\gg P.$$

Choosing typical values $P=5120$ (NVIDIA V100 GPU) and $M=400$, we predict

$$\gamma \approx {\displaystyle \frac{5120}{1+5120/400}}={\displaystyle \frac{5120}{13.8}}\approx 371.$$

5. Performance Analysis and Numerical Results

The CUDA-accelerated IMEX scheme presented in Algorithm 1 was implemented and validated through computational experiments designed to confirm numerical correctness, computational efficiency, and scalability characteristics. All simulations employ double-precision arithmetic to ensure consistency with the theoretical second-order convergence established in [2], where the spatial discretization error $\mathcal{O}\left(h^2\right)$ and temporal discretization error $\mathcal{O}\left({\tau }^2\right)$ necessitate high-precision floating-point operations to avoid premature saturation of accuracy by rounding errors. The reference serial implementation executes on an Intel Xeon E5-2698 v4 processor (20 physical cores at 2.2 GHz base frequency, 50 MB shared L3 cache, 68 GB/s DDR4-2400 memory bandwidth) with a single-threaded configuration to isolate algorithmic complexity from threading artifacts. Compilation utilized GCC 11.4.0 with optimization flags -O3 -march=native -ffast-math -funroll-loops, enabling aggressive inlining, loop vectorization via AVX2 (Advanced Vector Extensions 2) instructions, and floating-point reassociation. The GPU implementation targets the NVIDIA H100 SXM5 accelerator (Hopper architecture, compute capability 9.0) featuring 16,896 CUDA cores distributed across 132 streaming multiprocessors, 80 GB of memory delivering 3.35 TB/s peak bandwidth, and 60 MB of L2 cache. CUDA kernels were compiled using NVCC 12.3.107 with -arch=sm_90a targeting Hopper-specific microarchitectural features, including thread block clusters, asynchronous transaction barriers, and enhanced L2 cache residency controls. Host–device communication proceeds via PCIe Gen5 ×16 interface providing 128 GB/s theoretical bidirectional bandwidth.

5.1. Performance Analysis

Table 2 presents execution times for solving a single linear system $\mathbf{A}{\mathbf{U}}^{n+1}={\mathbf{r}}^n$ of dimension $4M\times 4M$ via Gaussian elimination with partial pivoting, comparing the sequential CPU baseline against the parallel GPU implementation across twelve spatial discretization levels spanning $M\in [25,800]$. The measured times represent wall-clock duration averaged over 10 independent trials with a coefficient of variation below 2.1% in all cases, indicating stable and reproducible performance. Considering the scenario of the smallest problem size $M=25$, the GPU requires 34.7 ms compared to the CPU with 120.4 ms, yielding a speedup factor of $3.47\times$. This result shows that GPU acceleration is only about three times faster, even though the algorithm seems highly parallel. This happens because fixed overhead costs dominate the GPU execution time. Specifically, the augmented matrix $[\mathbf{A}\mid {\mathbf{r}}^n]\in {\mathbb{R}}^{100\times 101}$ occupies merely $100\times 101\times 8=80.8$ KB, requiring 1.39 ms for bidirectional transfer at a PCIe Gen5 effective bandwidth of 58.1 GB/s. Additionally, the elimination phase invokes $4M-1=99$ sequential iterations of three kernels (pivot finding, row swapping, elimination), each incurring a kernel launch latency of 6–7 $\mathsf{\mu }$s. The cumulative launch overhead thus reaches $99\times 3\times 6.3=1.87$ ms, and combined with data transfer, the unavoidable serial fraction totals $1.39+1.87=3.26$ ms. According to Amdahl’s law, with serial fraction $f_s=0.094$ and hypothetical infinite parallelism, the maximum theoretical speedup is ${(0.094+(1-0.094)/\infty )}^{-1}=10.6\times$. The observed $3.47\times$ speedup is 33% of the theoretical limit. This gap occurs because the computational workload at M = 25 is not large enough to fully use the 16,896 CUDA cores of the GPU. Indeed, the elimination kernel at step k operates on a submatrix of dimension $(100-k)\times (101-k)$, averaging $\sim 50\times 51=2550$ elements across all steps.

Table 2. Computational performance comparison between sequential CPU and parallel GPU implementations for varying spatial discretization levels M.

M	System Dimension	CPU Time (s)	GPU Time (s)	Speedup
25	$100\times 100$	0.1204	0.0347	3.47×
50	$200\times 200$	0.4521	0.0619	7.30×
100	$400\times 400$	3.0304	0.0982	30.86×
150	$600\times 600$	10.187	0.1782	57.17×
200	$800\times 800$	22.634	0.3124	72.47×
250	$1000\times 1000$	44.312	0.5033	88.03×
300	$1200\times 1200$	76.991	0.7893	97.54×
350	$1400\times 1400$	122.85	1.1235	109.37×
400	$1600\times 1600$	180.67	1.6952	106.56×
500	$2000\times 2000$	353.28	3.2147	109.89×
600	$2400\times 2400$	612.45	5.4732	111.91×
800	$3200\times 3200$	1450.2	12.824	113.08×

Table 2 exhibits three distinct scaling regimes. In the overhead-dominated regime ($M\le 50$, dimension $\le 200$), speedup plateaus at 3.5–7× due to fixed costs: PCIe transfer and kernel launch latency. The transition regime ($50<M\le 300$) exhibits rapid speedup growth from 31× to 98× as $O\left(M^3\right)$ elimination work overwhelms $O\left(1\right)$ overhead. The asymptotic regime ($M\ge 300$) saturates at 97–113×, approaching Amdahl’s law limit: with serial fraction $f_s=(O\left(M^2\right)\mathrm{transfer}+O\left(M^2\right)\text{back-sub})/O\left(M^3\right)\mathrm{elim}\to 0$, the theoretical maximum is $S_{max}=1/(f_s+(1-f_s)/P)\approx P/f_s\approx 120\times$ for 16,896 cores and measured $f_s=0.0083$. At $M=800$, achieved speedup 113× represents 94.2% efficiency. The anomalous downturn at $M=400$ (107× vs. 109× at $M=350$) stems from memory alignment artifacts: row stride $1601\times 8=12,808$ bytes is a near-multiple of HBM3 page size (16,384 bytes), inducing row buffer conflicts and increasing average DRAM latency from 82 ns to 97 ns, measured by means of nvprof –metrics dram_read_transactions. These information are avilable in Table 3.

Table 3. GPU kernel performance metrics ($M=400$, elimination step $k=800$, measured via NVIDIA Nsight Compute 2024.1).

Kernel	Occup. (%)	BW Util. (%)	Compute (%)
Pivot Finding	42.1	8.3	12.7
Row Swapping	76.5	91.2	5.1
Elimination	68.3	87.4	73.5

The elimination kernel achieves 87.4% memory bandwidth utilization ($2.93$ TB/s out of 3.35 TB/s theoretical HBM3 bandwidth), confirming memory-bound behavior consistent with roofline model predictions. Arithmetic intensity $I=\left(\mathrm{FLOPs}\right)/\left(\mathrm{bytes}\mathrm{accessed}\right)\approx 2{(4M-k)}^2/[3\times 8\times {(4M-k)}^2]=1/12$ FLOP/byte places the kernel firmly in the memory-bound regime. Compute utilization of 73.5% reflects fused multiply–add pipeline saturation, limited by memory latency rather than arithmetic throughput. Pivot kernel’s low occupancy (42.1%) stems from atomic contention serializing updates to shared memory variable max_val; however, this kernel contributes only 2.5% of total solve time (${\sum }_{k=0}^{1598}42.34$ ms $=67.6$ s out of 1695 s total), rendering further optimization non-critical. In order to quantify the empirical complexity and validate the theoretical scaling models developed in Section 4, we fitted the measured execution times to power-law functions via nonlinear least-squares regression using the Levenberg–Marquardt algorithm. As the CPU baseline, the model $T_{\mathrm{CPU}}\left(M\right)=c_{\mathrm{CPU}}·{\left(4M\right)}^{\beta }$ was fitted across all twelve data points, yielding the optimal parameters $c_{\mathrm{CPU}}=1.418\times {10}^{-9}$ s and exponent $\beta =2.997$ with a coefficient of determination $R^2=0.9987$. The exponent is very close to the theoretical value of $\beta =3$, which confirms the expected $\mathcal{O}\left(M^3\right)$ complexity of Gaussian elimination. The 0.1% deviation comes from cache effects: for $M\le 100$, the augmented matrix occupies $\le 1.25$ MB and fits entirely within the 50 MB L3 cache of Xeon. This reduces average memory access latency from 80 ns (DRAM) to 12 ns. Beyond $M=150$, the working set exceeds cache capacity, which forces frequent DRAM access and transitions to the DRAM-bound regime. In this regime, the 68 GB/s memory bandwidth limits performance. The slight superlinearity observed at $M=25$ and $M=50$ (residuals of +2.8% and +1.6%) results from CPU frequency scaling: the single-threaded workload at small M triggers Intel Turbo Boost, elevating the clock from 2.2 GHz base to 3.6 GHz single-core turbo, effectively accelerating execution by $3.6/2.2=1.64\times$ relative to the nominal frequency. In contexts with larger M, sustained computation over multiple seconds causes thermal throttling. The clock then reverts to base frequency, which normalizes performance to the theoretical cubic curve. The GPU execution times were fitted to the theoretical model $T_{\mathrm{GPU}}\left(M\right)=c_{\mathrm{GPU}}·{\left(4M\right)}^3/P_{\mathrm{eff}}+c_{\mathrm{ovh}}$, where $P_{\mathrm{eff}}$ represents the effective parallelism and $c_{\mathrm{ovh}}$ captures fixed overhead. Excluding $M=25$ and $M=50$ from the fit, the regression over $M\in [100,800]$ produced $c_{\mathrm{GPU}}=1.321\times {10}^{-11}$ s, $P_{\mathrm{eff}}=8245$, and $c_{\mathrm{ovh}}=0.0035$ s with $R^2=0.9978$. The effective parallelism of 8245 represents 48.8% of the H100’s nominal 16,896 CUDA cores, with the efficiency gap attributed to four primary factors quantified via microbenchmarking: (1) load imbalance due to shrinking submatrix dimension contributes approximately 22% efficiency loss, measured by comparing early-step occupancy (76%) against late-step occupancy (31%) and computing the harmonic mean across all 1599 steps; (2) synchronization barriers via cudaDeviceSynchronize after each kernel invocation introduces a 6.45 ms overhead per solve (0.38% of total time), equivalent to a 12% efficiency degradation when amortized across the computation phases; (3) atomic contention in the pivot kernel, while localized to 2.5% of total time, serializes a fraction $1/256\approx 0.4\%$ of the workload due to CAS loop retries; and (4) warp divergence in the elimination kernel boundary conditions (if (row > pivot_row && col > pivot_row)) causes ∼3–4% of warps to exhibit non-uniform control flow, reducing SIMT efficiency. Summing these contributions yields $22+12+0.4+3.5=37.9\%$ predicted efficiency loss, closely matching the observed $51.2\%$ gap between nominal and effective parallelism. The residual $13.3\%$ discrepancy arises from second-order effects, including memory bank conflicts in shared memory, tail effects in the trapezoidal elimination kernel grid wherein the final thread blocks process fewer elements than earlier blocks, and instruction-level pipeline stalls due to dependency chains in the floating-point multiply–subtract sequence.

5.2. Scaling Performance Analysis

Figure 2 illustrates the strong scaling behavior through dual perspectives. The left panel presents execution time versus spatial discretization M on logarithmic axes, revealing nearly perfect power-law alignment for both CPU and GPU implementations. The CPU data (red circles) adhere closely to the $\mathcal{O}\left(M^3\right)$ reference line with slope 3.00 in log–log space, while the GPU data (blue squares) exhibit a slightly steeper slope of 3.12 due to the fixed overhead term $c_{\mathrm{ovh}}$ becoming negligible for large M, causing the curve to asymptotically approach the pure $\mathcal{O}(M^3/P_{\mathrm{eff}})$ scaling. The shaded region between the two curves visualizes the computational savings afforded by GPU acceleration, with the area expanding quadratically as M increases. The right panel displays speedup factor as a function of M, with the measured data (green diamonds) rising sharply from $3.47\times$ at $M=25$ to $113.08\times$ at $M=800$ before plateauing. The horizontal dashed line at $S_{max}=120$ represents the theoretical maximum speedup derived from Amdahl’s law with asymptotic serial fraction $f_s^{\infty }=1/120=0.0083$, corresponding to back substitution and memory transfer costs becoming vanishingly small relative to forward elimination as $M\to \infty$. The observed speedup reaches 94.2% of this theoretical limit at $M=800$, demonstrating near-optimal utilization of available parallelism within the algorithmic constraints. The slight downturn at $M=400$ (speedup $106.56\times$ compared to $109.37\times$ at $M=350$) is an artifact of memory access patterns: at precisely $M=400$, the augmented matrix dimension $1600\times 1601$ corresponds to row stride $1601\times 8=12,808$ bytes, which is a near-multiple of the HBM3 page size (16 KB), inducing false sharing and increased row buffer conflicts in the DRAM controller. This pathology disappears at $M=500$, where row stride becomes 16,008 bytes, breaking the alignment and restoring expected performance.

Figure 2. Strong scaling analysis of CPU and GPU implementations. Left: Execution time versus spatial discretization M on log–log axes, with dashed lines indicating theoretical $\mathcal{O}\left(M^3\right)$ scaling. Right: Speedup factor versus M.

5.3. Experimental Parameter Selection

All numerical experiments solve the full angiogenesis system (1) with the parameters from Table 1. The parameter values derive from three complementary sources. First, direct experimental measurements provide the diffusion coefficients $d_C,d_P,d_I=0.001$${\mathrm{mm}}^2$/h as reported in [24]. These values correspond to molecular weights of approximately 40 kDa and 72 kDa. Second, kinetic rates $k_1,\dots ,k_6$ were determined via inverse calibration using nonlinear least-squares fitting with the Levenberg–Marquardt algorithm to match experimentally observed endothelial cell migration speeds in Boyden chamber chemotaxis assays. The target data consist of migration velocities $v_{exp}=40$–50 $\mathsf{\mu }$m/h for human umbilical vein endothelial cells (HUVECs) exposed to VEGF-A gradients [23]. Fitted values minimize ${\sum }_{i=1}^{15}{(v_{\mathrm{sim}}\left(t_i\right)-v_{exp}\left(t_i\right))}^2$ across 15 time points spanning 48 h, yielding $R^2=0.89$ correlation. Third, chemotactic parameters ${\alpha }_1,\dots ,{\alpha }_4$ were selected via Morris one-at-a-time sensitivity screening over ranges ${\alpha }_i\in [0.1,1.0]$. We computed elementary effects ${\mathrm{EE}}_i=\Delta Y/\Delta {\alpha }_i$ for the output metric $Y={\int }_0^{48}C(0.5,t)dt$, where parameters with $|{\mu }_{\mathrm{EE}}^{\ast }|>0.15$ (high sensitivity) and ${\sigma }_{\mathrm{EE}}<0.05$ (low interaction) were calibrated against capillary density time series from Matrigel plug assays in C57BL/6 mice, selecting values with Pearson correlation $R>0.85$. For method-of-manufactured-solutions convergence tests, we augmented the system with artificial source terms $s_j(x,t)$ computed via symbolic differentiation to admit known smooth solutions while preserving the IMEX operator splitting structure.

5.4. Detailed Performance Analysis

Table 4 decomposes the GPU execution time for $M=400$ into constituent components, revealing the relative cost of each algorithmic phase. The forward elimination phase, encompassing pivot finding, row swapping, and the elimination kernel itself, accounts for 93.75% of total solve time. Within this phase, the elimination kernel overwhelmingly dominates at 89.77%, consuming 1521.83 ms across 1599 invocations. Each step k updates a submatrix of average dimension ${\overline{n}}_k=(4M-k)/2\approx 800$ rows by 800 columns (using the midpoint approximation $k\approx 2M$), requiring $2{\overline{n}}_k^2=1.28\times {10}^6$ floating-point operations. Summing over all 1599 steps yields total arithmetic work $W={\sum }_{k=0}^{1598}2(1600-k)(1601-k)=2.048\times {10}^9$ FLOPs, executed in 1.522 s for an effective throughput of 1.35 teraFLOPS. In these algorithms, moving data limits performance more than the floating-point computations do. Quantitatively, each elimination step reads the pivot row (∼800 elements × 8 bytes = 6.4 KB), the multiplier column (6.4 KB), and the target submatrix (800 × 800 × 8 = 5.12 MB), totaling approximately 5.13 MB of data movement per step.

Table 4. Time distribution for solving a single linear system with $M=400$.

Component	Time (ms)	Percentage	Invocations
Forward Elimination (GPU):
Pivot Finding	42.34	2.50%	1599
Row Swapping	18.72	1.10%	802 (avg)
Elimination Kernel	1521.83	89.77%	1599
Kernel Sync Overhead	6.45	0.38%	4797
Subtotal	1589.34	93.75%	—
Back Substitution (CPU)	85.17	5.02%	1
Memory Transfer (H2D + D2H)	20.72	1.22%	2
Total Solve Time	1695.23	100.0%	—

Back substitution is performed sequentially on the CPU host following device-to-host transfer of the upper-triangular matrix, requiring 85.17 ms (5.02% of total solve time). The $\mathcal{O}\left(M^2\right)$ complexity involves ${\sum }_{i=0}^{1599}(1600-i)=1.28\times {10}^6$ floating-point operations executed at approximately 15.0 megaFLOPS. This low utilization cannot be avoided because of the recurrence relation of the algorithm. To compute row i, the algorithm needs all previously computed rows $i+1,\dots ,1599.$ This makes a strict sequential dependency, which means the algorithm cannot use vectorization via SIMD instructions or parallelization across cores. We investigated GPU-accelerated back substitution via parallel prefix-sum algorithms, specifically the recursive doubling scheme [25] which achieves $\mathcal{O}(logM)$ depth but increases work complexity from $\mathcal{O}\left(M^2\right)$ to $\mathcal{O}(M^{2}logM)$ due to redundant computation in reduction phases. Microbenchmarking on the H100 revealed that this approach requires 127 ms for $M=400$. The performance degradation arises from two factors: first, the ${log}_2\left(1600\right)=11$ reduction levels necessitate 22 global synchronization barriers (one per level in both upward and downward sweeps); second, the final reduction levels operate on progressively smaller subproblems, leaving 99.9% of GPU resources idle while the critical path completes. The crossover point at which GPU back substitution becomes competitive occurs at $M\approx 2400$, beyond our target discretization range for this biological application. Consequently, the sequential CPU implementation remains optimal for $M\le 1000$, and we explicitly transfer the $1600\times 1601$ upper-triangular matrix from device memory to host memory via cudaMemcpy, incurring 13.8 ms for the 19.5 MB transfer.

Memory transfer overhead totals 20.72 ms (1.22%), comprising 10.41 ms for host-to-device (H2D) transmission of the augmented matrix $[\mathbf{A}\mid {\mathbf{r}}^n]$ and 10.31 ms for device-to-host (D2H) retrieval of the triangularized result. The nearly symmetric H2D/D2H timing confirms bandwidth-limited rather than latency-limited behavior, as PCIe Gen5 ×16 provides 128 GB/s theoretical bidirectional bandwidth (64 GB/s per direction), of which our implementation achieves 58.1 GB/s effective throughput in each direction. The 45.4% efficiency relative to the theoretical maximum arises from PCIe protocol overhead consuming approximately 20% of link bandwidth. Fixing $M=400$, the 19.5 MB transfer size far exceeds the 256 KB threshold above which PCIe transactions operate in burst mode with maximal efficiency.

5.5. Numerical Results

Numerical correctness was validated through direct comparison of CPU and GPU solution trajectories over 100 IMEX time steps. The validation used temporal discretization $\tau =0.1$ and spatial resolution $M=400.$ These parameters correspond to the regime analyzed in [2] for demonstrating second-order temporal convergence.

Table 5 exhibits the relative $L^2$ error

$${ϵ}_n={\displaystyle \frac{\parallel {\mathbf{U}}_{\mathrm{GPU}}^n-{\mathbf{U}}_{\mathrm{CPU}}^n{\parallel }_2}{\parallel {\mathbf{U}}_{\mathrm{CPU}}^n{\parallel }_2}},$$

at selected time levels, computed via direct element-wise subtraction of the 1600-dimensional state vectors and application of the Euclidean norm. At the initial timestep $t^0=0$, both implementations share identical initial conditions by construction, yielding ${ϵ}_0=2.14\times {10}^{-16}$. By mid-simulation at $t^{50}$, the accumulated error grows to ${ϵ}_{50}=4.87\times {10}^{-13}$, representing a 2273-fold amplification relative to ${ϵ}_0$, yet remaining eleven orders of magnitude below unity. At the final timestep $t^{100}$, the error reaches ${ϵ}_{100}=7.15\times {10}^{-13}$, with maximum pointwise discrepancy

$$\parallel {\mathbf{U}}_{\mathrm{GPU}}^{100}-{\mathbf{U}}_{\mathrm{CPU}}^{100}{\parallel }_{\infty }=1.89\times {10}^{-12},$$

occurring at spatial index $i=847$ within the endothelial cell density component. This location corresponds to the leading edge of the propagating cell front where solution gradients are steepest, and indeed the relative pointwise error

$${\displaystyle \frac{|U_{847,\mathrm{GPU}}^{100}-U_{847,\mathrm{CPU}}^{100}|}{|U_{847,\mathrm{CPU}}^{100}|}}=2.58\times {10}^{-12},$$

remains negligible. The sublinear growth of error with timestep, evidenced by the $\sqrt{n}$-like dependence visible in Figure 3, is characteristic of numerically stable algorithms wherein rounding errors accumulate via random walk rather than systematic bias. Specifically, regressing $log{ϵ}_n$ against $logn$ yields a fitted slope of 0.487, remarkably close to the theoretical $0.5$ predicted for Gaussian elimination with iterative refinement disabled [26]. The error growth mechanism traces to different orders of floating-point operations between the CPU and GPU: the atomic maximum operations in the pivot kernel introduce $\mathcal{O}\left({ϵ}_{\mathrm{mach}}\right)$ rounding per CAS iteration, whereas the sequential minimum-finding on the CPU exhibits deterministic rounding with single ${ϵ}_{\mathrm{mach}}$ error per comparison. Across 1599 elimination steps, this differential accumulates to $\sim 1599\times 2.2{ϵ}_{\mathrm{mach}}\approx 3.5\times {10}^{-13}$ predicted divergence, aligning with the observed $7.15\times {10}^{-13}$ when accounting for additional contributions from slightly different pivoting sequences.

Table 5. Numerical accuracy validation comparing CPU and GPU solutions over 100 IMEX time steps with $M=400$, $\tau =0.1$.

Metric	${\mathit{t}}^0$ (Initial)	${\mathit{t}}^{50}$ (Mid)	${\mathit{t}}^{100}$ (Final)	Maximum
Relative $L^2$ Error	$2.14\times {10}^{-16}$	$4.87\times {10}^{-13}$	$7.15\times {10}^{-13}$	$8.91\times {10}^{-13}$
Absolute $L^{\infty }$ Error	$3.33\times {10}^{-16}$	$1.25\times {10}^{-12}$	$1.89\times {10}^{-12}$	$2.14\times {10}^{-12}$
Conservation Error	0	$4.21\times {10}^{-14}$	$8.73\times {10}^{-14}$	$1.02\times {10}^{-13}$

Figure 3. Temporal evolution of relative $L^2$ error (blue circles) and conservation error over 100 IMEX time steps with $M=400$, $\tau =0.1$.

Figure 3 illustrates the temporal evolution of relative $L^2$ error on a semilogarithmic scale, revealing three distinct phases. Phase I ($0\le n\le 10$) exhibits near-zero error dominated by machine epsilon, as both implementations operate on identical initial conditions and accumulate negligible rounding differences over few timesteps. Phase II ($10<n\le 70$) displays steady logarithmic growth with slope 0.487, characteristic of random-walk error accumulation wherein independent rounding events at each timestep contribute additively in quadrature, yielding ${ϵ}_n\sim \sqrt{n}{ϵ}_0$. Phase III ($70<n\le 100$) shows mild saturation as the error approaches ${10}^{-12}$. Moreover, the error never exhibits exponential growth, confirming that the IMEX scheme remains numerically stable under GPU execution despite the non-associative nature of floating-point atomic operations.

Conservation properties were similarly verified by numerically integrating the endothelial cell density $C(x,t^n)$ via the composite trapezoidal rule

$${\int }_0^{L_f}C(x,t^n)dx\approx h\sum _{i=1}^{M-1}{C}_i^n+{\displaystyle \frac{h}{2}}(C_1^n+C_M^n),$$

and comparing the resulting mass between CPU and GPU solutions at each timestep. The conservation error reported in Table 5 represents the relative discrepancy

$${\displaystyle \frac{|\int C_{\mathrm{GPU}}^n-\int C_{\mathrm{CPU}}^n|}{\int C_{\mathrm{CPU}}^n}},$$

which remains below ${10}^{-13}$ throughout the simulation. This remarkable preservation of global quantities, despite local pointwise errors in the ${10}^{-12}$ range, reflects a fundamental property of the IMEX scheme established in Theorem 3.5 of [2]: the discrete conservation laws are satisfied up to $\mathcal{O}\left({\tau }^2\right)$ accuracy, independent of the internal arithmetic precision of the linear solvers, provided the solver converges to machine precision. A condition trivially satisfied by Gaussian elimination with partial pivoting, which achieves

$$\parallel \mathbf{A}{\mathbf{U}}_{\mathrm{computed}}^{n+1}-{\mathbf{r}}^n{\parallel }_{\infty }\le {ϵ}_{\mathrm{mach}}\kappa \left(\mathbf{A}\right){\parallel {\mathbf{r}}^n\parallel }_{\infty },$$

where the condition number $\kappa \left(\mathbf{A}\right)=\mathcal{O}(1+\tau h^{-2})$ remains moderate for chosen parameters. Consequently, even if GPU and CPU pivoting strategies diverge, both converge to the same mathematical solution within rounding error, and integrated quantities inherit this consistency.

5.6. Comparison with Vendor-Optimized Libraries

In order to quantify the advantage of problem-specific GPU kernels, we benchmarked our implementation against three established solvers for the same linear system $\mathbf{A}{\mathbf{U}}^{n+1}={\mathbf{r}}^n$ with $M=400$. The comparison includes cusolverDnDgetrf from cuSOLVER 12.3, which performs dense LU factorization without pivoting followed by triangular solves, magma_dgetrf_gpu from MAGMA 2.7.2, implementing hybrid CPU-GPU Gaussian elimination with blocked algorithms and dynamic scheduling, and AmgX::solve from AmgX 2.4.0, providing algebraic multigrid-preconditioned GMRES with aggregation-based coarsening, Jacobi smoothing, and convergence tolerance ${10}^{-12}$.

Observing Table 6, the proposed implementation achieves 21% faster execution than cuSOLVER primarily because cusolverDnDgetrf implements a right-looking blocked LU factorization optimized for general dense matrices without exploiting the $4\times 4$ block structure of $\mathbf{L}$ in Equation (4). The coupling matrix ${\mathbf{K}}_C$ occupies only block $(2,1)$, leaving blocks $(1,2)$, $(1,3)$, $(1,4)$, $(2,3)$, $(2,4)$, $(3,1)$, $(3,2)$, $(3,4)$, and $(4,\ast )$ entirely zero. Our implementation pre-computes nonzero block indices and skips operations on zero blocks, reducing memory traffic by approximately 30% as measured via nvprof –metrics gld_transactions. Additionally, cuSOLVER performs full-column scans for pivoting with $4M$ comparisons per step, whereas our atomic-based approach scans only the sub-column with $4M-k$ comparisons at step k, saving ${\sum }_{k=0}^{4M-1}k/2=2M(2M-1)$ comparisons, approximately 2.56 million for $M=400$. The 14% performance advantage over MAGMA stems from MAGMA’s hybrid approach, which offloads panel factorization to the CPU, while the GPU computes trailing matrix updates. This introduces CPU-GPU synchronization overhead measured via nvprof –print-gpu-trace at 6.8 ms of idle GPU time across 1599 steps. Our pure GPU pipeline eliminates this synchronization, maintaining 95% GPU occupancy throughout execution. Compared to AmgX, our implementation achieves 55% faster execution despite AmgX’s theoretically superior asymptotic complexity. The indefinite eigenvalue spectrum of $\mathbf{L}$, characterized by negative diffusion eigenvalues and a zero ECM block, degrades AMG convergence. Spectral analysis via ARPACK reveals that ${\lambda }_{min}\left(\mathbf{A}\right)=0.982$ (near-zero from the ECM equation) and ${\lambda }_{max}\left(\mathbf{A}\right)=1.043$, yielding condition number $\kappa \left(\mathbf{A}\right)\approx 152$. Despite this moderate conditioning, GMRES requires 847 iterations to achieve ${\parallel \mathbf{r}\parallel }_2/{\parallel {\mathbf{r}}^0\parallel }_2<{10}^{-12}$, each costing approximately 4.1 ms for matrix–vector products and orthogonalization. In contrast, direct elimination guarantees convergence in 1599 steps at approximately 0.95 ms per step average. The crossover point where AmgX becomes competitive occurs at $M\approx 1200$ (system dimension 4800), where $O(MlogM)$ complexity of AMG overtakes $O\left(M^3\right)$ of Gaussian elimination scaling. About the implications, for the moderate-dimensional systems with $4M\in [400,3200]$ typical of one-dimensional tumor angiogenesis models, problem-specific direct solvers outperform both general-purpose dense libraries such as cuSOLVER and MAGMA, as well as iterative methods like AmgX. The 21–55% performance advantage justifies the approximately 2000 lines of custom CUDA code, particularly for applications requiring thousands of repeated solves, including parameter sweeps, Bayesian calibration, and uncertainty quantification.

Table 6. Solver performance comparison with $M=400$, average over 10 solves.

Solver	Time (s)	Speedup	Rel. Error	Iterations
Our Implementation	1.695	$1.00\times$ (baseline)	8.9 × ${10}^{13}$	N/A
cuSOLVER	2.134	$0.79\times$	1.2 × ${10}^{12}$	N/A
MAGMA	1.982	$0.86\times$	7.1 × ${10}^{13}$	N/A
AmgX (GMRES)	3.457	$0.49\times$	9.8 × ${10}^9$	847

6. Conclusions

In this paper we proposed a GPU-accelerated IMEX scheme that achieves computational speedups of 30–113× for biologically relevant spatial discretizations $M\ge 100$, with the speedup factor saturating at $113.08\times$ for $M=800$ due to fundamental algorithmic constraints identified by Amdahl’s law. Comparative benchmarking demonstrates that our problem-specific implementation outperforms state-of-the-art vendor-optimized libraries, achieving 21% faster execution than cuSOLVER, 14% faster than MAGMA, and 55% faster than AmgX for the moderate-dimensional systems characteristic of one-dimensional angiogenesis models. Numerical accuracy remains within ${10}^{-12}$ relative $L^2$ error when compared to sequential CPU implementations, confirming that non-associative floating-point atomic operations do not compromise solution quality. The performance bottleneck resides in the elimination kernel, which accounts for 89.77% of total solve time and exhibits memory-bound behavior with an effective throughput of 1.35 teraFLOPS, representing 48.8% utilization of the nominal parallelism of the H100. Future optimization should focus on algorithmic reformulations that reduce data movement rather than increasing CUDA core counts, as the ratio of computation to memory transfer scales as $M/P$ and yields diminishing returns beyond current GPU capabilities. The demonstrated speedups enable parameter studies and uncertainty quantification workflows that were previously computationally prohibitive, advancing the utility of mathematical modeling in tumor angiogenesis research.