Automatic Sparse Matrix Format Selection via Dynamic Labeling and Clustering on Heterogeneous CPU–GPU Systems

Abstract

Sparse matrix–vector multiplication (SpMV) is a fundamental kernel in high-performance computing (HPC) whose efficiency depends heavily on the storage format across central processing unit (CPU) and graphics processing unit (GPU) platforms. Conventional supervised approaches often use execution time as training labels, but our experiments on 1786 matrices reveal two issues: labels are unstable across runs due to execution-time variability, and single-label assignment overlooks cases where multiple formats perform similarly well. We propose a dynamic labeling strategy that assigns a single label when the fastest format shows clear superiority, and multiple labels when performance differences are small, thereby reducing label noise. We further extend feature analysis to multi-dimensional structural descriptors and apply clustering to refine label distributions and enhance prediction robustness. Experiments demonstrate 99.2% accuracy in hardware (CPU/GPU) selection and up to 98.95% accuracy in format prediction, with up to 10% robustness gains over traditional methods. Under cost-aware, end-to-end evaluation that accounts for feature extraction, prediction, conversion, and kernel execution, CPUs achieve speedups up to 3.15× and GPUs up to 1.94× over a CSR baseline. Cross-round evaluations confirm stability and generalization, providing a reliable path toward automated, cross-platform SpMV optimization.

1. Introduction

Sparse matrix–vector multiplication (SpMV) is a fundamental computational kernel that is widely used in scientific computing, graph analysis [1,2,3,4], and machine learning [5,6,7,8,9]. The efficiency of SpMV is strongly influenced by the chosen storage format, such as Coordinate (COO), Compressed Sparse Row (CSR), Compressed Sparse Column (CSC), and Block Sparse Row (BSR). Each format exhibits different performance according to the sparsity pattern, the hardware platform, and the memory hierarchy [10,11], making the automatic selection of an optimal format a challenging problem [9,12,13]. Practically, robust automatic format/device selection reduces manual tuning and prevents negative returns due to format-conversion overheads on heterogeneous CPU–GPU systems. As illustrated in Figure 1, these formats adopt different indexing strategies, which directly affect memory access efficiency and computational performance.

Over the past decade, significant research has been devoted to optimizing SpMV performance across CPUs and GPUs. Early studies investigated static heuristics and handcrafted rules for format selection, but these approaches often failed to generalize to diverse datasets. More recent works have explored adaptive and machine learning-based methods. For instance, Zhao et al. proposed bridging deep learning models with sparse matrix format selection to improve portability and robustness [13]. Similarly, Zhou et al. developed DTSpMV, an adaptive framework for graph analysis that dynamically selects formats at runtime [14]. In parallel, Stylianou and Weiland emphasized the importance of performance portability by leveraging dynamic sparse matrices in portable linear algebra operations [15].

In addition to adaptive frameworks, surveys have systematically reviewed the landscape of SpMV optimization, highlighting the trade-offs among computation, memory access, and preprocessing overhead [7,9,10]. Other works, such as that by Wang et al., focused on overlapping memory copy and computation to further enhance GPU SpMV performance [16], while Ashoury et al. presented Auto-SpMV, a fully automated optimization pipeline for GPU kernels [17]. Moreover, Chen proposed hybrid storage formats for CPUs that adaptively combine different representations to maximize throughput [18], Our prior work [19] was the first to introduce a dynamic-labeling-based classification method (DyLaClass), which achieved high classification accuracy and robust performance on CPU platforms. Collectively, these studies underline the necessity of flexible and adaptive strategies for exploiting heterogeneous architectures.

Despite these advances, two persistent challenges remain. First, the reproducibility of “best format” labels is problematic: repeated experiments on the same matrix may yield different optimal formats due to the variability of the execution time. Second, current methods often assign a single best format, overlooking cases where multiple formats perform similarly well. These issues are illustrated in Figure 2, where two rounds of execution (each round averages 50 runs) demonstrate that the optimal formats of four real matrices can change between rounds. Furthermore, for matrices such as mesh1em1.mtx, several formats achieve nearly identical performance, highlighting that assigning only one “best format” may oversimplify the problem and reduce prediction robustness.

To address these limitations, our work makes the following contributions:

(1)

We generalize dynamic labeling to a true multi-label setting. For matrix A and format set $\mathcal{F}$, we define $\mathcal{L}\left(A\right)=\{f\in \mathcal{F}\mid T_f\left(A\right)\le (1+\delta )T_{min}\left(A\right)\}$, which admits any number of near-optimal formats within threshold $\delta$ (not just one runner-up).

(2)

We propose clustering-based consolidation in feature spaces beyond 2D. Besides the 2D plane, we use 3D (and optionally higher-k) spaces formed by top F-score features to refine label distributions. Fixed-label samples provide class centroids; multi-label samples are assigned to the nearest centroid among their candidate labels, reducing label noise and improving classifier stability.

(3)

We extend the study from CPUs to a unified CPU–GPU pipeline. We first select the device (Model S) and then the format, explicitly accounting for prediction time ($T_p$), conversion time ($T_c$), and execution time ($T_e$), with feature extraction time ($T_f$) executed in parallel with Model S. On a dataset of 1786 matrices, we show that multi-label dynamic labeling plus feature-based clustering substantially improves prediction stability and accuracy on both CPUs and GPUs.

2. Materials and Methods

Before diving into the details of dynamic labeling, feature extraction, and classification, it is essential to provide an overview of the entire experimental workflow. Figure 3 presents the overall framework: starting from an input sparse matrix, the system first decides whether the computation should be executed on the CPU or GPU. Next, feature extraction is performed to assist format prediction. To reduce overhead, the feature extraction time $T_f$ and the prediction time $T_p$ are executed in parallel (indicated by the dashed box in the figure). If the original format of the input matrix is already the optimal one, the system proceeds directly to computation; otherwise, the matrix is converted into the predicted format before the SpMV execution. This workflow integrates format prediction, dynamic labeling, and clustering-based optimization into a unified framework, providing the foundation for the subsequent sections.

2.1. Problem Formulation

Sparse matrix–vector multiplication (SpMV) is highly sensitive to the choice of storage format (e.g., COO/CSR/CSC/BSR) and to the target device (CPU or GPU). We model the end-to-end latency by explicitly accounting for (i) device selection, (ii) feature extraction, (iii) format prediction, (iv) format conversion, and (v) SpMV execution. Importantly, the feature extraction time and the device selection time proceed in parallel in our pipeline.

For a given sparse matrix A, the total latency is

$$T\left(A\right)=max\{T_s\left(A\right),T_f\left(A\right)\}+T_p\left(A\right)+T_c\left(A\right)+T_e\left(A\right),$$

(1)

where

• $T_s\left(A\right)$: device selection time (Model S; CPU vs. GPU);
• $T_f\left(A\right)$: feature extraction time;
• $T_p\left(A\right)$: prediction time of the format-selection model (Model P);
• $T_c\left(A\right)$: format conversion time (if needed);
• $T_e\left(A\right)$: SpMV execution time.

Compared with formulations that sums $T_f$ and $T_p$, our pipeline executes $T_s$ and $T_f$ in parallel, reducing their contribution to $max\{·\}$ and reflecting the actual scheduling in Figure 3.

Let $S=\{\mathrm{COO},\mathrm{CSR},\mathrm{CSC},\mathrm{BSR}\}$ be the set of formats and F be the feature space. For a candidate format $s\in S$ and feature vector $f\in F$, the end-to-end cost becomes

$$T_{\mathrm{total}}(s,f)=max\{T_s,T_f\left(f\right)\}+T_p\left(f\right)+T_c\left(s\right)+T_e\left(s\right),$$

(2)

where $T_f$ and $T_p$ depend on f (Model P uses the extracted features), while $T_c$ and $T_e$ depend on s. The optimization target is then

$$(s^{*},f^{*})\in arg\underset{s\in S,f\in F}{min}{T}_{\mathrm{total}}(s,f):=\left\{(s,f)|T_{\mathrm{total}}(s,f)\le T_{\mathrm{total}}(x,y),\forall x\in S,\forall y\in F\right\}.$$

(3)

In the sequel, we use $T_f$, $T_s$, $T_p$, $T_c$, $T_e$, and $T_{\mathrm{total}}$ for brevity to denote the corresponding terms in Equation (2).

2.2. Design of Model S

The first step of our framework is to determine whether a sparse matrix should be executed on the CPU or GPU. Small matrices tend to favor CPUs because feature-extraction and format-conversion overheads are comparable to execution time, whereas large matrices benefit from GPU throughput; Model S encodes this regime-aware decision. This decision is crucial because the efficiency of sparse matrix–vector multiplication (SpMV) depends not only on the selected storage format but also on the underlying hardware platform. Executing on the wrong platform may result in significant overhead due to suboptimal utilization of memory bandwidth, cache hierarchy, or parallelism. Therefore, designing an effective model to select the appropriate hardware platform, referred to as Model S, is a fundamental component of our framework.

To construct Model S, we collected execution time data for 1786 sparse matrices on both CPU and GPU platforms. The execution time includes four components: feature extraction ($T_f$), the prediction overhead of Model S ($T_p$), format conversion ($T_c$), and SpMV execution time ($T_e$). Since $T_f$ and $T_s$ (Model S inference) are executed in parallel, their contribution is considered as $max(T_f,T_p)$ rather than a simple sum. The overall time cost is thereby defined as

$$T_{total}=max(T_f,T_p)+T_c+T_e.$$

(4)

where the role of Model S is to minimize $T_{total}$ by predicting whether a given matrix should be executed on the CPU or GPU.

The impact of hardware selection is illustrated by two representative cases. For a small matrix mesh1em1.mtx with $nnz=306$, the COO execution on GPUs takes nearly 200 times longer than on CPUs due to the dominant cost of data transfer. In contrast, for a large matrix nd6k.mtx with $nnz=6,897,316$, the CSC execution on CPUs is nearly 8 times slower than on GPUs, where massive parallelism compensates for conversion overhead. These examples clearly demonstrate the necessity of Model S to select the proper platform according to matrix characteristics.

To identify the key determinants of hardware selection, we applied XGBoost for feature importance analysis across log-transformed structural attributes. Figure 4(1) shows that the number of nonzeros (nnz) and the number of rows (rows) are the dominant features, followed by $ndiags$, row bounce, and row variance. These features effectively describe the sparsity pattern and workload distribution, both of which strongly impact hardware efficiency.

To further validate the impact of nnz, we compared CPU and GPU performance across different hardware platforms. Figure 4(2) illustrates the density distribution of ${log}_{10}\left(nnz\right)$. Figure 4(2a) corresponds to AMD EPYC 7713 (CPU) with NVIDIA A40 (GPU), while Figure 4(2b) corresponds to an Intel(R) Xeon(R) Gold 6230 CPU @ 2.10 GHz with NVIDIA A40 (GPU). A clear boundary can be observed in both cases: smaller matrices are consistently more efficient on CPUs, whereas larger matrices benefit from GPU acceleration. Although the exact threshold varies across platforms, the overall trend is robust.

Once the hardware platform is determined by Model S, the next challenge is to select the most suitable storage format for SpMV. However, repeated experiments reveal that the reproducibility of the “best format” is problematic, motivating us to propose a dynamic labeling strategy. This strategy, combined with clustering-based refinement, will be introduced in the following sections.

2.3. Dynamic Multi-Labeling and Classification

Generalized Dynamic Labels

In our previous work, we categorized matrices into fixed-label matrices (FLMs) and mixed-label matrices (MLMs).

Let $\mathcal{L}\left(A\right)$ denote the label set of matrix A, determined by its performance across multiple storage formats. If $\left|\mathcal{L}\right(A\left)\right|=1$, A is considered a fixed-label matrix (FLM), where a single format significantly outperforms the others. If $\left|\mathcal{L}\right(A\left)\right|>1$, A is categorized as a multi-label matrix (MLM), where multiple formats achieve comparable execution times within a given threshold. This definition provides the foundation for our dynamic labeling strategy.

To illustrate, consider two representative matrices as shown in Figure 2. For freeFlyingRobot.mtx, the CSR format exhibits a total execution time that is much smaller than the other three formats, thus its label set satisfies $\left|\mathcal{L}\right(A\left)\right|=1$, making it a fixed-label matrix (FLM). In contrast, for mesh1em1.mtx, after two rounds of experiments (each round averaging 50 runs), the total execution times of COO, CSR, CSC, and BSR are extremely close. In this case, $\mathcal{L}\left(A\right)$ includes all four formats, making it a multi-label matrix (MLM).

In our previous work, only the two fastest formats among the multi-label candidates were preserved, but in this study we generalize the scheme to allow more than two labels, thereby accommodating cases where three or even all four formats perform similarly well. For visualization in Figure 5 and Figure 6, we plot the three axes as ${log}_{10}\left(\mathtt{nnz}\right)$, ${log}_{10}\left(\mathtt{rows}\right)$, and ${log}_{10}\left(\mathtt{cols}\right)$; the actual clustering operates on standardized (z-score) features. The clustering visualization belongs to the preprocessing/label-generation stage and is not part of end-to-end timing.

In this work, we extend the scheme to a generalized multi-mixed label strategy, where all formats with execution times within the defined threshold are retained as valid labels. Formally, let $S=\{s_1,s_2,s_3,s_4\}$ denote the set of supported formats (e.g., COO, CSR, CSC, BSR), and $T_e\left(s_i\right)$ denotes the execution time of format $s_i$ for a given matrix. We define the minimum execution time as

$$T_{min}=\underset{s_i\in S}{min}{T}_e\left(s_i\right).$$

(5)

A format $s_i$ is selected as a valid label if its execution time satisfies

$${\displaystyle \frac{T_e\left(s_i\right)-T_{min}}{T_{min}}}\le \theta ,$$

(6)

where $\theta$ is the predefined threshold.

When only one format meets Equation (6), the matrix is considered as a fixed-label matrix (FLM). When multiple formats satisfy the condition, the matrix is considered as a mixed-label matrix (MLM), extended here to allow multiple valid labels beyond two.

2.4. Dynamic Labeling and Clustering

Given the set-valued labels $L\left(A\right)$ defined in Section 2.3, we cluster in 2D/3D feature space around fixed-label centroids and assign multi-label matrices to their nearest centroid. In addition to the traditional 2D plane formed by ${log}_{10}\left(n\right)$ and ${log}_{10}\left(\mathrm{nnz}\right)$, we also consider 3D feature spaces composed of the top-ranked features identified by F-score analysis (e.g., ${log}_{10}\left(n\right)$, ${log}_{10}\left(\mathrm{nnz}\right)$, and ${log}_{10}\left(\mathrm{diags}\right)$). This clustering step reduces label noise, aligns multi-label matrices with stable fixed clusters, and improves classification robustness.

Figure 7 illustrates this process. In the left subfigure, multi-label matrix A (white circle) initially possesses multiple candidate labels due to similar execution times. Through clustering, it is assigned to the nearest fixed-label centroid (colored points). The middle subfigure demonstrates the use of a local radius to filter out dispersed points: the local radius, denoted by r, is drawn as the dashed circle, ensuring that anomalies outside this radius do not distort the clustering. The right subfigure shows how A eventually converges toward the cluster with the majority of consistent labels, thereby improving stability and reducing noise.

Formally, the local radius r in a 3D (or higher-dimensional) feature space is defined as follows.

$$r=\alpha \sqrt{\sum _{f\in F}{\left(\underset{\forall B\in N}{max}{f}_B-\underset{\forall B\in N}{min}{f}_B\right)}^2},$$

(7)

where $f_B$ is the value of feature f for matrix B, N is the set of all matrices in the feature space, and $\alpha$ is a scaling factor. This formulation generalizes naturally to 3D or higher dimensions depending on the chosen feature set F.

In our experiments, we empirically found that $\alpha =\frac{1}{15}$ provides the best trade-off between robustness and sensitivity, effectively reducing the influence of dispersed points.

Finally, the refined labels are used for classification. Given matrix A with feature vector $f\left(A\right)$ and candidate labels $\mathcal{L}\left(A\right)$, the classification decision is defined as follows.

$$\widehat{l}\left(A\right)=arg\underset{l\in \mathcal{L}\left(A\right)}{min}dist(f\left(A\right),C_l),$$

(8)

where $C_l$ is the centroid of class l, and $dist(·,·)$ is the Euclidean distance in the chosen feature space. By considering all possible labels, this approach avoids false labeling that can occur when only the top-performing format is used as ground truth, ensuring robust and reliable classification.

2.5. Feature Set

In this work, we employ a set of structural features extracted from sparse matrices to guide format prediction and clustering.

Table 1. List of extracted structural features.

Abbreviation	Full Name/Description
rows	Number of rows of the matrix
cols	Number of columns of the matrix
nnz	Number of non-zero elements
ndiags	Number of occupied diagonals
mean_rd	Mean number of non-zeros per row
max_rd	Maximum number of non-zeros per row
min_rd	Minimum number of non-zeros per row
std_rd	Standard deviation of non-zeros per row
mean_cd	Mean number of non-zeros per column
max_cd	Maximum number of non-zeros per column
min_cd	Minimum number of non-zeros per column
std_cd	Standard deviation of non-zeros per column
ER_RD	Row efficiency ratio
ER_CD	Column efficiency ratio
row_bounce	Average difference of non-zeros between adjacent rows
col_bounce	Average difference of non-zeros between adjacent columns
density	Matrix density ($nnz/(rows\times cols)$)
row_var	Variance of normalized row non-zeros
max_mu	Difference between maximum and mean row non-zeros

summarizes the abbreviations and full names of these features.

These features capture both global characteristics (such as matrix dimensions, density, and diagonal structure) and local statistical properties (such as row/column distributions and variations). By integrating them into the learning and clustering framework, we provide a richer representation of sparsity patterns, thereby enabling more accurate predictions of the optimal format across CPU and GPU platforms.

2.6. 3D Feature Space Before/After Dynamic Clustering (CPU)

We leverage the feature set introduced earlier and the labels refined by dynamic labeling and clustering to build a 3D predictive view. Concretely, we project matrices into a three-dimensional space using the top-correlated features ${log}_{10}\left(n\right)$, ${log}_{10}\left(\mathrm{nnz}\right)$, and ${log}_{10}\left(\mathrm{cols}\right)$, and visualize the class separation before and after clustering. As shown in Figure 5, (a) and (c) depict the raw distributions prior to clustering, while panels (b) and (d) show the results after applying the dynamic clustering procedure. The top row corresponds to an AMD EPYC 7713 CPU, and the bottom row corresponds to an Intel Xeon Gold 6230 CPU. Before clustering, samples of different formats overlap substantially, leading to fuzzy boundaries. After clustering, points aggregate around stable fixed-label centroids, the inter-class margins become clearer, and label noise is reduced, resulting in more robust and discriminative separation in the $({log}_{10}\left(n\right),{log}_{10}\left(\mathrm{nnz}\right),{log}_{10}\left(\mathrm{cols}\right))$ space.

2.7. GPU-Side 3D Distributions

On the GPU platform, dynamic labeling and clustering exhibit a distribution pattern that is distinctly different from the CPU side, as shown in Figure 6. Initially, formats such as COO, BSR, and DIA are almost entirely eliminated, leaving only CSR and CSC. Among them, CSC dominates with approximately ∼80% of the samples. After clustering (right), a small portion of CSR samples in the upper-right region of the 3D feature space (i.e., ${log}_{10}\left(\mathrm{nnz}\right),{log}_{10}\left(\mathrm{rows}\right),{log}_{10}\left(\mathrm{cols}\right))$ merge into the CSC cluster, resulting in clearer class boundaries.

The underlying causes are strongly related to the implementation characteristics of each format on GPUs: (i) COO, while storage-simple, requires atomic additions during SpMV, leading to high thread conflicts and synchronization overhead, which makes it significantly slower than CSR/CSC for large matrices; (ii) BSR/DIA can be efficient for highly structured matrices, but in general datasets their conversion costs are large and optimized GPU kernels are limited, so they rarely emerge as the best choice; (iii) CSR/CSC benefit from mature GPU implementations and better exploit memory bandwidth and thread parallelism. Among them, CSC tends to perform better for large, uniformly distributed sparsity patterns, which explains its dominance.

This result highlights the strong dependency of format selection on hardware characteristics and shows that optimal formats on GPUs differ significantly from those on CPUs.

3. Results

Scope of this section. We first summarize the experimental platforms and datasets, and then evaluate the (i) hardware-selection accuracy of Model S, (ii) format-prediction accuracy and robustness under dynamic multi-labeling with clustering, and (iii) end-to-end performance via slowdown/speedup ratios.

3.1. Experimental Platform

All experiments were conducted on two servers equipped with heterogeneous CPU processors but with identical GPU resources to ensure fair comparison. Each server was configured with NVIDIA GPUs, including GeForce RTX 3090 (24 GB), RTX 3060 (12 GB), and professional-grade A40 (48 GB). For reproducibility, all SpMV kernels were executed on the same GPU architecture to eliminate hardware bias.

Table 2. Hardware configuration of experimental platforms.

Component	Server 1	Server 2
CPU	AMD EPYC 7713 (64 cores, 1.7 GHz)	Intel Xeon Gold 6230 (20 cores, 2.1 GHz)
Sockets × Cores/Socket × Threads/Core	$2\times 64\times 2$ (=256 logical)	$2\times 20\times 2$ (=80 logical)
CPU threads used (OpenMP)	256	80
GPU	NVIDIA A40 (48 GB) × 2	NVIDIA A40 (48 GB) × 2
GPU SMs (per A40)	84	84
GPU peak memory bandwidth (per A40)	696 GB/s	696 GB/s
CUDA Driver	550.144.03	550.144.03
CUDA Version	12.4	12.4
Memory	512 GB DDR4	512 GB DDR4
OS	Ubuntu 22.04 LTS	Ubuntu 22.04 LTS

summarizes the hardware setup. Our experimental dataset consists of 1786 sparse matrices obtained from the University of Florida Sparse Matrix Collection (UFSMC, also known as TAMU) [20].

The software stack includes Python 3.8/3.9, CuPy (v12.3.0), NumPy (v1.24.4), and SciPy (v1.11.3) under conda-managed environments. All experiments were conducted using CUDA toolkit 12.4 and NVIDIA driver 550.144.03 (NVIDIA Corporation, Santa Clara, CA, USA) to ensure reproducibility.

We summarize the corpus characteristics over 1786 matrices in

Table 3. Dataset profile over 1786 matrices: summary statistics for rows (n), columns (m), and nonzeros (nnz).

	Min	P25	Median	P75	Max	Mean
rows (n)	32	1007.5	4004	16,141.75	683,446	18,419.82
cols (m)	32	1182	4938	17,670.5	683,446	21,248.38
nnz	48	10,440.75	46,366.5	207,284.5	28,715,634	306,433.42

Note: P25/P75 denote the 25th/75th percentiles (first/third quartiles).

, reporting the distributions of rows (n), columns (m), and nonzeros (nnz).

3.2. Model S Evaluation

To evaluate the effectiveness of Model S in hardware selection, we first verify its ability to distinguish whether a given sparse matrix should be executed on the CPU or GPU. Here, the decision outcome is represented by Rule, where Rule = CPU(0) indicates that the matrix is classified as better suited for execution on the CPU, while Rule = GPU(1) indicates that the GPU is predicted to be the more efficient platform.

Figure 8 illustrates the evaluation results. The left part shows the distribution of performance margins defined as $margin=T_{CPU}-T_{GPU}$, grouped by Rule choice. A margin near zero indicates that both hardware platforms yield similar performance, while negative or positive values correspond to performance advantages on GPUs or CPUs, respectively. The results demonstrate that when Model S selects GPUs (Rule = 1), the margin is generally positive, meaning that the GPU offers a significant speedup, whereas when CPU is chosen (Rule = 0), the performance gap remains very small, confirming that Model S makes stable and reliable hardware decisions. It is also noteworthy that the margin distribution on the GPU side exhibits a wider variance, indicating that execution time differences are more sensitive to changes in matrix scale and structural features when running on GPUs.

The right part further illustrates the decision boundary by showing the distribution of Rule = 0 and Rule = 1 over the feature planes ${log}_{10}\left(nnz\right)$ and ${log}_{10}\left(n\right)$. The figure reveals a clear separation: matrices with smaller sizes and lower nnz tend to favor CPU execution, while larger-scale matrices shift toward GPUs, demonstrating the effectiveness of the selected features in guiding the hardware decisions.

In this experiment, the hardware selection of Model S is guided by structural features extracted from the input matrices, including the number of rows ($rows$), the number of columns ($cols$), the number of nonzeros ($nnz$), density, row- and column-wise statistics (mean, max, min, and standard deviation), and derived metrics such as effective ratio ($ER\_RD$, $ER\_CD$) and row/column balance. Among these, $nnz$ and $rows$ were found to be the most influential, consistent with the observation that smaller, sparser matrices are better handled by CPUs, while larger, denser matrices benefit more from GPU acceleration.

Hardware-Selection Accuracy (Definition)

For completeness, we formalize the metric used for hardware decisions as follows:

$${\mathrm{Acc}}_{\mathrm{S}}={\displaystyle \frac{1}{N}}\sum _{i=1}^N\mathbb{I}\left[\widehat{r}\left(A_i\right)=r^{*}\left(A_i\right)\right],$$

(9)

Here, $r^{*}\left(A_i\right)\in \{\mathrm{CPU},\mathrm{GPU}\}$ denotes the device that minimizes the end-to-end latency $T\left(A_i\right)$ in Equation (1), and $\widehat{r}\left(A_i\right)$ is the prediction.

These features form the input space for Model S to predict the appropriate hardware platform. Overall, Model S achieves a prediction accuracy of 98.2%, demonstrating its reliability in guiding hardware selection.

3.3. Evaluation Metrics

We report format-prediction accuracy (single-label and dynamic-label), robustness across repeated rounds, and speedup/slowdown ratios. The definition of hardware-selection accuracy is given at the end of Section 3.2 (Equation (9)). Let ${\left\{A_i\right\}}_{i=1}^N$ denote the test matrices.

3.3.1. Format-Prediction Accuracy

In the single-label setting, the ground truth for $A_i$ is ${\ell }^{*}\left(A_i\right)=arg{min}_{s\in S}{T}_{\mathrm{total}}(s;A_i)$. In the dynamic-label setting, the ground truth is a near-optimal set $L\left(A_i\right)$ (Section 2.3). Denote $\widehat{\ell }\left(A_i\right)$ as the predicted format.

$${\mathrm{Acc}}_{\mathrm{P}}^{\mathrm{single}}={\displaystyle \frac{1}{N}}\sum _{i=1}^N\mathbb{I}\left[\widehat{\ell }\left(A_i\right)={\ell }^{*}\left(A_i\right)\right],$$

(10a)

$${\mathrm{Acc}}_{\mathrm{P}}^{\mathrm{dyn}}={\displaystyle \frac{1}{N}}\sum _{i=1}^N\mathbb{I}\left[\widehat{\ell }\left(A_i\right)\in L\left(A_i\right)\right].$$

(10b)

3.3.2. Robustness Across Rounds

We train on the clustered labels from the initial round and evaluate the same trained model on R additional rounds (new matrices/perturbed datasets). Robustness is summarized by the mean accuracy across rounds:

$$\overline{\mathrm{Acc}}={\displaystyle \frac{1}{R}}\sum _{r=1}^R{\mathrm{Acc}}^{\left(r\right)}.$$

(11)

3.3.3. Speedup/Slowdown Ratios

For CPU–GPU contrast on the same matrix, the equation is as follows:

$${\mathrm{Slowdown}}_{\mathrm{small}}={\displaystyle \frac{T_{\mathrm{GPU}}}{T_{\mathrm{CPU}}}}(>1\mathrm{means}\mathrm{GPU}\mathrm{is}\mathrm{slower}\mathrm{on}\mathrm{small}\mathrm{matrices}),$$

(12a)

$${\mathrm{Speedup}}_{\mathrm{large}}={\displaystyle \frac{T_{\mathrm{CPU}}}{T_{\mathrm{GPU}}}}(>1\mathrm{means}\mathrm{GPU}\mathrm{is}\mathrm{faster}\mathrm{on}\mathrm{large}\mathrm{matrices}).$$

(12b)

Accordingly, values like $4.56\times$, $20\times$, and $2.3\times$ in the “Small Matrices” column are instances of ${\mathrm{Slowdown}}_{\mathrm{small}}$, whereas the “Large Matrices” column reports ${\mathrm{Speedup}}_{\mathrm{large}}$.

3.3.4. Acceleration over CSR

Acceleration over the default CSR baseline is defined as follows.

$${\mathrm{Speedup}}_{\mathrm{CSR}}={\displaystyle \frac{T_{\mathrm{CSR}}}{T_{\mathrm{pred}}}},$$

(13)

where $T_{\mathrm{pred}}$ denotes the end-to-end latency after hardware selection and format prediction.

3.4. Prediction Accuracy Analysis

Table 4. Classification accuracy (%) of different models before and after dynamic labeling and clustering. Accuracy is defined in Equation (9).

Method	CPU1			CPU2			GPU
	DecisionTree	2_Layer_ANN	XGBoost	DecisionTree	2_Layer_ANN	XGBoost	DecisionTree	2_Layer_ANN	XGBoost
traditional	79.47	83.40	86.92	76.45	80.61	81.34	92.34	93.75	91.71
dynamic	88.20	90.06	89.71	84.27	86.83	85.76	98.27	97.92	98.95

reports the classification accuracy of three models (DecisionTree, 2-layer ANN, and XGBoost) across two CPU platforms and one GPU platform, evaluated under both traditional single-label and proposed dynamic labeling schemes.

As shown in Table 4, dynamic labeling and clustering significantly improve prediction accuracy across all platforms and models. On CPU1, the accuracy of DecisionTree increased from 79.47% to 88.20%, while the 2-layer ANN improved from 83.40% to 90.06%. Similarly, CPU2 saw an average improvement of 6–8% across all models. On GPUs, the gains were even more remarkable: the DecisionTree accuracy rose from 92.34% to 98.27%, and XGBoost achieved the highest accuracy of 98.95%. Overall, dynamic labeling combined with clustering enhanced model robustness and reduced label noise, with accuracy improvements ranging from 5% to 10%.

These quantitative improvements are consistent with the visual evidence. As shown in Figure 5, dynamic labeling and clustering lead to clearer boundaries in the 3D feature space $({log}_{10}\left(n\right),{log}_{10}\left(\mathrm{nnz}\right),{log}_{10}\left(\mathrm{cols}\right))$. The per-format decomposition of Figure 5b, presented in Figure 9, further illustrates that COO, CSR, CSC, and BSR formats occupy distinct and compact regions. This clear separation explains why CPU-side classification accuracy is significantly improved, as also reflected in Table 4.

As shown in

Table 5. Accuracy comparison between 2D and 3D clustering under different numbers of features. Accuracy is defined in Equations (10a) and (10b).

Platform	Dim	2f	3f	4f	5f	6f	7f	8f
CPU1	2D	85.11%	86.14%	85.42%	83.27%	85.75%	86.04%	84.82%
	3D	88.75%	90.06%	90.32%	89.75%	88.59%	89.34%	89.50%
CPU2	2D	85.67%	86.25%	86.75%	85.90%	85.21%	84.95%	85.69%
	3D	85.39%	86.83%	86.02%	86.98%	84.95%	86.07%	84.94%
GPU	2D	97.71%	97.80%	97.54%	98.08%	98.19%	97.82%	97.93%
	3D	97.64%	97.92%	98.03%	98.26%	97.88%	97.93%	98.15%

, incorporating three-dimensional (3D) feature spaces significantly improves classification accuracy for CPU platforms compared to traditional two-dimensional (2D) clustering. For CPU1, the accuracy increases from 86.14% (2D, 3f) to 90.06% (3D, 3f), while for CPU2 the improvement is from 86.25% (2D, 3f) to 86.83% (3D, 3f). This demonstrates that 3D clustering is more effective in capturing the intrinsic structure of feature distributions, leading to more stable label assignments.

In contrast, GPU classification results show relatively small differences between 2D and 3D clustering, with accuracies already exceeding 97% across all feature combinations. This suggests that GPU label distributions are more concentrated, and additional feature dimensions provide only marginal improvements.

Overall, these results indicate that the benefit of 3D clustering is more pronounced on CPU platforms, where execution time variability and feature heterogeneity are greater. On GPU platforms, the already stable label distribution limits the additional gains from higher-dimensional feature spaces. Across additional rounds, the mean accuracy remains stable, indicating reproducibility under dynamic labeling; we therefore report the multi-round mean as the robustness measure.

3.5. Acceleration Effect Analysis

Before reporting ratios, Figure 10 visualizes how feature-extraction time ($T_f$) relates to conversion/build time ($T_c$, proxy; left) and to execution time ($T_e$, minimal SpMV kernel time; right) over 1786 matrices. $T_f$ is strongly correlated with both $T_c$ ($r=0.875$) and $T_e$ ($r=0.969$), reflecting their common dependence on matrix size/density. For small matrices, $T_f$ is often of the same order as $T_c$ and thus non-negligible in the end-to-end latency; as matrices grow larger, $T_e$ increases more rapidly than $T_f$, and $T_f$ becomes a small fraction of T. With this end-to-end accounting in place, we next report slowdown/speedup relative to CSR and contrast our cost-aware pipeline with traditional kernel-time-only selectors, to clarify improvements in end-to-end latency.

While the previous section focused on prediction accuracy, here we analyze the acceleration effect of hardware selection across small and large sparse matrices.

Table 6. Performance comparison of GPUs versus CPUs on small and large matrices. Values larger than $1\times$ indicate slowdown, while speedup is reported for large matrices. Ratios are defined in Equations (12a) and (12b).

Platform	Small Matrices (GPU Slowdown)		Large Matrices (GPU Speedup)
	Average	Maximum	Average	Maximum
Server 1 (AMD EPYC 7713 + A40)	$4.56\times$	$20\times$	$1.6\times$	$6.84\times$
Server 2 (Intel Xeon Gold 6230 + A40)	$2.3\times$	$8.2\times$	$1.98\times$	$20.1\times$

reports the performance overhead or speedup of GPU execution compared with CPUs across two platforms. For small matrices, GPU execution is inefficient due to memory transfer overheads: on Server 1 (AMD EPYC 7713, Advanced Micro Devices, Inc., Santa Clara, CA, USA + A40, NVIDIA Corporation, Santa Clara, CA, USA), the GPU is on average $4.56\times$ slower than the CPU, with the worst case reaching $20\times$; on Server 2 (Intel Xeon Gold 6230, Intel Corporation, Santa Clara, CA, USA + A40, NVIDIA Corporation, Santa Clara, CA, USA), the average slowdown is $2.3\times$, and the worst case reaches $8.2\times$.

In contrast, for large matrices, GPU execution significantly outperforms CPUs. On Server 1, the GPU achieves an average speedup of $1.6\times$ over the CPU, with the maximum reaching $6.84\times$. On Server 2, the GPU is even more advantageous, achieving an average speedup of $1.98\times$, with the maximum reaching $20.1\times$.

These results demonstrate that the benefit of GPU computation strongly depends on matrix scale and characteristics. Hence, accurate hardware decision-making through Model S is crucial to avoid performance degradation on small matrices and to fully leverage GPU acceleration on large ones.

3.6. Acceleration over Baseline CSR

Beyond analyzing the raw CPU–GPU trade-offs, we further compare the acceleration achieved by Model S relative to the default CSR format.

Table 7. Acceleration over baseline CSR after Model S format prediction. CSR acceleration is defined in Equation (13).

Platform	CPU		GPU
	Average	Maximum	Average	Maximum
Server 1 (AMD EPYC 7713 + A40)	$1.10\times$	$2.10\times$	$1.30\times$	$1.60\times$
Server 2 (Intel Xeon Gold 6230 + A40)	$1.16\times$	$3.15\times$	$1.24\times$	$1.94\times$

summarizes the average and maximum speedups on both CPUs and GPUs across the two experimental platforms.

On CPUs, Server 1 achieves an average speedup of $1.10\times$ and a maximum of $2.10\times$, while Server 2 yields a higher average of $1.16\times$ with a maximum of $3.15\times$. On GPUs, Server 1 obtains an average $1.30\times$ speedup, peaking at $1.60\times$, whereas Server 2 achieves an average of $1.24\times$ with a maximum of $1.94\times$.

These results indicate that even relative to the widely adopted CSR baseline, our hardware-aware format prediction framework delivers consistent and measurable improvements, validating the effectiveness of Model S.

To further evaluate the robustness of our method, we trained the model using the clustered labels from the initial round and tested it on five additional rounds of experiments.

Table 8. Robustness test on CPU1 across five rounds. Accuracy and speedup are compared among traditional, 2D, and 3D clustering labels.

Round	Accuracy (%)			Speedup
	Traditional	2D	3D	Traditional	2D	3D
1	74.33	82.41	81.15	0.81×	1.17×	1.15×
2	73.29	80.34	82.10	0.76×	1.10×	1.10×
3	70.82	81.63	84.82	0.92×	1.02×	1.26×
4	69.16	76.27	83.87	1.03×	0.97×	1.20×
5	72.67	77.47	84.80	0.95×	1.00×	1.23×

,

Table 9. Robustness test on CPU2 across five rounds. Accuracy and speedup are compared among traditional, 2D, and 3D clustering labels.

Round	Accuracy (%)			Speedup
	Traditional	2D	3D	Traditional	2D	3D
1	71.20	79.92	82.33	1.00×	1.13×	1.16×
2	67.52	80.43	80.06	1.02×	1.02×	1.10×
3	74.36	83.78	84.65	0.95×	1.18×	1.16×
4	70.05	78.60	81.17	0.82×	1.07×	1.00×
5	66.39	81.97	85.31	0.79×	1.01×	1.20×

,

Table 10. Robustness test on GPU across five rounds. Accuracy and speedup are compared among traditional, 2D, and 3D clustering labels.

Round	Accuracy (%)			Speedup
	Traditional	2D	3D	Traditional	2D	3D
1	91.20	95.30	96.10	1.03×	1.05×	1.25×
2	89.76	96.75	97.24	1.12×	1.20×	1.16×
3	74.36	94.87	97.52	0.92×	1.17×	1.14×
4	70.05	95.81	95.43	0.83×	1.27×	1.22×
5	66.39	97.19	96.70	1.05×	1.21×	1.31×

summarize the results for CPU1, CPU2, and GPU platforms, respectively. Across all platforms, dynamic labeling with clustering (2D and 3D) consistently outperforms the traditional approach. On CPU1, 3D clustering improves accuracy by up to 14% (round 4) and achieves speedups as high as 1.26×. CPU2 exhibits a similar trend, though with slightly smaller gains. On the GPU platform, the effect is most pronounced: accuracy consistently exceeds 95%, and 3D clustering delivers speedups up to 1.31×. These results demonstrate the robustness of the proposed method—models trained with initial clustered labels generalize well to new rounds of data—and, taken together, they substantiate our accuracy–robustness–acceleration objectives under end-to-end accounting; Section 4 discusses practical implications, limitations, and future directions.

4. Discussion

This study revisits SpMV format prediction on heterogeneous (CPU–GPU) platforms through four practical challenges that often cause instability or misleading conclusions: (C1) label instability across hardware/workloads, where the “best” format may fluctuate across repeated runs [12,13]; (C2) missing end-to-end cost accounting beyond kernel time (i.e., overlooking feature extraction and format conversion costs); (C3) conflating small- vs. large-matrix regimes in CPU–GPU comparisons; and (C4) the lack of a unified CPU–GPU pipeline that yields consistent device/format decisions across devices.

Recent work strengthens format selection and autotuning but typically addresses only subsets of the above challenges. GPU-centric predictors or CPU-only heuristics emphasize kernel execution while under-accounting conversion/feature costs [9,14,16,17,21,22,23]. Within the MDPI literature, Chen et al. propose an adaptive hybrid format strategy for multi-core SIMD CPUs that learns when to combine/choose formats [18], and Sanderson & Curtin show automatic switching among sparse formats within Armadillo, highlighting engineering trade-offs of conversion and storage [24]. The Overhead-Conscious Method (OCM) explicitly models format-conversion overhead and shows that keeping CSR can avoid negative returns at low iteration counts [25,26]; however, OCM relies on single-label procedures and does not include the feature-extraction cost, which may dominate for small matrices. By contrast, our pipeline jointly reasons about hardware selection (CPU vs. GPU), low-cost features, dynamic multi-labeling with clustering, format conversion, and execution in a unified heterogeneous setting.

Section 3 provides end-to-end evidence for these choices. The hardware selector (Model S) attains 98.2% hardware-selection accuracy (Section 3.2). Dynamic multi-labeling with clustering improves format-prediction accuracy by 5–10% over the single-label baseline (Section 3.3). Robustness is evaluated as multi-round reproducibility: the same trained model is tested on R additional rounds and we report the mean accuracy across rounds (Section 3.4). To avoid regime conflation, we report ${\mathrm{Slowdown}}_{\mathrm{small}}=T_{\mathrm{GPU}}/T_{\mathrm{CPU}}$ for small matrices and ${\mathrm{Speedup}}_{\mathrm{large}}=T_{\mathrm{CPU}}/T_{\mathrm{GPU}}$ for large matrices, and we quantify acceleration over CSR as ${\mathrm{Speedup}}_{\mathrm{CSR}}$ (Section 3.5).

Aligned with the four challenges above—and relative to OCM [25,26] and CPU-only/GPU-only predictors [9]—this work (a) optimizes end-to-end latency with explicit feature and conversion cost accounting, (b) stabilizes labels via dynamic multi-labeling + clustering (capturing near-optimal sets), and (c) separates small vs. large regimes in CPU–GPU comparisons; we further develop a unified heterogeneous pipeline with regime-aware reporting.

Explicitly modeling label uncertainty improves reproducibility and reduces overfitting to noisy measurements. Higher-dimensional feature clustering uncovers structure–hardware affinities: large matrices with high nonzeros-per-row density benefit more from GPU acceleration, whereas small matrices tend to favor CPUs due to transfer/conversion overheads. These observations are consistent with prior CPU/GPU-focused studies and the MDPI literature on adaptive/hybrid formats [18,24].

We currently evaluate four storage formats and do not cover distributed or multi-GPU settings. ELL width is chosen heuristically, and the near-optimal threshold may affect the dynamic-label set on edge cases. The UF/SMC corpus composition (sizes, sparsity ranges, application tags) may also introduce bias in format prevalence.

The pipeline can be integrated into heterogeneous HPC/ML/graph-analytics stacks to automate device/format choice with cost awareness. Promising directions include runtime adaptation with online re-labeling, expanding the format set (e.g., DIA/SELL-C/hybrid variants), learned cost models for features/conversion, and evaluation in distributed/multi-GPU environments; unified intermediate representations such as UniSparse [27] may further ease cross-format transformations.

Study Limitations

While the proposed pipeline achieves reproducible accuracy and end-to-end gains, several limitations should be noted.

(1)

Threshold/radius selection under distribution shift. We do not claim a universally optimal threshold or local-radius setting for dynamic/mixed labeling and clustering. Different matrix corpora can induce different label distributions, and clustering entails randomness (e.g., initialization, sampling). A principled, data-adaptive choice would require a separate methodological study and a large-scale sensitivity analysis, which are beyond the scope of this work.

(2)

Stochasticity of clustering and training. Label construction and learning involve stochastic procedures that can introduce run-to-run variance. We control randomness (fixed seeds, repeated trials) and report robust statistics, but a full variance decomposition across seeds/initializations is not included.

(3)

Scope and coverage. This study currently evaluates four storage formats and does not cover distributed or multi-GPU settings. Although feature-extraction and format-conversion costs are explicitly considered in our end-to-end evaluation, learning-based cost models for these components have not yet been integrated into the unified framework. This is a natural extension of the pipeline for future work.

5. Conclusions

In this paper, we take an end-to-end, cost-aware view of SpMV format prediction on heterogeneous (CPU–GPU) platforms. We unify hardware selection, low-cost feature extraction, dynamic multi-labeling with clustering, format conversion, and execution into a single pipeline, and evaluate it with regime-aware metrics.

Our contributions are threefold: (i) a dynamic multi-labeling strategy with clustering that stabilizes labels and improves format-prediction accuracy over a single-label baseline; (ii) a unified, cost-aware CPU–GPU pipeline that explicitly accounts for feature- and conversion-costs in addition to kernel time; and (iii) a regime-aware reporting protocol that separates small- and large-matrix cases (GPU slowdown for small; GPU speedup for large) and quantifies acceleration over CSR.

This design has a practical impact for heterogeneous HPC/ML/graph workloads, providing reproducible, cost-aware automation of device/format choices and clarifying when CPUs or GPUs are preferable and which formats dominate under realistic overheads.

Future work will extend the pipeline along several directions: runtime adaptation with online re-labeling; broader format sets (e.g., DIA/SELL-C/hybrid variants); learned cost models for features and conversion; and evaluation in distributed/multi-GPU settings, potentially aided by unified intermediate representations such as UniSparse.