From Kernel Matrices to Kernel Functions: An Eigenfunction-Based Approach

Abstract

Kernel-combination procedures used in classification often return only a combined kernel matrix on the training sample, rather than a kernel function that can be evaluated consistently at new points. This limitation is especially important for supervised or label-aware combinations, whose entries may depend on training labels and therefore have no immediate out-of-sample meaning. We study the problem of constructing an inductive, finite-rank kernel extension from such empirical matrices. The proposed framework makes the non-uniqueness of this extension explicit: it is determined by empirical coordinates, a positive-semidefinite coefficient matrix, and a continuation model for the coordinates. Experiments on vector, tabular, and relational classification problems give a deliberately diagnostic picture. Smooth direct combinations are stable: on Synthetic, the direct mean gives error $0.0793\pm 0.0227$, essentially matching the best individual RBF kernel ($0.0809\pm 0.0231$), and on Telco it remains close to the best individual polynomial kernel ($0.2061\pm 0.0154$ versus $0.2045\pm 0.0154$). In the controlled Synthetic oracle diagnostic, reconstructing a smooth sum/mean gives relative Frobenius error $4.13\times {10}^{-6}\pm 9.41\times {10}^{-6}$ and functional MSE at numerical scale. By contrast, abrupt label-aware matrix-only rules are less robust: the Synthetic percentile_inout_auto rule has error $0.1404\pm 0.1198$, Telco matrix-only supervised rules are around $0.307$–$0.326$ error, and the Chickenpieces pickout_auto rule fails under strict out-of-sample reconstruction ($0.3545\pm 0.2666$ error), whereas direct relational combinations match the best individual relational kernel within ${10}^{-3}$. Overall, the empirical evidence supports the method as a bridge from finite matrix-level information fusion to deployable kernels, while also identifying abrupt label-aware geometries as the main limitation for stable generalization.

1. Introduction

Kernel methods provide an effective and natural way to bring distances and similarities directly into learning problems. A kernel matrix can encode how close, comparable, or compatible two observations are, even when the objects are not conveniently described by ordinary Euclidean coordinates. This makes kernels particularly useful for classification, regression, and similarity-based learning: geometric or relational information is transformed into an inner-product representation in a reproducing kernel Hilbert space (RKHS), where linear operations become available, nonlinear decision rules can be handled through linear algorithms, and standard classifiers can work from similarities rather than only from raw coordinates [1,2,3]. Their practical success depends heavily on the choice of the kernel, and in many applications a single kernel is not rich enough to capture all relevant aspects of the problem. For that reason, combinations of kernels or similarities have been extensively studied, both as a way of fusing heterogeneous information and as a mechanism for improving discrimination in supervised tasks  [4,5,6].

A central difficulty appears when the combination rule produces only a matrix on a finite sample. This happens, for instance, in several label-aware or similarity-driven fusion rules [6,7]: the practitioner obtains a combined empirical matrix $K_S^{*}$ on the training set $S={\left\{x_i\right\}}_{i=1}^n$, but not a genuine kernel function $K^{*}(x,z)$ that can be evaluated at new points. Once the learning algorithm leaves the training sample, the combined object is no longer directly available. In consequence, the resulting model cannot be deployed consistently unless one reconstructs an out-of-sample extension of the combined kernel.

Recent work has confirmed that this passage from finite kernel or proximity information to an out-of-sample evaluator remains an active problem. Fanuel et al. [8] constructed data-dependent positive-semidefinite kernels with explicit out-of-sample formulas for positive-semidefinite embeddings, while Münch et al. [9] studied spectral fusion of heterogeneous, possibly indefinite, proximity data. These approaches are complementary to the present paper: we start from a combined matrix already produced by a supervised or multi-source rule and seek a functional representation of that final matrix, up to the selected spectral rank, without using labels for new points.

To make the background and scope more explicit, we position the problem at the intersection of three related literatures. Multiple-kernel learning and kernel-alignment methods study how to combine kernels for supervised learning [4,5,10,11]. Similarity and proximity learning study how finite pairwise matrices, including non-Euclidean or indefinite objects, can be used or rectified for learning [7,12]. Kernel approximation and data-dependent RKHS bases study how finite spectral or interpolation information can be evaluated stably away from the sample [8,9,13,14]. The present paper uses the first two lines of work as sources of finite combined matrices and the third as the mechanism for obtaining deployable out-of-sample evaluators. This positioning clarifies why the empirical question is not only whether a combination improves classification, but also whether the combined matrix can be converted into a stable kernel on unseen observations.

This article addresses precisely that matrix-to-kernel step. Starting from a positive-semidefinite combined matrix, we construct one out-of-sample extension through a finite feature representation and data-dependent bases in the empirical RKHS [13,14,15]. Such an extension is not unique: a finite matrix admits infinitely many positive-semidefinite extensions outside the observed sample. The contribution of the paper is therefore not to recover “the” true kernel, but to define a controlled inductive extension specified by empirical coordinates, a PSD coefficient matrix, and a regression model for those coordinates. The resulting kernel can then be evaluated between training and test points without access to test labels.

Figure 1 summarizes the conceptual pipeline studied in this paper. The figure emphasizes that the objective is not merely to train a classifier on a fixed matrix, but to transform sample-based kernel or dissimilarity constructions into evaluable functions.

Contributions.

The paper makes four concrete contributions.

1.

It formulates the passage from a combined sample matrix $K_S^{*}$ to an out-of-sample kernel $K^{*}(x,z)$ as a reconstruction problem with three target properties: interpolation on the sample, positive definiteness, and practical out-of-sample evaluation.

2.

It proposes a reconstruction strategy based on data-dependent bases in the empirical RKHS, with Newton and SVD bases as the two practically relevant choices.

3.

It provides a reproducible experimental protocol that separates base kernels, matrix combination, kernel reconstruction, and downstream classification.

4.

It validates the method on completed strict runs for Synthetic, Breast Cancer, Ionosphere, Telco Customer Churn, and Chickenpieces, covering both ordinary vector data, heterogeneous tabular data, and relational dissimilarity data.

Illustrative Example: From a Sample-Based Construction to an Out-of-Sample Function

Before turning to the full experimental protocol, it is useful to isolate the main objective on a simple one-dimensional example. The central issue is not primarily to propose a classifier competing with highly specialized alternatives, but rather to transform a construction originally defined only on a finite sample into a function that can be evaluated at previously unseen points.

Suppose that we observe sample points $x_1,\dots ,x_n\in \mathcal{X}$ and a kernel matrix $K_S$ defined on these points. Let ${\{{\lambda }_j,{\phi }_j\}}_{j\ge 1}$ denote the eigenvalue–eigenfunction pairs associated with the corresponding integral operator, or, in practice, the finite-sample approximation derived from the kernel matrix. A truncated representation with $m_0$ terms has the form

$$f_{m_0}\left(x\right)={\displaystyle \sum _{j=1}^{m_0}}{a}_j{\phi }_j\left(x\right).$$

Once the coefficients $a_j$ have been determined from the sample, the expression above defines an object that can be evaluated not only at the observed points, but also at a new point $x_{\mathrm{new}}$, provided that the basis functions admit an out-of-sample extension.

This is precisely the situation considered throughout the paper. In many applications, one starts from kernels or dissimilarity-based constructions that are explicitly available only on the training sample. The contribution of the proposed framework is to show how such constructions, including ad hoc kernel combinations, can be turned into evaluable functions outside the sample by means of finite basis representations. In this sense, the classification experiments should be interpreted mainly as a validation that the resulting extensions remain informative and competitive, rather than as an attempt to outperform specialized classification methods.

Figure 2 fixes the idea visually. A smooth target function is observed only at finitely many input locations. From a kernel-based basis constructed on the sample, one obtains a truncated approximation

$${\widehat{f}}_{m_0}\left(x\right)={\displaystyle \sum _{j=1}^{m_0}}{\widehat{a}}_j{\widehat{\phi }}_j\left(x\right),$$

which interpolates or approximates the observed data and can then be evaluated at new points. The role of the truncation parameter $m_0$ is to control the balance between fidelity to the sample-based construction and regularity of the resulting extension. The same principle underlies the methodology developed below for combined kernels.

The main message of the paper is deliberately cautious. The proposed reconstruction is especially reliable for sum-type and modular combinations, where the out-of-sample kernel remains close to the oracle combination and classification performance is preserved. In contrast, more abrupt label-driven combinations such as pick-out are harder to reconstruct faithfully and do not consistently improve classification once they are extended outside the sample.

The rest of the paper is organized as follows. Section 2 introduces the RKHS framework and formulates the reconstruction problem. Section 3 presents the proposed method. Section 4 describes the experimental protocol. Section 5 reports the numerical results, and Section 6 discusses what they imply for the use of combined kernels in practice.

2. Background and Problem Formulation

2.1. Positive-Definite Kernels, RKHSs, and Integral Operators

Let $K:\mathcal{X}\times \mathcal{X}\to \mathbb{R}$ be a continuous, symmetric, positive-definite kernel on a compact domain $\mathcal{X}$. By the classical Moore–Aronszajn construction, K induces a unique reproducing kernel Hilbert space (RKHS) ${\mathcal{H}}_K$ of real-valued functions on $\mathcal{X}$ such that point evaluation is continuous and the reproducing property

$$f\left(x\right)={\langle f,K(·,x)\rangle }_{{\mathcal{H}}_K},f\in {\mathcal{H}}_K,$$

holds for every $x\in \mathcal{X}$ [1]. The empirical subspace associated with a sample $S=\{x_1,\dots ,x_n\}$ is

$${\mathcal{H}}_K\left(S\right)=\mathrm{span}\{K(·,x_i):1\le i\le n\},$$

which is the finite-dimensional space where all computations in the paper take place.

If $\nu$ is a Borel probability measure on $\mathcal{X}$, the kernel defines the integral operator

$$\left(L_{K}f\right)\left(x\right)={\int }_{\mathcal{X}}K(x,t)f\left(t\right)d\nu \left(t\right).$$

(1)

Under our standing assumptions, $L_K$ is self-adjoint, positive, and compact. This operator-theoretic viewpoint is crucial because it links the analytic kernel K to a countable eigensystem and therefore provides the language in which sample matrices can be interpreted as discretizations of a functional object. This operator-theoretic viewpoint is the standard bridge between kernels, RKHSs, and kernel-induced operators in learning theory and in earlier basis-combination work that motivates the present manuscript [16,17,18]. The common message is that kernel reconstruction is best understood spectrally rather than only entrywise.

2.2. Mercer Expansions, Regularization, and Projection onto the RKHS

Mercer’s theorem states that if K is a Mercer kernel on a compact domain and $\nu$ is non-degenerate, then there exist non-negative eigenvalues ${\left\{{\lambda }_j\right\}}_{j\ge 1}$ and orthonormal eigenfunctions ${\left\{{\varphi }_j\right\}}_{j\ge 1}\subset L_{\nu }^2\left(\mathcal{X}\right)$ such that

$$K(x,z)=\sum _{j\ge 1}{\lambda }_j{\varphi }_j\left(x\right){\varphi }_j\left(z\right),$$

(2)

with absolute and uniform convergence on $\mathcal{X}\times \mathcal{X}$ [15,16,17]. In particular, each eigenfunction associated with a positive eigenvalue belongs to ${\mathcal{H}}_K$, and the RKHS norm can be expressed through the Mercer coordinates of the function. This is what turns the eigenfunctions into the natural degrees of freedom of the problem.

Given data ${\left\{(x_i,y_i)\right\}}_{i=1}^n$, learning in the RKHS is often posed through Tikhonov regularization:

$$\underset{f\in {\mathcal{H}}_K}{min}{\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{\left(f\left(x_i\right)-y_i\right)}^2+\gamma {\parallel f\parallel }_{{\mathcal{H}}_K}^2,\gamma >0.$$

(3)

By the Representer Theorem, the solution admits the finite expansion

$$f_{\gamma }\left(x\right)={\displaystyle \sum _{i=1}^n}{\alpha }_{i}K(x_i,x),$$

(4)

and the coefficients solve a linear system in the sample kernel matrix  [19]. In the population picture, the corresponding minimizer is ${(L_K+\gamma I)}^{-1}{L}_K{f}_{\nu }$, where $f_{\nu }$ is the regression function. This regularized projection point of view is important here for two reasons. First, it shows how functions in the RKHS are recovered from sample evaluations. Second, it clarifies that out-of-sample continuation is not an arbitrary interpolation problem, but a regularized projection problem in a space whose geometry is already fixed by the kernel. Classical RKHS regularization theory and earlier basis-combination work emphasize this projection interpretation, which is why we use it as the conceptual entry point to kernel reconstruction [17,18,19].

2.3. Finite-Sample Eigenfunction Approximation and Nyström-Type Ideas

Closed-form eigensystems of $L_K$ are exceptional. In practice one observes only a finite sample and its Gram matrix

$$K_S={\left[K(x_i,x_j)\right]}_{i,j=1}^n.$$

Let

$$K_{S}U=U{\Lambda }^{\mathrm{mat}},$$

(5)

where the columns of U are orthonormal eigenvectors and ${\Lambda }^{\mathrm{mat}}=\mathrm{diag}({\lambda }_1^{\mathrm{mat}},\dots )$ contains the eigenvalues of the matrix $K_S$. If the empirical matrix is viewed as a quadrature approximation of the integral operator, then the corresponding operator eigenvalues are scaled as

$${\widehat{\mu }}_j={\displaystyle \frac{{\lambda }_j^{\mathrm{mat}}}{n}},{\widehat{\varphi }}_j\left(x_i\right)\approx \sqrt{n}{u}_{ij}.$$

With this convention the sample reconstruction is correctly normalized:

$$\sum _j{\widehat{\mu }}_j{\widehat{\varphi }}_j\left(x_i\right){\widehat{\varphi }}_j\left(x_k\right)=\sum _j{\displaystyle \frac{{\lambda }_j^{\mathrm{mat}}}{n}}\left(\sqrt{n}{u}_{ij}\right)\left(\sqrt{n}{u}_{kj}\right)={\left(K_S\right)}_{ik}.$$

Thus the factor $1/n$ belongs to the operator eigenvalue, not to the matrix eigenvalue.

For the computational reconstruction used in the rest of the paper we avoid this normalization ambiguity by working directly with matrix coordinates

$${\widehat{\psi }}_j\left(x_i\right)=u_{ij}.$$

(6)

The finite-rank kernel is then assembled as

$${\widehat{K}}_r(x,z)={\displaystyle \sum _{j=1}^r}{\lambda }_j^{\mathrm{mat}}{\widehat{\psi }}_j\left(x\right){\widehat{\psi }}_j\left(z\right).$$

(7)

At the sample points this gives exactly $U_r{\Lambda }_r^{\mathrm{mat}}{U}_r^{\top }$. The Mercer-normalized notation is still useful for interpretation, but (6) and (7) are the normalization used by the algorithm.

When the original analytic kernel K is known, a classical way to continue operator-normalized eigenfunctions outside the sample is the Nyström formula,

$${\widehat{\varphi }}_j\left(x\right)\approx {\displaystyle \frac{1}{\sqrt{n}{\widehat{\mu }}_j}}{\displaystyle \sum _{i=1}^n}{u}_{ij}K(x,x_i),$$

(8)

or equivalently the same formula written with ${\lambda }_j^{\mathrm{mat}}=n{\widehat{\mu }}_j$. Nyström is computationally appealing and widely used in low-rank kernel methods  [20,21,22,23]. However, it is intrinsically a low-rank approximation strategy and it requires access to the kernel section $K(x,x_i)$ of the kernel being extended. For matrix-only combinations, this is exactly the unavailable object. The present paper therefore extends the coordinate traces (6) through learned data-dependent coordinates rather than assuming that $K^{*}(x,x_i)$ is already known.

The out-of-sample extension of empirical eigentraces has also been revisited recently from different perspectives. Fanuel et al. [8] derived extension formulas that turn finite PSD embeddings into data-dependent positive-semidefinite kernels. Neural and parametric alternatives have also appeared; for example, Deng et al. [24] approximated leading eigenfunctions of kernel integral operators by neural networks. Our approach remains non-parametric and basis-driven: the coordinate traces of the combined matrix are extended through stable empirical bases rather than through a newly learned embedding or neural parametrization.

For the present paper, this is precisely the point where alternate RKHS bases become useful: instead of extending eigenvectors only through the raw kernel sections, we first move them to a numerically better coordinate system.

2.4. Interpolation Viewpoint and Why Alternate Bases Are Needed

The need for alternate bases can be understood through a classical interpolation problem [14,25,26,27,28]. Given scattered data ${\left\{(x_i,y_i)\right\}}_{i=1}^n\subset \Omega \times \mathbb{R}$, one seeks a function s such that $s\left(x_i\right)=y_i$ for all i. A generic finite-dimensional ansatz has form

$$s\left(x\right)={\displaystyle \sum _{j=1}^n}{c}_j{B}_j\left(x\right),$$

(9)

and is well posed if and only if the interpolation matrix $B={\left[B_j\left(x_i\right)\right]}_{i,j}$ is nonsingular. In one dimension, suitable polynomial bases may work, but in several dimensions the Haar–Mairhuber–Curtis theorem shows that no fixed finite-dimensional space of continuous functions can serve as a universal interpolation space for arbitrary sets of points  [25,26]. This is one of the reasons why kernel methods are so natural here: instead of fixing the basis in advance, they let the basis depend on the data through the kernel sections.

With the standard kernel basis the interpolant becomes

$$s\left(x\right)={\displaystyle \sum _{j=1}^n}{c}_{j}K(x,x_j)=k{\left(x\right)}^{\top }c,K_{S}c=y,$$

(10)

where $k{\left(x\right)}^{\top }=(K(x,x_1),\dots ,K(x,x_n))$ and $y={(y_1,\dots ,y_n)}^{\top }$. Positive definiteness guarantees uniqueness whenever the sample points are distinct. The drawback is numerical: even when the interpolant exists uniquely, the basis $\left\{K(·,x_i)\right\}$ can be severely ill-conditioned and therefore unstable for out-of-sample evaluation. This is exactly the motivation for data-dependent bases in kernel approximation  [13,14,18]. The role of Lagrange, Newton, and SVD bases is not merely cosmetic; they are the mechanism that makes the same empirical subspace usable in practice.

2.5. Data-Dependent Alternate Bases in the Empirical RKHS

The standard kernel sections

$$k{\left(x\right)}^{\top }=(K(x,x_1),\dots ,K(x,x_n))$$

span ${\mathcal{H}}_K\left(S\right)$, but this representation is often ill-conditioned. A unifying result for kernel-based spaces states that every data-dependent basis $v{\left(x\right)}^{\top }=(v_1\left(x\right),\dots ,v_r\left(x\right))$ spanning the same empirical space can be written as

$$v{\left(x\right)}^{\top }=k{\left(x\right)}^{\top }T,$$

(11)

with a sample matrix $V=\left[v_j\left(x_i\right)\right]$ satisfying

$$K_S=V{T}^{-1}$$

(12)

Ref. [13]. This factorization viewpoint is central to the present paper: interpolation, eigentrace continuation, and out-of-sample kernel reconstruction can all be expressed in the same coordinates.

The three bases relevant here are Lagrange, Newton, and SVD. The Lagrange basis $\ell {\left(x\right)}^{\top }=k{\left(x\right)}^{\top }{K}_S^{-1}$ is the exact cardinal basis and therefore the natural reference point, but it inherits the instability of $K_S^{-1}$. The Newton basis is obtained from a Cholesky factorization $K_S=N{N}^{\top }$ through

$$n{\left(x\right)}^{\top }=k{\left(x\right)}^{\top }{N}^{-\top },$$

(13)

and yields a triangular, pivoted, adaptive basis that is particularly natural when a rank cap is imposed. The SVD basis follows from $K_S=Q{\Sigma }^2{Q}^{\top }$ and is given by

$$s{\left(x\right)}^{\top }=k{\left(x\right)}^{\top }Q{\Sigma }^{-1},$$

(14)

which aligns the basis with the empirical spectral directions of the matrix.

We keep these definitions and their role in the main text, but move the construction details—including the Newton recursion, the power function, the weighted-SVD variant, and the Hermite-type alternative—to Appendix A [13,14,18]. This keeps the body of the paper focused on the central matrix-to-function argument while still documenting the ingredients required to reproduce the method.

This basis-centered view is consistent with recent stabilized greedy kernel algorithms. Wenzel, Santin and Haasdonk [29] show that adaptive, data-dependent selection strategies can improve stability and convergence properties in kernel interpolation. In the present framework this supports treating the basis-extension step as part of the reconstruction model, not merely as a numerical implementation detail.

2.6. Combination Rules and Positive-Semidefinite Rectification

Suppose now that several base kernels $K_1,\dots ,K_m$ are available on the same sample. We first combine them at matrix level, obtaining

$$K_S^{*}=F(K_{1,S},\dots ,K_{m,S},y),$$

(15)

where $y=(y_1,\dots ,y_n)$ if the rule is supervised. The two families used in this paper are standard matrix-combination rules from kernel fusion and from earlier kernel-combination development [5,6,18].

2.6.1. Sum-Type Combinations

The simplest possibility is a linear or convex combination,

$$K_S^{\mathrm{sum}}={\displaystyle \sum _{\ell =1}^m}{w}_{\ell }{K}_{\ell ,S},w_{\ell }\ge 0,$$

(16)

which is automatically positive semidefinite whenever the base matrices are. This is the main family used in Telco, where different kernels are associated with different business blocks and multiple Gaussian scales.

2.6.2. Pick-Out/Max–Min

The supervised rule inherited from the kernel-combination literature and preliminary work is the pick-out or Max–Min construction, which acts entrywise on the sample matrix [6,18]:

$$K^{*}(x_i,x_j)=\left\{\begin{array}{cc}max\{K_1(x_i,x_j),\dots ,K_m(x_i,x_j)\},\hfill & y_i=y_j,\hfill \\ min\{K_1(x_i,x_j),\dots ,K_m(x_i,x_j)\},\hfill & y_i\ne y_j.\hfill \end{array}\right.$$

(17)

For $m=2$ this becomes

$$K^{*}(x_i,x_j)={\displaystyle \frac{1}{2}}\left(K_1(x_i,x_j)+K_2(x_i,x_j)\right)+{\displaystyle \frac{1}{2}}{y}_i{y}_j\left|K_1(x_i,x_j)-K_2(x_i,x_j)\right|.$$

(18)

The broader parametric family studied for supervised kernel-matrix combination is

$$K^{*}(x_i,x_j)={\displaystyle \frac{1}{m}}{\displaystyle \sum _{t=1}^m}{K}_t(x_i,x_j)+\tau y_i{y}_j\sum _{t<\ell }g\left(K_t(x_i,x_j)-K_{\ell }(x_i,x_j)\right),$$

(19)

where g is convex and $\tau >0$ controls the supervised deformation  [6]. These rules are useful because they may sharpen class separation, but they are also precisely the rules that are naturally defined as sample matrices rather than as analytic kernels on all of $\mathcal{X}\times \mathcal{X}$.

2.6.3. Rectification

Supervised combinations may produce indefinite matrices. Several standard corrections are therefore used in similarity-based kernel learning  [6,7,30]: (i) spectral clipping or MDS-style truncation, which keeps only positive eigenvalues; (ii) absolute-value rectification, which replaces negative eigenvalues by their modulus; and (iii) diagonal shifting, which adds a constant to the diagonal to move the spectrum into the positive half-line. In the experiments reported here we use spectral clipping as the default rectification:

$$K_{+}=Q{\Lambda }_{+}{Q}^{\top },{\Lambda }_{+}=\mathrm{diag}(max\{{\lambda }_j,0\}).$$

This choice is the most transparent for the proposed spectral reconstruction, because only positive directions are retained. To monitor how much the rule has altered the original matrix, we report the relative PSD perturbation

$${\Delta }_{\mathrm{PSD}}={\displaystyle \frac{\parallel K_{+}{-K\parallel }_F}{{\parallel K\parallel }_F}}.$$

A recent alternative is the polar-decomposition correction of Münch, Röder and Schleif [31], which maps non-PSD kernel matrices to PSD matrices while aiming to preserve the topology of the data. Comparing that correction with elementary spectral clipping before the coordinate-extension step is left for future work.

2.7. When Kernel Combinations Are Analytic and When They Are Only Sample Objects

An important distinction in multiple-kernel learning is whether the combination is a genuine pointwise kernel or only a sample-level matrix rule. Some combinations are defined pointwise from the start. If the base kernels $K_1,\dots ,K_m$ are analytic on $\mathcal{X}\times \mathcal{X}$ and the rule is, for instance, a convex sum,

$$K^{*}(x,z)={\displaystyle \sum _{\ell =1}^m}{\alpha }_{\ell }{K}_{\ell }(x,z),{\alpha }_{\ell }\ge 0,$$

(20)

then $K^{*}$ is already an out-of-sample kernel and no reconstruction problem arises [3,4,5]. By contrast, supervised rules such as pick-out, Max–Min, or other matrix-level transformations may only be available through their sample matrix $K_S^{*}$ because they depend on the observed labels or on sample-specific rectifications. In that case the analytic object is not given directly even if all the base kernels are known pointwise.

This distinction explains why the matrix-to-function problem is substantive rather than formal. The issue is not “combining kernels” in the abstract, but recovering an evaluable kernel once the combination has been performed at matrix level. This matters not only for classification but for any task that needs a coherent out-of-sample geometry, including clustering, embedding, or information fusion [6,7,32]. Our emphasis on train–test blocks $K^{*}(S,Z)$ is therefore only one operational face of a broader problem: how to turn a finite, possibly supervised, similarity object into a positive-semidefinite finite-feature kernel that can be used consistently beyond the training sample. When the continued coordinates are continuous on a compact domain, this finite-feature kernel is a Mercer kernel.

2.8. Fusion Kernel as an Alternative Reconstruction Route

The Fusion Kernel (FK) construction is the natural alternative to the direct basis-based route [18,32]. Starting from the same combined matrix $K_S^{*}$, FK seeks an out-of-sample kernel by expressing the target eigenfunctions as linear combinations of eigenfunctions of the original base kernels. Conceptually, FK works through the eigenspaces of $K_1,\dots ,K_m$, while the method proposed in this paper works directly with the eigensystem of $K_S^{*}$.

We keep FK visible in the main text because it clarifies the relation between the proposed basis route and previous fusion-kernel approaches. At the same time, the full algebra of FK is not needed to follow the main argument. We therefore keep only the conceptual comparison here and move the explicit FK representation and the discussion of when both routes coincide or diverge to Appendix C. Empirically, FK is retained as a secondary baseline but is not one of the central methods in the main tables.

The distinction is also useful for positioning the method relative to the recent work of Fanuel et al. [8] and Münch et al. [9]. Fanuel et al. provided explicit out-of-sample formulas for PSD embeddings, whereas our input is the final combined matrix and our central task is to reconstruct its eigentraces through stable RKHS bases. Münch et al. designed subspace-fusion mechanisms for heterogeneous proximity data; our method can instead be applied after such a fusion stage to obtain a kernel evaluator on new points. Studying combinations of their extension or fusion formulas with the basis reconstruction proposed here is a natural direction for future work.

2.9. Problem Statement and Target Properties

We formulate the matrix-to-function problem through four target properties, in line with the basis-combination motivation and with the kernel-fusion literature. Starting from a sample matrix $K_S^{*}$, we do not seek the unique continuation of $K_S^{*}$, because no such unique continuation exists. Instead, we seek an inductive finite-feature extension

$$\widehat{K}(x,z)=g{\left(x\right)}^{\top }Cg\left(z\right),C⪰0,$$

where the feature map g is learned from the training representation available for the objects. This representation may consist of ordinary vector covariates, tabular variables, distances to the training objects, or other relational descriptors. The extension should satisfy:

1.

sample agreement: $\widehat{K}(x_i,x_j)={\left(K_S^{*}\right)}_{ij}$ whenever the retained coordinates are exact on the sample, or approximately so under a controlled truncation or coordinate-regression error;

2.

positive definiteness: the extension is PSD because $C⪰0$;

3.

operational out-of-sample evaluation: the block $\widehat{K}(S,Z)$ can be computed without access to test labels;

4.

discriminative fidelity: the geometry induced by the combined matrix is preserved up to a reconstruction error that can be monitored.

These requirements are trivial when the combination rule is already defined pointwise, but they are the central difficulty when F is only a matrix rule. A related strategy, also described in earlier work on functional learning of kernels, is the Fusion Kernel (FK) [18,32], which reconstructs the target kernel by expressing its eigenfunctions as linear combinations of the eigenfunctions of the base kernels,

$${\varphi }_h\left(x\right)={\displaystyle \sum _{\ell =1}^m}{\displaystyle \sum _{j=1}^{d_{\ell }}}{c}_{j\ell }{\varphi }_j^{(\ell )}\left(x\right),$$

(21)

so that the target kernel is then recovered through a Mercer-like expansion in those fused eigenfunctions. FK provides an important conceptual baseline because it shows that one may reconstruct the combined kernel either indirectly through the eigensystems of the base kernels or directly from the eigensystem of the combined matrix. The method developed here follows the second route: rather than first expanding on the base kernels, we reconstruct directly in data-dependent bases associated with the combined matrix itself. This direct matrix-to-function route is what allows us to connect the spectral decomposition of $K_S^{*}$ with a stable out-of-sample evaluator.

Overall, the theoretical framework can be summarized as follows. Classical RKHS and Mercer theory justify interpreting a PSD matrix spectrally; alternate bases make the empirical eigentraces stable enough to evaluate away from the sample; and recent PSD-embedding and subspace-fusion approaches show that the same out-of-sample issue appears in neighboring settings [8,9,31]. The contribution of the present paper is not a new fusion rule and not a new embedding objective. It is a reconstruction step that lifts a given empirical kernel matrix—often produced by supervised or multi-source combination—to an evaluable kernel function. This perspective opens up future work on combining PSD-embedding formulas, subspace-fusion rules, and PSD corrections with the basis-extension step studied here.

3. Proposed Reconstruction Method

3.1. Spectral Decomposition of the Combined Matrix

Let $K_S^{*}$ be the rectified combined matrix on the training sample. Its thin eigendecomposition is

$$K_S^{*}=U_r{\Lambda }_r{U}_r^{\top },$$

(22)

where ${\Lambda }_r=\mathrm{diag}({\lambda }_1,\dots ,{\lambda }_r)$ contains the retained positive matrix eigenvalues and $U_r=(u_1,\dots ,u_r)$ the corresponding orthonormal eigenvectors. In the computational method we use the associated matrix-coordinate functions

$${\widehat{\psi }}_h\left(x_i\right)=u_{ih},1\le i\le n.$$

(23)

This convention differs from the Mercer-operator normalization ${\widehat{\varphi }}_h\left(x_i\right)\approx \sqrt{n}{u}_{ih}$, whose eigenvalues are ${\lambda }_h/n$. Both conventions are equivalent if the scaling is used consistently, but (23) is preferable algorithmically because it reconstructs $K_S^{*}$ directly with the matrix eigenvalues:

$$K_S^{*}={\displaystyle \sum _{h=1}^r}{\lambda }_h{u}_h{u}_h^{\top }.$$

3.2. Extending Empirical Coordinates

The previous step gives only sample coordinates. To evaluate a new object we must learn a map from the available object representation to those coordinates. Let

$$G_S=U_r={\left[g_h\left(x_i\right)\right]}_{i,h}\in {\mathbb{R}}^{n\times r},g{\left(x_i\right)}^{\top }=(u_{i1},\dots ,u_{ir}).$$

For each coordinate h we fit a scalar regression problem

$$x_i⟼u_{ih},$$

or, in relational data, a regression from the available dissimilarities to the training objects. The fitted coordinate functions are denoted ${\widehat{g}}_h\left(x\right)$, and 

$$\widehat{g}{\left(x\right)}^{\top }=({\widehat{g}}_1\left(x\right),\dots ,{\widehat{g}}_r\left(x\right)).$$

This is the formal step that turns a sample matrix into an inductive object. It also makes clear why the construction is not unique: changing the coordinate model, the retained rank, or the basis used for coordinate regression changes the resulting extension while leaving the same training matrix as the target.

The same step can be described through Newton or SVD bases. Let $v_1,\dots ,v_q$ be a data-dependent basis, with sample matrix $V=\left[v_j\left(x_i\right)\right]\in {\mathbb{R}}^{n\times q}$. Each coordinate vector $u_h$ may be projected onto that basis through

$$a_h=arg\underset{a\in {\mathbb{R}}^q}{min}{∥Va-u_h∥}_2^2+\eta {\parallel a\parallel }_2^2,$$

(24)

where $\eta \ge 0$ is a small numerical ridge. The resulting continuation is

$${\widehat{g}}_h\left(x\right)={\displaystyle \sum _{m=1}^q}{a}_{mh}{v}_m\left(x\right).$$

(25)

When V is square and nonsingular, this is an exact change of coordinates. When a rank cap is imposed or a numerical ridge is added, it is a stable projection of the empirical coordinate onto the retained basis.

3.3. Finite-Feature Kernel and Validity

After the coordinates have been extended, the reconstructed kernel is

$${\widehat{K}}_r(x,z)={\displaystyle \sum _{h=1}^r}{\lambda }_h{\widehat{g}}_h\left(x\right){\widehat{g}}_h\left(z\right)=\widehat{g}{\left(x\right)}^{\top }{\Lambda }_r\widehat{g}\left(z\right).$$

(26)

More generally, for a basis representation $v\left(x\right)$ we use

$${\widehat{K}}_r(x,z)=v{\left(x\right)}^{\top }Cv\left(z\right),C⪰0.$$

(27)

The coefficient matrix is diagonal only in coordinates aligned with the retained eigenvectors. In a pivoted Newton basis, for example, C is generally dense but still positive semidefinite.

The following proposition is the basic mathematical guarantee used throughout the paper.

Proposition 1 (PSD extension, interpolation, and coordinate error).

Let $g:\mathcal{X}\to {\mathbb{R}}^r$ be any feature map and let $C⪰0$. Then

$$\widehat{K}(x,z)=g{\left(x\right)}^{\top }Cg\left(z\right)$$

is a positive-semidefinite finite-feature kernel. Let G be the sample matrix with rows $g{\left(x_i\right)}^{\top }$. If $GC{G}^{\top }=K_S^{*}$, then $\widehat{K}(S,S)=K_S^{*}$. If the fitted sample coordinates are $\widehat{G}=G+E$ then

$${\parallel \widehat{G}C{\widehat{G}}^{\top }-GC{G}^{\top }\parallel }_F\le 2{\parallel G\parallel }_F{\parallel C\parallel }_2{\parallel E\parallel }_F+{\parallel C\parallel }_2{\parallel E\parallel }_F^2.$$

Proof. 

For any finite collection $x_1,\dots ,x_N$ and coefficients $c_1,\dots ,c_N$,

$$\sum _{i,j}{c}_i{c}_j\widehat{K}(x_i,x_j)={\left(\sum _i{c}_{i}g\left(x_i\right)\right)}^{\top }C\left(\sum _j{c}_{j}g\left(x_j\right)\right)\ge 0,$$

because $C⪰0$. The interpolation statement follows immediately from $GC{G}^{\top }=K_S^{*}$. For the error bound, expand

$$(G+E)C{(G+E)}^{\top }-GC{G}^{\top }=EC{G}^{\top }+GC{E}^{\top }+EC{E}^{\top }$$

and apply submultiplicativity of the Frobenius and spectral norms.    □

Equation (27) also clarifies the interpolation issue. If the full effective rank is retained and the coordinate model is exact on the sample then ${\widehat{K}}_r(S,S)=K_S^{*}$. If the spectrum is truncated, the PSD part is clipped, or the coordinate regression is imperfect then agreement is exact only for the retained fitted coordinates. For this reason the article reports predictive metrics together with PSD and reconstruction diagnostics rather than claiming a unique exact reconstruction.

3.4. Out-of-Sample Basis Evaluation

To evaluate ${\widehat{K}}_r$ on new points it remains to compute the extended coordinate vector $\widehat{g}\left(z\right)$, or equivalently $\widehat{v}\left(z\right)$ when a non-spectral basis is used. In our implementation this is done by regressing each coordinate on the representation available for the objects and then predicting the coordinate values on the new points [18]. For vector or tabular data the regressors use the observed variables. For relational data they may use the dissimilarities from the new object to the training objects. Let $v_m\left(x_i\right)$ be the m-th basis coordinate on the sample. We fit a scalar regression model

$$x_i⟼v_m\left(x_i\right),$$

for each $m=1,\dots ,q$, and obtain predicted values ${\widehat{v}}_m\left(z\right)$ on the new point. Stacking them gives

$$\widehat{v}{\left(z\right)}^{\top }=({\widehat{v}}_1\left(z\right),\dots ,{\widehat{v}}_q\left(z\right)).$$

The train–test block required by a precomputed-kernel classifier is then assembled as

$${\widehat{K}}_r(S,Z)=VC{\widehat{V}}_Z^{\top },$$

(28)

where ${\widehat{V}}_Z={\left[{\widehat{v}}_j\left(z_{\ell }\right)\right]}_{\ell ,j}$. In our implementation this regression step is implemented through kernelized regression models, but the theory does not depend on the particular regressor: any method that estimates the coordinates stably can be used.

This basis-regression view is what differentiates the present method from both Nyström and FK. Nyström extends eigenfunctions directly through the original kernel sections. FK reconstructs the target eigensystem through the eigensystems of the base kernels. In contrast, our method first changes coordinates to a stable basis attached to the combined matrix and only then performs the out-of-sample continuation. In the language of the kernel approximation and fusion literature, the procedure combines two ingredients that are often treated separately [6,13,14]: the interpolation-oriented perspective of alternate bases and the kernel-fusion perspective of matrix combinations. The whole point is that these are not separate technicalities but consecutive parts of the same pipeline.

3.5. Algorithmic Summary

The whole procedure can be summarized as follows.

1.

Compute the base kernel matrices on the training sample and build the combined matrix $K_S^{*}$ with the chosen combination rule.

2.

Symmetrize and project $K_S^{*}$ to the PSD cone if needed; record the relative PSD perturbation.

3.

Compute the eigendecomposition (22) and retain the desired positive-rank subspace.

4.

Define empirical matrix coordinates by ${\widehat{\psi }}_h\left(x_i\right)=u_{ih}$, avoiding the Mercer scaling ambiguity in the practical algorithm.

5.

Optionally express those coordinates in a Newton or SVD basis by solving (24).

6.

Learn a coordinate map from the available object representation to the retained coordinates.

7.

Form the finite-feature kernel through (26) and (27) and assemble the train–test block (28).

8.

Classify the test points using a precomputed-kernel SVM  [3,33].

This reorganizes the sample-to-function algorithm from earlier basis-combination work [18] in an explicitly spectral way, so that each stage has a direct interpretation in Mercer terms.

3.6. Complexity, Conditioning, and Practical Remarks

The dominant costs are the eigendecomposition or the Cholesky/SVD factorization of the combined matrix and the set of regressions used to predict the basis coordinates. If $r\ll n$, storage is $O\left(nr\right)$ and out-of-sample evaluation is reduced to matrix products involving V, C, and the predicted basis matrix on the test set. Newton and SVD therefore provide two practically complementary regimes: Newton is attractive when one wants rank-adaptive, pivoted, or triangular structure; SVD is attractive when one wants the most direct spectral route and a basis already aligned with the dominant sample eigendirections.

This cost profile should be compared with alternative out-of-sample devices with some care. Nyström-type methods are typically cheaper when a small landmark set or a known kernel section $K(x,x_i)$ is available, but for a matrix-only supervised combination the section $K^{*}(x,x_i)$ is exactly the missing object. Fusion-kernel constructions can also continue a combined matrix, but they do so through the eigenspaces of the base kernels rather than directly through the eigensystem of $K_S^{*}$. The present implementation therefore benchmarks the matrix-to-function step primarily through leakage-free train–test gaps and, where an analytic oracle exists, relative Frobenius and functional-MSE errors. A full large-scale runtime benchmark against all low-rank extension strategies is outside the present experimental scope.

In the experiments, the two reconstructions were consistently close to each other, which indicates that the main limitation often lies in the quality of the combined matrix itself rather than in the basis used for continuation. This is why the article keeps Fusion Kernel only as a secondary computational baseline: for the direct basis-based route, Newton and SVD became the two methods that best capture the stable matrix-to-function transition.

4. Materials and Methods

4.1. Aim of the Experiments

The experiments are intended as a validation of the matrix-to-function step, not as a search for a universally optimal classifier. The question is the following: after a finite kernel matrix has been produced on a training sample, can it be converted into a kernel function that is evaluable on new points, and does the classifier built from that function behave on held-out points as it behaves on the points used to build the extension?

For this reason the predictive metrics reported below should be read as functional diagnostics. We compare the error measured on the training matrix by inner cross-validation with the error obtained after extending the same matrix to the test points. A small discrepancy between these two quantities indicates that the conversion from matrix to function has preserved the relevant geometry of the classifier. We deliberately do not tune the kernel dictionary to obtain optimal classification results. The kernels are the same broad dictionary used in the earlier kernel-combination experiments: linear, cosine, polynomial, multiscale Gaussian, Laplacian, rational-quadratic, class-scale, local-scaling, prototype, and supervised weighted variants. This makes the test more honest: the goal is to show that the reconstruction works even for a historical, non-specialized collection of kernels.

We distinguish three experimental cases.

• Case A: analytic combinations. Direct means and alignment-weighted sums of evaluable base kernels already have a train–test formula. They are used as stability references. If the matrix-to-function implementation is sound, these combinations should remain close to the behavior observed on the training matrix.
• Case B: matrix-only supervised rules. Pick-out/max–min and percentile rules are constructed only on the training matrix. They are the main stress test for the reconstruction layer, because no test labels are available when the train–test block is generated.
• Case C: relational data. Chickenpieces starts from dissimilarity matrices between shape objects. Here the usefulness of the framework is most visible: the input is not a privileged coordinate representation, but a family of pairwise shape comparisons.

The protocol follows a strict split discipline. Training and test sets are formed before any kernel parameter, scaling constant, alignment score, kernel selection, or combination rule is estimated, as required for unbiased model assessment in supervised learning [34,35]. For label-aware rules, labels are used only on the training set. Test labels are never used to construct test kernels. This point is essential for supervised matrix combinations such as max–min/pick-out.

The computational workflow is implemented as ISO C++17 source code, with accompanying preparation scripts written for R version 4.2.0 or later and Python version 3.10 or later.

4.2. Data Sets

The main vector-data evidence reported in the present tables uses four complete 30-repetition strict runs: Synthetic, Breast Cancer, Ionosphere, and Telco Customer Churn. Telco is the heterogeneous tabular benchmark, while Chickenpieces is reported separately as the relational case in which kernels are built from dissimilarity matrices.

The data sets were selected to cover complementary regimes rather than to form a large-scale leaderboard. Synthetic provides a controlled nonlinear problem in which an oracle train–test block can be computed for smooth analytic rules. Breast Cancer gives a small ordinal biomedical benchmark; Ionosphere gives a continuous radar benchmark; Telco gives a heterogeneous tabular problem with binary, categorical, account, and billing variables; and Chickenpieces gives a pure relational setting in which the objects are available through multiple shape dissimilarities. The resulting collection tests whether the proposed matrix-to-function step behaves consistently across vector, heterogeneous business-table, and dissimilarity-only settings. It does not exhaust the range of possible applications. Larger or more complex data sets would require the same mathematical construction combined with scalable eigensolvers, landmark or randomized low-rank factorizations, and faster coordinate-regression or basis evaluation schemes, as discussed in the complexity remarks and conclusions.

• Synthetic. A balanced two-class banana-style simulation inherited from the preliminary experiments [18]. A latent variable $u\sim \mathcal{U}[-1,1]$ and Gaussian noise $e\sim \mathcal{N}(0,0.01)$ are generated, and the two classes are defined by $(x_1,x_2)=(u+1,u^2+e)$ and $(x_1,x_2)=(u+\frac{7}{5},-u^2+1+e)$. This produces two curved clouds that are not linearly separable and provides a controlled setting in which an oracle train–test kernel block can be computed for diagnostic purposes.
• Breast Cancer Wisconsin (Original). A compact biomedical benchmark retained for continuity with the preliminary experiments [36]. The original database contains 699 fine-needle aspirate cases described by nine ordinal cytological attributes measured on a 1–10 scale. After removing the 16 cases with missing BareNuclei, the usable sample contains 683 observations, with benign and malignant diagnoses as the target classes.
• Ionosphere. A radar-return classification benchmark from the UCI repository [37]. The data were collected by a phased-array system in Goose Bay, Labrador, and they contain 351 observations with 34 continuous predictors derived from the complex autocorrelation values of 17 pulse numbers. The binary label distinguishes “good” returns, which show ionospheric structure, from “bad” returns.
• Telco Customer Churn. A heterogeneous customer-level table distributed as an IBM/Kaggle churn benchmark [38]. The original file contains 7043 customers and 21 columns, including the binary response Churn. Its predictors combine demographic information, subscribed telephone and internet services, contract type, payment method, tenure, and billing amounts. Telco is therefore a natural tabular fusion benchmark: different kernels can be attached to different semantic blocks of variables, rather than merely to different bandwidths on a single homogeneous feature space.
• Chickenpieces. A purely relational benchmark consisting of 446 silhouette objects from five chicken-part classes (wing, back, drumstick, thigh-and-back, and breast) [12,39]. The PRDisData release provides 44 dissimilarity matrices obtained from 11 contour normalizations and four edit-cost settings. Each silhouette is encoded through a contour-string representation. The raw objects are therefore not treated as vectors of coordinates; they are compared through several shape dissimilarities, which are subsequently converted into kernel similarities for classification.

One of the 44 Chickenpieces matrices can be obtained as follows. Fix one contour normalization r and one edit-cost setting c. Let $s_i^{\left(r\right)}$ and $s_j^{\left(r\right)}$ be the resulting contour strings for silhouettes i and j. Since the starting point on a closed contour is arbitrary, the comparison minimizes the edit cost over all cyclic shifts of the second string:

$$D_{ij}^{(r,c)}=\underset{\tau \in \mathcal{C}}{min}{Edit}_c\left(s_i^{\left(r\right)},\tau (s_j^{\left(r\right)})\right).$$

Here, $\mathcal{C}$ is the set of cyclic shifts. In the rotation-invariant cyclic edit distance of Bunke and Bühler [40], substituting contour directions with angles $\alpha$ and $\beta$ is weighted by $|\alpha -\beta |$, whereas insertions and deletions receive the fixed penalty associated with the chosen cost setting c. Computing this value for all pairs gives one $446\times 446$ dissimilarity matrix $D^{(r,c)}$. The 44 matrices arise from the $11\times 4$ choices of $(r,c)$; the preprocessing step below turns each of them into RBF and Laplacian similarity kernels.

Telco requires a separate comment because it is not a purely numerical benchmark. The customer identifier is removed from the model, and the snapshot shown in

Table 1. Selected rows and representative columns from the Telco Customer Churn table. The example shows the coexistence of binary, categorical, account/ordinal, and continuous variables within the same churn-prediction task.

Demographics			Service/Account			Contract and Billing				Target
Gender	Senior	Partner	Tenure	Phone	Internet	Contract	Payment	Monthly	Total	Churn
Female	0	Yes	1	No	DSL	Month-to-month	Electronic check	29.85	29.85	No
Male	0	No	34	Yes	DSL	One year	Mailed check	56.95	1889.50	No
Male	0	No	2	Yes	DSL	Month-to-month	Mailed check	53.85	108.15	Yes
Male	0	No	45	No	DSL	One year	Bank transfer auto	42.30	1840.75	No
Female	0	No	2	Yes	Fiber optic	Month-to-month	Electronic check	70.70	151.65	Yes
Female	0	No	8	Yes	Fiber optic	Month-to-month	Electronic check	99.65	820.50	Yes

is only illustrative: the full source file contains additional service indicators, including online security, online backup, device protection, technical support, streaming TV, and streaming movies. The displayed rows combine binary variables such as SeniorCitizen, categorical service variables such as InternetService, account variables such as Contract and PaymentMethod, continuous billing variables such as MonthlyCharges and TotalCharges, and the target variable Churn. This mixture makes Telco a natural case for kernel fusion: blocks of variables may encode complementary notions of customer similarity.

For visual context, Figure 3 shows six distinct binary silhouettes from the Chickenpieces benchmark, without contour extraction or contour-string processing. These examples were selected from the public example panel of the data set in [41], avoiding near-duplicate visual shapes. The panel is intended only to indicate the type of binary shape objects used in this benchmark.

The 30-repetition vector experiments use train–test sizes $350/150$ for Synthetic, $478/205$ for Breast Cancer, $245/106$ for Ionosphere, and $1400/600$ for Telco. Each Telco repetition is based on a stratified subsample of 2000 customers from the full 7043-customer table. Chickenpieces uses the full set of 446 objects and five classes; the repeated splits use a $70/30$ train–test partition.

4.3. Preprocessing

For the vector data sets, numeric variables are standardized using training statistics only. Categorical variables are encoded using a training-based one-hot design. In Telco, the customer identifier is removed, categorical fields are expanded, and the continuous variables tenure, MonthlyCharges, and TotalCharges are standardized on the training split.

For Chickenpieces, each dissimilarity matrix is symmetrized when needed, its diagonal is set to zero, and it is re-scaled by its positive off-diagonal median. Two distance-to-kernel transformations are then used:

$$K_{\mathrm{RBF}}(i,j)=exp\left(-{\displaystyle \frac{D_{ij}^2}{2{\sigma }^2}}\right),K_{\mathrm{LAP}}(i,j)=exp\left(-{\displaystyle \frac{D_{ij}}{\sigma }}\right),$$

with $\sigma$ estimated from the corresponding training distances. Each kernel is normalized before entering the comparison.

4.4. Kernel Dictionary for Vector Data

The vector experiments use the same deliberately broad dictionary in every split:

$$\mathcal{D}=\{\mathtt{LIN},\mathtt{POLY}\mathtt{2},\mathtt{COS},\mathtt{RBF}\mathtt{01},\dots ,\mathtt{RBF}\mathtt{10},\mathtt{LAP},\mathtt{RQ},\mathtt{INTRA},\mathtt{INTER},\mathtt{WRBF},\mathtt{LOCAL},\mathtt{PROTO}\}.$$

Table 2. Vector-data kernel dictionary. The formulas are written for a generic training split; $x,z\in {\mathbb{R}}^p$, Q denotes the empirical quantile set of non-zero training distances, and all label-dependent quantities are estimated only from training labels.

Name	Definition	Role in the Dictionary
LIN	$K(x,z)=x^{\top }z$	Linear baseline.
POLY2	$K(x,z)={\left(1+x^{\top }z/p\right)}^2$	Low-degree interaction baseline.
COS	$K(x,z)=x^{\top }z/(\parallel x\parallel \parallel z\parallel )$	Angular similarity, useful when scale is less informative than direction.
RBF01–RBF10	$K_q(x,z)={exp\{-\parallel x-z\parallel }^2/\left(2{\sigma }_q^2\right)\}$, with ${\sigma }_q$ the q-quantile of non-zero Euclidean training distances and $q\in Q$	Multiscale Gaussian core of the dictionary [42].
LAP	$K(x,z)=exp\{-\parallel x-z{\parallel }_1/{\sigma }_{L1}\}$	Less-smooth distance kernel based on the median $L^1$ training scale.
RQ	$K(x,z)={\left({1+\parallel x-z\parallel }^2/\left(2\alpha {\sigma }_0^2\right)\right)}^{-\alpha }$, with $\alpha =1$	Scale-mixture alternative to a single Gaussian bandwidth.
INTRA	$K(x,z)=exp\{-\parallel x-z{\parallel }^2/\left(2{\sigma }_{\mathrm{intra}}^2\right)\}$	Gaussian bandwidth set by the median within-class training distance.
INTER	$K(x,z)=exp\{-\parallel x-z{\parallel }^2/\left(2{\sigma }_{\mathrm{inter}}^2\right)\}$	Gaussian bandwidth set by the median between-class training distance.
WRBF	$K(x,z)=exp\{-{\sum }_{r=1}^p{w}_r{(x_r-z_r)}^2/\left(2{\sigma }_w^2\right)\}$	Supervised diagonal metric; $w_r$ measures training-set variable relevance.
LOCAL	$K(x_i,x_j)=exp\{-\parallel x_i-x_j{\parallel }^2/\left({\sigma }_i{\sigma }_j\right)\}$	Self-tuning kernel with ${\sigma }_i$ given by a nearest-neighbor distance [43].
PROTO	$p_c\left(x\right)=exp\{-\parallel x-{\mu }_c{\parallel }^2/\left(2{\sigma }_p^2\right)\}$, $K(x,z)=p{\left(x\right)}^{\top }p\left(z\right)$	Similarity through training-set class prototypes ${\mu }_c$.

gives the definitions used before trace normalization. All quantities that depend on the data—bandwidths, nearest-neighbor scales, class scales, prototype locations, and variable weights—are estimated on the training split only.

The RBF kernels use empirical distance quantiles as bandwidths, the Laplacian kernel uses the median $L^1$ distance, and the rational-quadratic kernel uses the median Euclidean scale. The class-scale kernels INTRA and INTER estimate their bandwidths from within-class and between-class training distances. LOCAL is a self-tuning kernel with a nearest-neighbor scale [43]. PROTO is a supervised prototype kernel estimated from training labels. All kernels are trace-normalized before combination:

$${\tilde{K}}_{\ell }={\displaystyle \frac{n{K}_{\ell }}{tr\left(K_{\ell }\right)}}.$$

Automatic Selection Protocol

The suffix auto has a fixed meaning throughout the tables. Within each outer training split, kernels are first ranked by centered alignment with the training-label kernel, computed on the training block only. For a candidate number $m_0$, the retained index set is

$$I_{m_0}=\left\{\mathrm{the}{m}_0\mathrm{highest}\text{-}\mathrm{ranked}\mathrm{training}\mathrm{kernels}\right\}.$$

The pair $(m_0,C_{\mathrm{SVM}})$ is then selected by stratified inner cross-validation on the training split. The vector-data runs use $m_0\in \{3,5,7,10\}$ and $C_{\mathrm{SVM}}\in \{0.25,0.5,1,2,4,8,16\}$; the Chickenpieces runs use the same $C_{\mathrm{SVM}}$ grid and $m_0\in \{3,5,7,10,15\}$. For mean_auto, the selected kernels are averaged with uniform weights. For alignment_auto, they are combined with non-negative weights proportional to their positive centered alignments. For matrix-only rules, the same selected kernels are first combined on the training block, the resulting matrix is projected onto the PSD cone when necessary, and the train–test block is obtained by the spectral/KRR continuation described below. The label “best individual” is a descriptive reference chosen from the completed single-kernel summaries; inside each outer split only its SVM penalty is tuned by inner CV. The label “best matrix-only” denotes the predeclared matrix-only rule with the lowest mean inner-CV error for that data set; it is not selected from held-out test performance.

4.5. Combination Rules

We compare three families of combinations.

1.

Direct convex combinations. We use a simple mean and an alignment-weighted mean over the selected kernels:

$$K^{\mathrm{mean}}={\displaystyle \frac{1}{m}}\sum _{\ell \in I_m}{\tilde{K}}_{\ell },K^{\mathrm{align}}=\sum _{\ell \in I_m}{w}_{\ell }{\tilde{K}}_{\ell }.$$

The weights $w_{\ell }$ are proportional to the positive centered alignment between ${\tilde{K}}_{\ell }$ and the ideal training label kernel [10,11].

2.

Label-aware matrix combinations. The pick-out/max–min rule is defined on training pairs by

$$K_{ij}^{\mathrm{MM}}=\left\{\begin{array}{cc}\underset{\ell \in I_m}{max}{\tilde{K}}_{\ell ,ij},\hfill & y_i=y_j,\hfill \\ \underset{\ell \in I_m}{min}{\tilde{K}}_{\ell ,ij},\hfill & y_i\ne y_j.\hfill \end{array}\right.$$

Percentile variants replace the maximum and/or minimum by upper and lower empirical percentiles [6]. These rules are supervised and matrix-defined; their test blocks are obtained only through the out-of-sample reconstruction.

3.

Hybrid score selection. As an additional diagnostic, we tested a score

$$score\left(K_{\ell }\right)=\alpha rankCV\left(K_{\ell }\right)+(1-\alpha )rankAlignment\left(K_{\ell }\right),$$

but it did not improve the direct mean combinations and is therefore not used as a principal method.

When a label-aware combination is indefinite, it is symmetrized and projected onto the positive semidefinite cone by clipping negative eigenvalues [7,30].

4.6. Out-of-Sample Evaluation

Direct convex combinations have explicit train–test blocks because the base kernels can be evaluated between test and training points. In contrast, pick-out and percentile rules define a matrix only on the training set. For these rules, we use the eigenfunction-based reconstruction described in the theoretical sections. The training matrix is decomposed spectrally, its positive part is retained, empirical eigenvectors are interpreted as traces of eigenfunctions, and the resulting functions are extended through the chosen basis. The C++ runs reported here use a spectral/KRR continuation for these matrix-only rules. This is stricter than evaluating an oracle matrix, because no test label is used when constructing the train–test block. Concretely, after PSD projection we retain positive eigenvalues larger than ${10}^{-10}$ until the retained positive spectrum explains $99\%$ of the positive trace. If $K_0$ denotes the direct mean of the selected base kernels, the held-out coordinate matrix is obtained by kernel ridge continuation with ridge $\eta ={10}^{-6}$:

$$B={(K_{0,S}+\eta I)}^{-1}{U}_r,{\widehat{U}}_Z=K_{0,ZS}B,$$

and the reconstructed train–test block is

$${\widehat{K}}_{ZS}={\widehat{U}}_Z{\Lambda }_r{U}_r^{\top }.$$

The percentile variants use the upper and lower empirical percentiles $0.75$ and $0.25$, respectively. All SVM evaluations use precomputed-kernel C-SVMs with C selected by the same inner-CV grid; Chickenpieces, the only multiclass benchmark, is handled by a one-versus-one multiclass reduction.

The controlled Synthetic experiment also includes an oracle diagnostic. For rules such as the sum or the mean of base kernels, the true train–test block is available because the analytic combination is known. We can therefore compare the reconstructed block against the oracle block directly. This diagnostic is not used for model selection; it is only a check that the matrix-to-function conversion recovers the off-sample behavior when the training matrix comes from a smooth evaluable combination.

5. Results

5.1. Vector Data Sets: 30-Repetition Validation

Table 3. Main vector-data results over 30 repetitions. Mean classification error and balanced accuracy are reported as mean ± standard deviation.

Data Set	Method	Kernel/Combination	Error	Balanced Accuracy
Synthetic	best individual	RBF03	$0.0809\pm 0.0231$	$0.9191\pm 0.0231$
Synthetic	direct combination	mean_auto	$0.0793\pm 0.0227$	$0.9207\pm 0.0227$
Synthetic	direct combination	alignment_auto	$0.0798\pm 0.0226$	$0.9202\pm 0.0226$
Synthetic	best matrix-only	percentile_inout_auto	$0.1404\pm 0.1198$	$0.8596\pm 0.1198$
Breast	best individual	COS	$0.0289\pm 0.0112$	$0.9736\pm 0.0114$
Breast	direct combination	mean_auto	$0.0311\pm 0.0108$	$0.9711\pm 0.0111$
Breast	direct combination	alignment_auto	$0.0311\pm 0.0113$	$0.9709\pm 0.0116$
Breast	best matrix-only	percentile_inout_auto	$0.0358\pm 0.0124$	$0.9616\pm 0.0164$
Ionosphere	best individual	LOCAL	$0.0484\pm 0.0183$	$0.9441\pm 0.0239$
Ionosphere	direct combination	mean_auto	$0.0519\pm 0.0171$	$0.9381\pm 0.0219$
Ionosphere	direct combination	alignment_auto	$0.0528\pm 0.0174$	$0.9370\pm 0.0230$
Ionosphere	best matrix-only	percentile_in_auto	$0.0487\pm 0.0166$	$0.9411\pm 0.0234$
Telco	best individual	POLY2	$0.2045\pm 0.0154$	$0.6959\pm 0.0226$
Telco	direct combination	mean_auto	$0.2061\pm 0.0154$	$0.6909\pm 0.0218$
Telco	direct combination	alignment_auto	$0.2063\pm 0.0158$	$0.6900\pm 0.0229$
Telco	best matrix-only	percentile_inout_auto	$0.3069\pm 0.0197$	$0.6321\pm 0.0251$

summarizes the main 30-repetition vector-data experiments. For each data set we report the best individual kernel, the two direct combinations, and the supervised matrix-only rule selected by the inner-CV criterion described above after strict out-of-sample reconstruction. The table is not intended to claim that the combined kernels are optimized classifiers. It is intended to show that a matrix built from the historical kernel dictionary can be converted into an evaluable kernel whose test behavior remains close to the behavior observed on the training matrix.

The results support the intended interpretation. On Synthetic, the simple mean slightly improves the best individual RBF scale, showing that the direct combination can recover a useful multiscale geometry. On Breast, the best individual kernel is the cosine kernel, and both direct combinations remain within about $0.002$ error units of it. On Ionosphere, the best individual kernel is the local-scaling kernel; the matrix-only percentile rule and the direct mean are both close to it. On Telco, the best fixed individual kernel by classification error is the quadratic polynomial kernel, and the direct mean and alignment-weighted combinations remain within about $0.002$ error units of it. The matrix-only percentile rule is considerably worse in Telco, which reinforces the distinction between stable analytic combinations and abrupt supervised matrix transformations. These outcomes are sufficient for the purpose of the article: the base kernels were not selected because they are special optimal classifiers, yet their matrix combinations can still be converted into usable out-of-sample kernels.

The large variability of percentile_inout_auto on the Synthetic data deserves a separate interpretation. This rule is not an analytic kernel combination: it is an entrywise, label-aware matrix transformation based on within-class and between-class empirical percentiles. As a result, small changes in the outer training split can change the selected kernels, the percentile thresholds, and the label-dependent entries of the combined matrix. The spectral/KRR continuation then has to extrapolate coordinate traces of an abrupt, sample-dependent geometry, while test labels are not available. This explains why the standard deviation of this method is much larger than that of mean_auto and alignment_auto. The result should therefore be read as a stability diagnostic for abrupt supervised matrix-only rules, rather than as a failure of direct kernel combination.

The fact that the best individual kernel changes across data sets is also expected. The Synthetic data favor a Gaussian scale adapted to a smooth curved geometry; Breast Cancer favors an angular/cosine similarity on ordinal biomedical attributes; and Telco favors a low-degree polynomial interaction on a heterogeneous tabular encoding. Thus, kernel choice does affect classification performance. However, the direct combinations remain close to the best individual choices, suggesting that stable combinations can reduce the need to commit to one manually selected kernel while preserving a deployable out-of-sample formula.

5.2. Telco Customer Churn

The Telco run consists of 30 independent stratified repetitions with 1400 training and 600 test customers per repetition. Each split uses the same 20-kernel vector dictionary and the same strict inner-CV protocol as the other vector benchmarks.

Table 4. Selected individual-kernel results on Telco over 30 repetitions. The rows show the lowest-error kernel, the best RBF scale, the highest balanced-accuracy kernel, two alignment diagnostics, and the main failure mode.

Kernel	Family	Role	Error	Balanced Accuracy	Alignment	Support Vectors
POLY2	polynomial	lowest mean error	$0.2045\pm 0.0154$	$0.6959\pm 0.0226$	$0.0924$	$672.4$
RBF10	RBF quantile	best RBF scale	$0.2050\pm 0.0145$	$0.6975\pm 0.0222$	$0.1014$	$689.4$
LIN	linear	highest balanced accuracy	$0.2071\pm 0.0150$	$0.7066\pm 0.0222$	$0.0948$	$662.6$
COS	cosine	high alignment reference	$0.2112\pm 0.0166$	$0.6958\pm 0.0242$	$0.1183$	$651.1$
PROTO	prototype	largest alignment	$0.2150\pm 0.0143$	$0.6714\pm 0.0250$	$0.1672$	$718.8$
WRBF	weighted RBF	failure mode	$0.3419\pm 0.1741$	$0.5009\pm 0.0036$	$-0.1408$	$0.7$

reports the most informative single-kernel rows. The best fixed individual kernel by mean classification error is POLY2, with error $0.2045\pm 0.0154$. The best balanced accuracy among the fixed individual kernels is obtained by LIN, with balanced accuracy $0.7066\pm 0.0222$, although its mean error is slightly larger. The prototype kernel has the largest alignment with the training label kernel, but it does not produce the best held-out classifier; this is a useful reminder that alignment is a screening statistic rather than a complete substitute for validation. The supervised weighted RBF is a clear failure mode on this encoding, with balanced accuracy essentially at chance.

The discrepancy between alignment and held-out accuracy is expected on this heterogeneous table. Centered alignment measures agreement with the ideal label kernel on the training block; it does not measure the SVM margin, the stability of local neighborhoods, or how well a similarity can generalize to unseen customers. The prototype kernel obtains the largest alignment because it summarizes customers through class-prototype similarities estimated on the training data, but this compression can remove within-class heterogeneity that is useful for prediction. Numerically, PROTO has the largest alignment ($0.1672$) but a worse error ($0.2150\pm 0.0143$) than POLY2 ($0.2045\pm 0.0154$). The WRBF row is even more informative: its negative alignment ($-0.1408$), near-chance balanced accuracy ($0.5009\pm 0.0036$), and nearly degenerate support-vector count indicate that the supervised diagonal RBF metric is poorly matched to the mixed one-hot and standardized Telco encoding. The automatic combination strategy is therefore not based on alignment alone: alignment is used only to rank candidate kernels, while the number of retained kernels and the SVM penalty are selected by inner cross-validation on the training split.

Table 5. Telco combination results over 30 repetitions. Direct combinations have explicit train–test blocks; matrix-only rules use the spectral/KRR continuation and no test labels.

Combination	Method	${\mathit{m}}_0$	C	${\Delta }_{\mathbf{PSD}}$	Inner CV Error	Test Error	Balanced Accuracy
mean_auto	direct	$7.50$	$2.40$	$0.0000$	$0.2016\pm 0.0090$	$0.2061\pm 0.0154$	$0.6909\pm 0.0218$
alignment_auto	direct	$7.67$	$2.30$	$0.0000$	$0.2018\pm 0.0090$	$0.2063\pm 0.0158$	$0.6900\pm 0.0229$
percentile_inout_auto	spectral/KRR	$10.00$	$2.50$	$0.0161$	$0.2893\pm 0.0161$	$0.3069\pm 0.0197$	$0.6321\pm 0.0251$
percentile_in_auto	spectral/KRR	$10.00$	$2.70$	$0.1316$	$0.2942\pm 0.0165$	$0.3114\pm 0.0188$	$0.6274\pm 0.0243$
pickout_auto	spectral/KRR	$10.00$	$2.30$	$0.1183$	$0.2997\pm 0.0162$	$0.3194\pm 0.0197$	$0.6241\pm 0.0246$
percentile_out_auto	spectral/KRR	$10.00$	$2.40$	$0.0417$	$0.3044\pm 0.0156$	$0.3264\pm 0.0196$	$0.6212\pm 0.0239$

gives the corresponding combination results. The two direct combinations are stable and almost indistinguishable from each other: mean_auto gives error $0.2061\pm 0.0154$, and alignment_auto gives $0.2063\pm 0.0158$. Their inner-CV errors are about $0.202$, so the train–test discrepancy is small. In contrast, all matrix-only supervised rules are substantially worse after spectral/KRR continuation. The best of them is percentile_inout_auto, but its mean error is $0.3069\pm 0.0197$ and its balanced accuracy is $0.6321\pm 0.0251$. Thus, Telco supports the same conservative conclusion as the other experiments: smooth averages are deployable and stable, whereas label-aware matrix-only operations produce a harder out-of-sample extension problem.

The Telco gap is also linked to the heterogeneous tabular nature of the data. The predictors combine demographic variables, categorical service indicators, contract and payment information, tenure, and billing amounts. Direct convex combinations preserve the smooth train–test structure of the base kernels and the block-level similarities induced by these variables. By contrast, matrix-only supervised rules impose an additional within-class/between-class deformation on the training matrix. In a heterogeneous table, this deformation can overwrite useful local similarities and create a coarse label-driven geometry that is not well aligned with the original covariate structure. The diagnostic in

Table 6. Training-matrix criterion versus strict out-of-sample performance over 30 repetitions. The test block for matrix-only rules is obtained by reconstruction and does not use test labels.

Data Set	Combination	Inner CV Error	Reconstructed/Direct Test Error	Absolute Gap
Synthetic	mean_auto	$0.0630\pm 0.0110$	$0.0793\pm 0.0227$	$0.0163$
Synthetic	percentile_inout_auto	$0.0844\pm 0.0226$	$0.1404\pm 0.1198$	$0.0561$
Breast	mean_auto	$0.0241\pm 0.0041$	$0.0311\pm 0.0108$	$0.0069$
Breast	percentile_inout_auto	$0.0278\pm 0.0055$	$0.0358\pm 0.0124$	$0.0080$
Ionosphere	mean_auto	$0.0431\pm 0.0066$	$0.0519\pm 0.0171$	$0.0088$
Ionosphere	percentile_in_auto	$0.0463\pm 0.0065$	$0.0487\pm 0.0166$	$0.0024$
Telco	mean_auto	$0.2016\pm 0.0090$	$0.2061\pm 0.0154$	$0.0044$
Telco	percentile_inout_auto	$0.2893\pm 0.0161$	$0.3069\pm 0.0197$	$0.0177$

shows that the train–test gap is moderate but the absolute error is already high, indicating that the main limitation in Telco is the induced supervised training geometry itself, not only the continuation step.

5.3. Training-Matrix Behavior Versus Out-of-Sample Behavior

Table 6 gives the most direct diagnostic for the matrix-to-function claim. The inner-CV column is measured using only the training matrix, that is, the points on which the matrix-to-function conversion is built. The test column is measured after extending the same construction to new points. The absolute gap is small for the direct mean on all four vector data sets. For the best matrix-only rule it is also small on Breast and Ionosphere, moderate on Telco, and large on Synthetic. Telco is especially informative because the best matrix-only rule has a moderate train–test gap but poor classification performance, so its limitation is the learned supervised matrix itself rather than only the continuation step.

This table is the main experimental evidence for the functionality of the extension. The relevant comparison is not whether the test error is the smallest possible among all classifiers, but whether the classifier obtained after extending the matrix behaves like the classifier assessed on the matrix used to build that extension. The direct mean passes this diagnostic robustly, including on Telco. The matrix-only percentile rules pass it most clearly on Breast and Ionosphere. On Telco the gap is not large, but the absolute error remains high, showing that the supervised matrix transformation did not produce a competitive geometry for this heterogeneous table. Synthetic shows a complementary negative case, where an abrupt supervised transformation can be hard to continue faithfully.

Quantitatively, the average absolute train–test gap of the direct mean over the four vector data sets is $0.0091$, with a maximum of $0.0163$ on Synthetic. The corresponding average for the selected matrix-only supervised rows is $0.0211$, more than twice as large, and the largest vector-data gap is $0.0561$ for Synthetic percentile_inout_auto. Including the relational stress test makes the contrast sharper: Chickenpieces mean_auto has a gap of $0.0115$, whereas pickout_auto has a gap of $0.3141$. This is the most compact numerical summary of the robustness pattern: smooth direct continuation is stable across data types, while abrupt supervised matrix-only continuation is data-dependent and can become unstable.

The two negative cases in Table 6 should therefore be distinguished. In Synthetic, percentile_inout_auto has both a larger train–test gap and a much larger across-split standard deviation, which points to instability of the abrupt label-aware continuation. In Telco, by contrast, the train–test gap is moderate, but both the inner-CV and test errors are high. This indicates that the matrix-only supervised rule has already produced a less competitive training geometry. These two behaviors support the same methodological conclusion: the reconstruction layer can make matrix-level rules deployable, but it cannot guarantee that every label-aware training matrix defines a stable and useful out-of-sample geometry.

5.4. Numerical Reconstruction Diagnostics

Table 7. Numerical diagnostics for selected direct and matrix-only rules. Vector-data values are averaged over 30 repetitions; Chickenpieces values are averaged over 10 repeated stratified splits. $m_0$ and C are inner-CV selections averaged over repetitions.

Data Set	Rule	Method	${\mathit{m}}_0$	C	${\Delta }_{\mathbf{PSD}}$	Inner CV Error	Test Error/Gap
Synthetic	mean_auto	direct	$6.17$	$8.77$	$0.0000$	$0.0630\pm 0.0110$	$0.0793\pm 0.0227$/$0.0163$
Synthetic	percentile_inout_auto	spectral/KRR	$4.77$	$6.71$	$0.0309$	$0.0844\pm 0.0226$	$0.1404\pm 0.1198$/$0.0561$
Breast	mean_auto	direct	$5.00$	$2.60$	$0.0000$	$0.0241\pm 0.0041$	$0.0311\pm 0.0108$/$0.0069$
Breast	percentile_inout_auto	spectral/KRR	$8.20$	$3.49$	$0.0104$	$0.0278\pm 0.0055$	$0.0358\pm 0.0124$/$0.0080$
Ionosphere	mean_auto	direct	$6.47$	$7.50$	$0.0000$	$0.0431\pm 0.0066$	$0.0519\pm 0.0171$/$0.0088$
Ionosphere	percentile_in_auto	spectral/KRR	$7.23$	$4.38$	$0.0204$	$0.0463\pm 0.0065$	$0.0487\pm 0.0166$/$0.0024$
Telco	mean_auto	direct	$7.50$	$2.40$	$0.0000$	$0.2016\pm 0.0090$	$0.2061\pm 0.0154$/$0.0044$
Telco	percentile_inout_auto	spectral/KRR	$10.00$	$2.50$	$0.0161$	$0.2893\pm 0.0161$	$0.3069\pm 0.0197$/$0.0177$
Chickenpieces	mean_auto	direct	$8.90$	$6.60$	$0.0000$	$0.0317\pm 0.0046$	$0.0433\pm 0.0144$/$0.0115$
Chickenpieces	pickout_auto	spectral/KRR	$12.50$	$4.75$	$0.0169$	$0.0403\pm 0.0085$	$0.3545\pm 0.2666$/$0.3141$

records the numerical quantities monitored during the same runs. For analytic direct means, the PSD perturbation is zero and the main diagnostic is the inner-CV/test gap. For matrix-only rules, there is generally no oracle block $K^{*}(S,Z)$, so the reported diagnostics are the selected $m_0$, selected SVM penalty, PSD perturbation, inner-CV error, test error, and their absolute gap. The controlled oracle diagnostic in

Table 8. Oracle reconstruction diagnostic on the controlled Synthetic experiment. The relative Frobenius error compares the reconstructed train–test block with the oracle block over 100 repetitions.

Rule	Relative Frobenius Error	Functional MSE	${\Delta }_{\mathbf{PSD}}$	Interpretation
sum	$4.13\times {10}^{-6}\pm 9.41\times {10}^{-6}$	$2.20\times {10}^{-10}$	$2.18\times {10}^{-15}$	smooth analytic rule
mean	$4.13\times {10}^{-6}\pm 9.41\times {10}^{-6}$	$5.51\times {10}^{-11}$	$2.18\times {10}^{-15}$	smooth analytic rule
pickout	$0.4501\pm 0.7299$	$0.4074$	$0.1078$	label-aware negative control

provides the additional Frobenius and MSE block errors in the synthetic case where an oracle block is available.

These diagnostics are chosen to avoid label leakage. For matrix-only supervised rules such as pick-out and the percentile variants, there is generally no genuine oracle block $K^{*}(S,Z)$ at prediction time, because the rule depends on labels and the labels of the new points are not available. Constructing such a block with test labels would answer a different, retrospective question and would not represent a valid deployable classifier. For this reason, Table 7 reports diagnostics that are available under the strict protocol: PSD perturbation ${\Delta }_{\mathrm{PSD}}$, selected model complexity, inner-CV error, test error, and the train–test gap. These quantities measure both the numerical effect of PSD rectification and the stability of the out-of-sample continuation without using test labels.

5.5. Oracle Reconstruction Diagnostic

The controlled Synthetic setting allows a second check. When the combined matrix comes from a known analytic rule such as a sum or a mean of base kernels, the oracle train–test block is available. Table 8 compares this oracle block with the block reconstructed from the training matrix only. Besides the relative Frobenius error, we report the functional block mean squared error

$${\mathrm{MSE}}_{\mathrm{func}}={\displaystyle \frac{1}{\left|S\right|\left|Z\right|}}\sum _{i\in S}\sum _{z_j\in Z}{\left(\widehat{K}(x_i,z_j)-K_{\mathrm{oracle}}(x_i,z_j)\right)}^2.$$

The sum and mean rows have relative Frobenius error essentially at numerical precision. Pick-out is included as a negative control: it is a sharp label-aware matrix rule and does not correspond to the same smooth analytic continuation. In that row the oracle block uses test labels only for this retrospective negative-control diagnostic; test labels are never used to train, select, or reconstruct the evaluated model.

The oracle diagnostic clarifies the scope of the proposal. When the matrix is produced by a smooth combination of evaluable kernels, the matrix-to-function conversion recovers the off-sample kernel block. When the matrix is produced by an abrupt label-aware operation, the training matrix can still be represented spectrally, but the out-of-sample continuation is a harder modeling problem.

The relative Frobenius error and the functional MSE in Table 8 are therefore reconstruction-accuracy metrics for the case in which a leakage-free oracle block is meaningful. The pick-out row is included only as a negative control to show how an abrupt label-aware matrix rule differs from a smooth analytic combination.

5.6. Chickenpieces

Chickenpieces is the most important relational experiment because the input is not a feature matrix but a family of dissimilarities between shapes  [12,39].

Table 9. Chickenpieces results over 10 repeated stratified splits. The dictionary contains 88 kernels obtained from 44 dissimilarity matrices by RBF and Laplacian transformations.

Method	Kernel/Combination	Error	Balanced Accuracy
best individual	LAP::norm29_cost60	$0.0425\pm 0.0141$	$0.9528\pm 0.0176$
direct combination	mean_auto	$0.0433\pm 0.0144$	$0.9524\pm 0.0188$
direct combination	alignment_auto	$0.0433\pm 0.0144$	$0.9524\pm 0.0188$
matrix-only	pickout_auto	$0.3545\pm 0.2666$	$0.6425\pm 0.2805$

reports the best individual kernel and the main combinations.

The best individual kernel is a Laplacian transformation of the norm29/cost60 contour dissimilarity. The mean and alignment combinations essentially recover the best individual kernel without selecting it manually: the difference in mean error is less than ${10}^{-3}$. This is the strongest evidence that the workflow is useful for relational data, where the natural modeling question is not which coordinate system to use, but which dissimilarity source or kernel transformation should be trusted.

The pick-out rule is not competitive in Chickenpieces under strict out-of-sample reconstruction. This is not a contradiction of the method; it is a diagnostic result. The training matrix created by pick-out contains abrupt label-dependent information, and the experiment shows that such geometry is harder to continue to new silhouettes without test labels.

This result should not be interpreted as a general impossibility statement for label-aware matrix combinations on relational dissimilarity data. It shows that the particular abrupt pick-out rule is not robust in the strict Chickenpieces out-of-sample setting. The direct mean and alignment combinations remain stable because they preserve the similarity structures induced by the distance-to-kernel transformations. Pick-out, on the other hand, builds a sharply label-dependent training matrix, and the labels needed to reproduce that rule are unavailable for new silhouettes. In this sense, pickout_auto functions here as a negative-control or stress-test rule. Relational data remain a natural application of the proposed matrix-to-function framework, but smoother supervised combinations are needed if one wants to exploit label information while preserving stable deployability.

A practical way to improve relational label-aware rules would be to regularize the supervised deformation itself. Examples include convex shrinkage of pick-out towards the direct mean, temperature-smoothed max–min or percentile rules, PSD-constrained supervised weights, graph smoothing of the continued eigentraces, and cross-validation of the continuation ridge or retained spectral rank with a criterion that penalizes large train–test gaps. These modifications would preserve the relational input format while making the label-dependent geometry less discontinuous.

5.7. Benchmarking Perspective Relative to Alternative Extension Routes

The experiments provide quantitative benchmarking for the matrix-to-function step, but not a full leaderboard against all possible kernel classifiers. This distinction is important. Existing out-of-sample methods such as Nyström approximations or random features assume that the kernel section to the new point is known or approximable from a known analytic kernel. For matrix-only supervised rules, that section is precisely unavailable without using the unknown test label. The leakage-free comparisons that are meaningful in this setting are therefore: (i) direct analytic combinations versus reconstructed matrix-only rules in held-out classification; (ii) training-matrix inner-CV versus held-out test performance; and (iii) oracle train–test block error in the controlled Synthetic case where an analytic smooth rule is available.

Table 10. Leakage-free benchmarking perspective for the out-of-sample extension problem. The comparisons are those that are meaningful for matrix-only supervised combinations without using test labels.

Comparison Route	Applicability	Quantitative Diagnostic	Main Outcome
Direct analytic kernels	Known train–test formula	Table 3 and Table 6	Direct mean gaps $0.0044$–$0.0163$ on vector data
Synthetic oracle block	Smooth analytic rule only	Table 8	Frobenius error $4.13\times {10}^{-6}\pm 9.41\times {10}^{-6}$ for sum/mean
Spectral/KRR continuation	Matrix-only rules without test labels	Table 6 and Table 7	Stable on Breast/Ionosphere; unstable on Synthetic and Chickenpieces pick-out
Nyström-type extension	Requires $K^{*}(z,x_i)$ or landmarks from a known kernel	Conceptual baseline	Not leakage-free for pick-out or percentile rules without an oracle block
Fusion Kernel route	Uses eigenspaces of base kernels	Appendix C	Complementary route; systematic runtime benchmark left for future work

summarizes these comparisons. The key numerical message is that the proposed continuation is essentially exact for smooth analytic matrices in the Synthetic oracle diagnostic, while classification robustness depends on the regularity of the matrix geometry being continued.

5.8. Additional Hybrid Score Experiment

We also ran the hybrid selector based on a convex combination of CV-rank and alignment-rank. These runs used only five repetitions and are therefore treated as auxiliary. On Breast, the best score version reached error 0.0322 ± 0.0122; on Synthetic, 0.0787 ± 0.0119; and on Ionosphere, 0.0566 ± 0.0094. These values do not improve the principal direct means in Table 3. The experiment was useful as a diagnostic, but it will not be retained as a main method until it is re-run with the same 30-repetition protocol.

6. Discussion

The final experiments lead to a deliberately conservative interpretation.

First, the experiments support the central matrix-to-function claim. For the stable mean combination, the classifier evaluated on held-out points behaves like the classifier assessed on the training matrix used to construct the extension. The 30-repetition train–test gaps are small in Synthetic, Breast, Ionosphere, and Telco for the direct mean. The oracle Synthetic diagnostic is even stronger: when the training matrix is generated by a smooth sum or mean of base kernels, the reconstructed train–test block agrees with the oracle block up to numerical precision.

Second, the results should not be read as an optimized classification benchmark. The kernel dictionary was deliberately inherited from earlier kernel-combination work and was not redesigned to make these data sets easy. It contains simple baselines, multiple Gaussian scales, class-scale variants, local scaling, prototype similarities, and a supervised weighted RBF. The fact that the mean or alignment combinations remain close to the best individual kernel is therefore useful evidence: the conversion works even with a non-specialized historical battery of kernels.

Third, alignment is useful as a diagnostic but not as a universal replacement for model selection [10,11]. In Breast, alignment identifies the cosine kernel, which is also the best individual kernel. In Ionosphere and Telco, however, prototype-type kernels have high alignment but poorer classification performance than simpler alternatives. This confirms that alignment should be treated as a screening measure rather than as a complete performance criterion.

Fourth, the label-aware rules reveal the distinction between a discriminative training matrix and a deployable out-of-sample kernel [6,18]. Percentile rules extend well in Breast and Ionosphere. In Telco, the best percentile rule has a moderate train–test gap but is still far less accurate than the direct means, so its limitation is the induced matrix geometry. The Synthetic and Chickenpieces results show an even stronger failure mode: abrupt supervised operations can be hard to continue faithfully. Pick-out is therefore best interpreted as a stress test or negative control, not as the empirical claim of the paper.

Fifth, Telco strengthens the heterogeneous-tabular side of the motivation. The customer table mixes categorical service fields, account variables, and billing amounts. The direct mean remains within $0.002$ error units of the best fixed individual kernel, which indicates that the broad dictionary can be aggregated without losing deployability. Sixth, Chickenpieces strengthens the relational side of the motivation. It shows that the matrix-to-function problem is not merely a tabular-kernel issue. In a relational problem with 44 dissimilarity matrices, the proposed protocol can compare and aggregate many kernels, identify a useful transformation family, and evaluate the resulting kernel out of sample.

Seventh, the benchmarking claim should be interpreted in the appropriate problem class. The state-of-the-art methods most often associated with out-of-sample kernels, such as Nyström approximations and random features, are designed for situations where the kernel section or an analytic base kernel is available. They do not directly solve the deployment of a final label-aware matrix rule whose entries are defined only on the training pairs. For that reason, the strongest quantitative evidence in this paper is not a generic runtime leaderboard but the combination of held-out classification baselines, train–test stability diagnostics, and the Synthetic oracle block comparison. Scaling the same reconstruction to much larger problems will require low-rank or randomized eigensolvers and faster coordinate evaluation, which we regard as an implementation direction rather than a change in the mathematical problem.

The unstable and negative-control cases discussed above clarify the scope of the framework. Kernel choice matters, because different data geometries favor different base kernels; nevertheless, stable direct combinations can aggregate a broad dictionary while keeping an explicit train–test formula. Reconstruction diagnostics also need to be interpreted according to the type of combination: for smooth analytic rules, relative Frobenius error and functional MSE can be computed against an oracle block, whereas for label-aware matrix-only rules the valid deployment diagnostics are PSD perturbation, inner-CV/test gaps, and held-out classification metrics without test-label access. Finally, the poor performance of percentile_inout_auto in Synthetic and pickout_auto in Chickenpieces should be viewed as informative negative controls. They show that a discriminative training matrix is not automatically a stable deployable kernel.

The final message is therefore not that kernel combinations always outperform the best individual kernel. Rather, the contribution is a principled and strictly validated way to move from a finite kernel matrix or a family of kernel matrices to an out-of-sample kernel evaluator. When the combination is smooth and geometrically stable, as in direct sums and means, the evaluator preserves the training-matrix behavior on new points. When the combination is highly label-driven, as in pick-out, the method exposes the difficulty of extending that matrix faithfully.

7. Conclusions

This paper studies inductive finite-rank extensions of out-of-sample kernel functions from empirical kernel matrices, with particular emphasis on matrices produced by kernel-combination rules. The proposed approach uses spectral decompositions and data-dependent RKHS bases to convert a matrix defined on a training sample into a controlled kernel evaluator for new points.

The experiments support three conclusions. First, the conversion from matrix to function is reliable for smooth, stable combinations such as sums and means. Quantitatively, the direct mean is essentially tied with the best individual kernel on Synthetic ($0.0793\pm 0.0227$ versus $0.0809\pm 0.0231$ error), remains within about $0.002$ error units of the best individual kernel on Telco ($0.2061\pm 0.0154$ versus $0.2045\pm 0.0154$), and matches the best relational Chickenpieces kernel within ${10}^{-3}$ error units. In the controlled Synthetic oracle diagnostic, smooth sum/mean reconstruction gives relative Frobenius error $4.13\times {10}^{-6}\pm 9.41\times {10}^{-6}$ and functional MSE at numerical scale. Second, the base kernels do not need to be specially engineered for optimal classification in order for the reconstruction idea to work; the broad historical dictionary used in earlier combination experiments is sufficient to obtain stable deployable kernels. Third, purely supervised matrix-only combinations such as pick-out and some percentile rules are harder to deploy: Synthetic percentile_inout_auto has error $0.1404\pm 0.1198$, the best Telco matrix-only supervised rule has error $0.3069\pm 0.0197$, and Chickenpieces pickout_auto has error $0.3545\pm 0.2666$. This limitation is not a contradiction of the reconstruction method; rather, it identifies the boundary between smooth or moderately regular matrix combinations, which are naturally extendable, and abrupt supervised transformations, which require additional regularization or smoother design before they can be recommended as deployable kernels.

The evidential basis for these conclusions is now made explicit. Table 3 and Table 6 support the stability claim for direct combinations through held-out classification and train–test gap diagnostics. Table 8 supports the reconstruction-accuracy claim for smooth analytic rules through relative Frobenius error and functional MSE. Table 5 and Table 9 delimit the scope of label-aware matrix-only rules by showing that abrupt supervised transformations may perform poorly even when the matrix can be represented spectrally. Table 10 clarifies the relationship with alternative extension routes and explains why methods requiring an analytic test kernel section are not directly applicable to label-aware matrix-only combinations without an oracle. The conclusions are therefore deliberately restricted: the paper does not claim universal superiority of kernel combinations, does not present a large-scale state-of-the-art classifier benchmark, and does not recommend abrupt supervised matrix-only rules as default deployable kernels.

Future work should focus on three directions. The first is computational: large-scale eigensolvers, landmark approximations, and faster basis-evaluation schemes are needed for larger data sets. The second is methodological: smoother supervised combination rules, possibly constrained to remain positive semidefinite and easier to extend, may retain the discrimination of label-aware rules while preserving the stability of direct averages. For relational data, this includes shrinkage of abrupt pick-out matrices towards smooth means, temperature-smoothed max–min rules, and graph-regularized eigentrace continuation. The third direction is comparative: the present reconstruction should be combined and benchmarked with PSD-embedding extensions [8], subspace-fusion rules for heterogeneous proximities [9], polar-decomposition PSD corrections [31], and scalable Nyström or randomized low-rank variants when an analytic kernel section is available.