Application of Non-Sparse Manifold Regularized Multiple Kernel Classifier

Abstract

Non-sparse multiple kernel learning is efficient but not directly able to be applied in a semi-supervised scenario; therefore, we extend it to semi-supervised learning by using a manifold regularization. The manifold regularization is based on a graph constructed on all the data samples including the labeled and the unlabeled, and forces the regularized classifier smooth along the graph. In this study, we propose the manifold regularized p-norm multiple kernels model and provide the solutions with proofs. The risk bound is briefly introduced based on the local Rademacher complexity. Experiments on several datasets and comparisons with several methods show that the efficiency of the proposed model to be used in semi-supervised scenario.

1. Introduction

Multiple kernel learning (MKL) has attracted many researchers as it is flexible and adaptable in different areas [1,2,3,4]; there are important works about algorithms designs and theoretical proofs [5,6,7,8]. Kloft et al. proposed the p-norm MKL to extend traditional 1-norm to p-norm, which could describe different properties of the same object in various complementary views [9]. However, p-norm MKL is used in only supervised situations. We adopt the manifold regularization (MR) technique [10] to extend p-norm MKL to semi-supervised leaning (SSL). The proposed model is based on manifold assumption in SSL, which states that if two samples, x1 and x2, are close, then so should be the corresponding label, y1 and y2. This characterization is captured by a weighted graph that estimates the local relations of each sample.

The advantage of non-sparse Multiple Kernel Learning (MKL) stems from its balanced allocation of weights across multiple kernels, preserving the contributions of all kernels. In semi-supervised scenarios, the geometric structure of unlabeled data is typically constrained through manifold regularization (MR), which better aligns with the regularization requirements for unlabeled data in SSL. The MR technique promotes the SSL in various directions and shows its superiority, such as semi-supervised classification [11], clustering [12], feature selection [13], image processing [14] and so on. Combing MKL with MR has been demonstrated to be effective and applicable in addressing machine learning problems characterized by complex nonlinear data conditions [15].

In contrast, sparse MKL may neglect critical features associated with manifold structures due to its excessive selection of a limited number of kernels [16]. For instance, when different kernels capture complementary local neighborhood information, non-sparse weighting preserves such information, thereby enabling more effective utilization of manifold regularization to constrain classifier smoothness. Furthermore, the flexibility of p-norm MKL (controlling sparsity through adjustment of the p-value) allows adaptation to diverse data distributions [17]. While existing studies have explored p-norm MKL in supervised learning [17,18], its potential in SSL remains underexplored. It motivates the proposal of this work.

This work presents the first semi-supervised classification framework integrating p-norm MKL with manifold regularization (MR). Compared with sparse MKL, our non-sparse weighting mechanism enhances the efficiency of manifold regularization in leveraging unlabeled data by preserving the global complementarity of multiple kernels. Specifically, manifold regularization constrains classifier smoothness based on the full-sample graph structure, while non-sparse MKL provides enriched feature representations through multi-kernel fusion. Their synergistic interaction improves generalization performance in semi-supervised settings. The experimental results demonstrate that this integration achieves superior performance compared to conventional sparse MKL approaches across multiple benchmark datasets.

In this study, we firstly compute the weighted graph on the whole data samples and obtain the Laplacian matrix that can describe the relationships of the samples. By using Laplacian matrix and manifold assumption, we add the MR to the p-norm multiple kernel classifier; further, we provide the proofs of solutions and the risk bound analysis. Experiments with other methods are conducted to show the efficiency of the proposed model.

The paper is organized as follows. In Section 2, the MR method is illustrated and p-norm multiple kernel classifier is reviewed. In Section 3, we provide the solutions and the corresponding proofs and give the procedure of algorithm and elementary risk bound analysis. In Section 4, we show the results of the experiments. Finally, conclusions of the paper are in Section 5.

2. Related Works

For notation consistency, assume that we are given ℓ labeled data samples {x_i,y_i}, i = 1,…, ℓ and u unlabeled data samples {x_i}, i = 1,…, u. In each sample, x_i ∈ $𝒳$ is the input space and y_i ∈ R is the label.

2.1. Manifold Regularization

The manifold regularization is based on a graph that is built on the whole data samples {x_i}, i = 1,…, n. We define the graph as G = (V,E), where V is the vertice set, each v_i ∈ V corresponds to the sample x_i and E is the edge set; each e_ij ∈ V links the adjacent vertices. Suppose x_i and x_j are adjacent; then there exist an edge e_ij between x_i and x_j. On each edge, a weight is assigned based on the degree of distance. First, to judge two samples that are adjacent is based on kNN method; that is to say if x_i and x_j are adjacent, then x_i is in the k-nearest neighbor of x_j. Next, to provide the proper weights on the edges is based on Gaussian function; suppose an edge e_ij links x_i and x_j; then the weight w_ij is defined in the following way:

$$w_{ij}=\exp \left(-{‖x_i-x_j‖}^2/{\sigma }^2\right)$$

(1)

where σ is the width parameter of Gaussian function. From Equation (1), we know that the larger weight corresponds to the closer relationship of x_i and x_j. After all the samples are computed to construct the graph G, we obtain the weight matrix W.

The manifold regularization is a regularizer added to a classifier to make it smooth along the graph; in this way, the classifier can deal with those unlabeled data samples. Now, suppose a classifier is f, f: $𝒳$→R. We define the manifold regularization in the following way:

$${‖f‖}_I^2={\displaystyle \frac{1}{2}}{{\displaystyle {\sum }_{i,j=1}^n{w}_{ij}\left(f(x_i)-f(x_j)\right)}}^2=f^{T}Lf$$

(2)

where L = D-W is called the Laplacian graph and D is a diagonal matrix of which the element d_ij = ∑ _iw_ij, and $f={\left(f(x_1),\cdots ,f(x_n)\right)}^T$. The Equation (2) can force the classifier f to keep the local relationship of samples; further, it complies with the manifold assumption in SSL.

2.2. P-Norm Multiple Kernel Classifier

In kernel learning methods, a function ϕ(x) is used to map the input space $𝒳$ to feature space $\mathscr{H}$, where $\mathscr{H}$ is called reproducing kernel Hilbert space (RKHS) associated with a kernel function k(x,x’) = <ϕ(x), ϕ(x’)>. The operator <,> is the inner product in $\mathscr{H}$. The RKHS has the following important properties:

$$f(\cdot )=〈f(x),k(x,\cdot )〉,k(x,x^{\prime })=〈k(x,\cdot ),k(x^{\prime },\cdot )〉$$

(3)

In MKL, suppose there exist M mappings, then ϕ₁,…, ϕ_M correspond to M RKHS $\mathscr{H}$₁,…, $\mathscr{H}$_M. For each $\mathscr{H}$_m, a kernel function k_m is associated. A unified space is constructed by using $\mathscr{H}$ = ⊕_m$\mathscr{H}$_m with weight d = {d₁,…, d_M}. According to [19], the space $\mathscr{H}$ has the following kernel:

$$k(x,x^{\prime })={\displaystyle {\sum }_{m=1}^M{d}_m{k}_m}(x,x^{\prime }),$$

(4)

To avoid an over-fitting problem, the weight d needs to be constrained. The most often used constraint is the 1-norm:

$${\displaystyle {\sum }_{m=1}^M{d}_m}=1,d_m\ge 0,$$

(5)

As previous research works [20] show, 1-norm often leads to the sparsity of d, while Ref. [21] proposed that the sparsity could miss information especially for the low correlations among different kernels K = {k₁,…, k_M} or inadequate for the complementary data sources. Therefore, p-norm MKL suggests that the constraint on d adopt the following:

$$\begin{array}{l}{‖d‖}_p^2\le 1\\ d_m\ge 0,\forall m\end{array},$$

(6)

when p > 1, the solution is not sparse, which keeps the whole information brought in by multiple kernels.

In each $\mathscr{H}$_m, we define the classifier in the following way:

$$f_m=〈{\omega }_m,{\varphi }_m(x)〉,$$

(7)

It is natural to have the sum of classifiers in $\mathscr{H}$:

$$\begin{array}{l}f={\displaystyle {\sum }_{m=1}^M{f}_m}\\ f=〈\omega ,\varphi (x)〉+b\end{array},$$

(8)

where b is the additional bias to the final f. According to the works of Bach and Kloft, we have the following classifier of p-norm multiple kernels:

$$\begin{array}{l}F=\underset{d_m,{\omega }_m,b,t}{\min }{\displaystyle \frac{1}{2}}{\displaystyle {\sum }_{m=1}^M\frac{{‖{\omega }_m‖}_2^2}{d_m}}+C{\displaystyle {\sum }_{i=1}^{\mathcal{l}}v\left(t_i,y_i\right)}\\ s.t.\left({\displaystyle {\sum }_{m=1}^M〈{\omega }_m,{\varphi }_m(x_i)〉}+b\right)=t_i,i=1,\cdots ,\mathcal{l}\\ \hspace{1em}\hspace{0.17em}\hspace{0.17em}{‖d‖}_p^2\le 1,d_m\ge 0\end{array},$$

(9)

where function v is a convex loss function. The duality optimization analysis is usually used to obtain the solution of problem (9). In this study, we focus on the extension of (9) to the semi-supervised classification by using the manifold regularization, and the detail proposal and theoretical analysis are in the following sections.

3. Proposed Classifier

3.1. Manifold Regularized P-Norm Multiple Kernel Classifier

Section 2.1 gives us a method to use manifold regularizer; therefore in space $\mathscr{H}$, we have the following additional manifold constraint:

$$\begin{array}{l}{‖f‖}_I^2=f^{T}Lf\\ f={\displaystyle {\sum }_{m=1}^M〈{\omega }_m,{\varphi }_m(\mathit{x})〉}\end{array},$$

(10)

and

$${\varphi }_m(\mathit{x})=\left({\varphi }_m(x_1),\cdots ,{\varphi }_m(x_n)\right),$$

(11)

Now, as for the labeled and unlabeled data samples {x_i}, i = 1,…, n, we propose the following manifold regularized non-sparse multiple kernel classifier:

$$\begin{array}{l}F=\underset{d_m,{\omega }_m,b,t}{\min }C{\displaystyle {\sum }_{i=1}^{\mathcal{l}}v\left(t_i,y_i\right)+}{\displaystyle \frac{r_A}{2}}{\displaystyle {\sum }_{m=1}^M\frac{{‖{\omega }_m‖}_2^2}{d_m}}+{\displaystyle \frac{r_I}{2}}{f}^{T}Lf\\ s.t.\left({\displaystyle {\sum }_{m=1}^M〈{\omega }_m,{\varphi }_m(x)〉}+b\right)=t_i,i\in \left\{1,\cdots ,\mathcal{l}\right\}\\ \hspace{1em}\hspace{0.17em}\hspace{0.17em}{‖d‖}_p^2\le 1,d_m\ge 0\end{array},$$

(12)

where r_A and r_I are regularization parameters, and matrix L is the Laplacian matrix obtained by all the samples. The optimal $f^{*}$ is just the solution of problem (12), and when it is obtained, the term $f^{T}Lf$ becomes constant. We specify the loss function v to be hinge loss, that is v(t,y) = max(0,1-ty). In this way, we provide two theorems and proofs.

Theorem 1. 

The optimal $f^{*}$ can be formed in the following:

$$f^{*}={\displaystyle {\sum }_{m=1}^M〈{\omega }_m^{*},{\varphi }_m〉={\left(r_{A}I+r_{I}LK\right)}^{-1}K{J}^T}Y\alpha$$

(13)

, where

$$K={\displaystyle {\sum }_{m=1}^M{d}_m}{K}_m,K_{m,ij}=k_m(x_i,x_j),$$

(14)

and α ≥ 0 and I is the identity matrix and Y is a diagonal matrix with element Y_ii = y_i and J = [I 0] with the size ℓ × (ℓ + u).

Theorem 2. 

The update of weight d is computed as follows:

$$d_m={\displaystyle \frac{{‖{\omega }_m‖}_2^{2/(p+1)}}{{\left({\displaystyle {\sum }_{m=1}^M{‖{\omega }_m‖}_2^{2p/(p+1)}}\right)}^{1/p}}}\forall m,$$

(15)

With these two theorems, we can compute the optimal classifier $f^{*}$ and use it to infer the labels of the whole input space. In the next section, the proofs are provided.

3.2. Proofs of the Theorems

As we have specified the loss function to be the hinge loss function, we can establish the following Lagrangian formula of (12):

$$\begin{array}{l}\mathcal{L}={\displaystyle \frac{r_A}{2}}{\displaystyle {\sum }_{m=1}^M\frac{{‖{\omega }_m‖}_2^2}{d_m}}+{\displaystyle \frac{r_I}{2}}{f}^{T}Lf-{\displaystyle {\sum }_{i=1}^{\mathcal{l}}{\delta }_i}{\xi }_i+{\displaystyle \frac{\lambda }{2}}\left({‖\mathit{d}‖}_p^2-1\right)+\\ {\displaystyle {\sum }_{i=1}^{\mathcal{l}}{\alpha }_i}\left(1-{\xi }_i-y_i({\displaystyle {\sum }_{m=1}^M〈{\omega }_m,{\varphi }_m(x_i)〉}+b)\right)+C{\displaystyle {\sum }_{i=1}^{\mathcal{l}}{\xi }_i}\end{array},$$

(16)

where α,δ and λ are Lagrangian multipliers. Here, we firstly assume the non-negative of d is always met.

Proof of Theorem 1. 

Set the derivative of (16) with respect to ω_m to zero:

$${\displaystyle \frac{\mathsf{\partial }\mathcal{L}}{\mathsf{\partial }{\omega }_m}}=r_A{\displaystyle \frac{{\omega }_m}{d_m}}+r_I〈{\omega }_m,L{K}_m〉-Y{\varphi }_m^{\mathcal{l}}\alpha =0,$$

(17)

where ${\varphi }_m^{\mathcal{l}}=\left({\varphi }_m(x_1),\cdots ,{\varphi }_m(x_{\mathcal{l}})\right)$. We multiply ${\varphi }_m^n$ to (17), and by using K_m = <ϕ_m, ϕ_m^T > , the following formula is obtained:

$$r_A〈{\omega }_m,{\varphi }_m〉+r_I〈{\omega }_m,{\varphi }_m〉\cdot L{d}_m{K}_m-d_m{K}_m{J}^{T}Y\alpha =0,$$

(18)

By using f_m = <ω_m, ϕ_m> and the sum of m from 1 to M, we obtain

$$r_{A}f+r_{I}f\cdot LK-K{J}^{T}Y\alpha =0,$$

(19)

By Equation (19), we can obtain the solution of theoretical optimal f, and that is just the conclusion of Theorem 1. ☐

Proof of Theorem 2. 

Set the derivative of (16) with respect to d_m to zero:

$$-{\displaystyle \frac{r_A}{2}}{\displaystyle \frac{{‖{\omega }_m‖}_2^2}{d_m^2}}+\lambda d_m^{p-1}{‖\mathit{d}‖}_p^{2-p}=0,$$

(20)

The constraint ∥d∥_p ≤ 1 is at the upper bound, that is, ∥d∥_p = 1 holds for an optimal d. Hence, the Equation (20) translates into the following condition:

$$d_m=\delta {‖{\omega }_m‖}_2^{2/(p+1)}\forall m,$$

(21)

where δ is irrelevant to ω_m. By complying with ∥d∥_p = 1, we can obtain that δ has the form:

$$\delta ={\left({\displaystyle {\sum }_m{‖{\omega }_m‖}_2^{2p/(p+1)}}\right)}^{-1/p},$$

(22)

Inserting the above equation to (21), we obtain the conclusion of Theorem 2.

We will summarize the procedure of the algorithm of the proposed manifold regularized p-norm multiple kernel classifier. ☐

3.3. Procedure of the Algorithm

We set the derivatives of (16) with respect to all the decision variables to zero, and take them to (12). Before that, we introduce the representer theorem [22], which is used in this mathematical transformation, i.e., any bounded f in RKHS that minimizes the regularized risk functional admits a representation of the form

$$f\left(\cdot \right)={\displaystyle {\sum }_i{\beta }_{i}K}\left(\cdot ,x\right),$$

(23)

By taking the regularizer ∥ω_m∥ to ∥f_m∥ and the representer theorem, we obtain the dual problem of (12):

$$\begin{array}{l}\mathcal{L}=\underset{\beta ,\eta }{\max }{{\displaystyle {\sum }_{i=1}^{\mathcal{l}}{\alpha }_i-‖{\left(\frac{1}{2}{\alpha }^{T}YJ{K}_{m}Q{J}^{T}Y\alpha \right)}_{m=1}^M‖}}_{p*}\\ s.t.{\displaystyle {\sum }_{i=1}^{\mathcal{l}}{\alpha }_i}{y}_i=0,0\le \alpha \le C\mathsf{1}\\ \hspace{1em}\hspace{0.17em}\hspace{0.17em}\beta ={\left({\gamma }_{A}I+{\gamma }_{I}LK\right)}^{-1}{J}^{T}Y\alpha \end{array},$$

(24)

where

$$Q={\left({\gamma }_A+{\gamma }_{I}LK\right)}^{-1}and{\displaystyle \frac{1}{p}}+{\displaystyle \frac{1}{p^{*}}}=1,$$

(25)

Thus, the Algorithm 1 is summarized as follows:

Algorithm 1. Algorithm of the proposed model

Step 1. Set M kernel functions {k1,…, kM}, initial d with each dm = 1/M, initial kernel matrix K = ∑dmkm, initial norm parameter p, and other parameters C, rA, rI;  Step 2. Given labeled data {xi,yi},i =1,…, ℓ, and unlabeled data {xi}, i = 1,…, u., use all the data samples to construct the adjacent graph G, and use the kNN method and Gaussian function to decide the edges of G and the weights on edges, i.e., W and the Laplacian graph L = D-W.  Step 3. Repeat.  Obtain the optimal α from (24) with a SVM solver.  Obtain the current optimal d according to Theorem 2, denoted as dt.  Re-compute the kernel matrix K = ∑dmkm with the current dt.  Update the problem formation of (24)  Until ||dt + 1-dt|| ≤ ε;  Step 4. Output the final optimal classifier $f^{*}$ according to Theorem 1.

3.4. Risk Bound Analysis

The optimal classifier f is in space $\mathscr{H}$. We know that in MKL, f = ∑_m <ω_m,ϕ_m>; therefore, the class of functions of p-norm multiple kernels can be described as

$$\mathcal{H}=\left\{f:x\mapsto 〈\omega ,\varphi (x)〉|\omega =\left({\omega }_1,\dots {\omega }_M\right),{‖\omega ‖}_{2,p}\le D\right\},$$

(26)

In MKL, global Rademacher complexity (GRC) is commonly used [23]. For data samples {x_i}, i = 1,…, n, drawn from the same distribution, the empirical GRC is defined as

$$R\left(\mathcal{H}\right)=E_{\sigma }su{p}_{f\in \mathcal{H}}〈\omega ,\frac{1}{n}{\displaystyle {\sum }_i{\sigma }_i{x}_i}〉,$$

(27)

where {σ_i} i = 1,…, n is an i.i.d family of Rademacher variables with random signs of equal possibility. In this paper, we can express (27) as

$$R_n\left(\mathcal{H}\right)={\displaystyle \frac{1}{n}}{E}_{\sigma }\left[su{p}_{f\in \mathcal{H}}{\displaystyle {\sum }_{i=1}^n{\sigma }_{i}f(x_i)}\right],$$

(28)

From (23), we can obtain

$$R_n\left(\mathcal{H}\right)={\displaystyle \frac{1}{n}}{E}_{\sigma }\left[su{p}_{d,\beta }{\sigma }^{T}K\beta \right],$$

(29)

According to the conclusion in [24], the following is admitted:

$$P\left(y\ne f(x)\right)\le P_n\left(y\ne f(x)\right)+{\displaystyle \frac{R_n\left(\mathcal{H}\right)}{2}}+\sqrt{{\displaystyle \frac{\ln \left(1/\delta \right)}{2n}}}$$

(30)

where P_n is the average output of an indicator function.

4. Experiments

The experimental datasets are in two parts: several UCI datasets [25] and Benchmark datasets [26]. We take the following methods as comparisons:

• LapSVM [10]. An SVM with manifold regularization adopts kernel of Gaussian with parameter σ in [0.1, 0.3, 0.5]. The actual parameter was selected by five-fold cross validation;
• Transductive SVM (TSVM) [27]. This model could let the classification hyper-plane traverse the low-density data area. It adopts the radial basis function kernel such as k(a,b) = exp(-r∥a-b∥²) with r = 0.1;
• Low-density separation (LDS) [28]. It forms a kernel function by using the metrics of a graph and its kernel function is the same as in TSVM;
• Harmonic function [29]. This method is actually a labelling scheme. This process is asked to be on a graph by keeping the similarities of all the data samples. The classifier has a harmonic property;
• Support vector machine (SVM). It is implemented by LIBSVM [30] and adopts the radial basis function. The parameter of the kernel in SVM could be the mean distances of the whole data samples. Only the labeled data are used to learn SVM model in training stage;
• Graph-structured MKL (GMKL) [31]. It proposed a MKL model that the pruned kernel combination is fulfilled by feedback collected from a graph, developing a novel scheme which actively chooses relevant kernels. Online, there are time-efficiency advantages.

4.1. Experimental Settings

In practice, hyper-parameter C = 100 is a common choice, and the regularization parameter r_A and r_I are both selected from [10⁻⁵,…, 10⁻²] by five-fold cross validation. The neighborhood of each node of a graph is computed by kNN method. For kNN method, the crucial parameter k is selected from the set [6,7,8] by five-fold cross-validation.

As for multiple kernels, during experiment tests for UCI datasets, we select two kinds of the candidate kernels: one comprises 10 Gaussian kernels with 10 different parameters which are chosen from the set [2⁻⁴, 2⁻³,…, 2⁵]; the other three polynomial kernels such as k(x,x’) = (1 + x∙x’)^d with different parameter d are chosen from the set [1,2,3]. During the tests for benchmark datasets, there are also two kinds of the candidate kernels: Gaussian kernels with parameters σ chosen from the set [0.01, 0.05, 0.1, 0.5, 1, 2, 5] and polynomial kernels with parameters d = [1,2,3]. All the kernel matrices are needed to be pre-processed to have a unit trace. The SVM solver refers to [30]. For norm parameter p, we select p = 1, 4/3, 2, 3, 4, 8, 10.

4.2. UCI Datasets

Among UCI data repositories, we take eight datasets. For conducting semi-supervised classification, this study dealt with each dataset of 100 randomly labeled samples by a 10-times choosing process and the rest of the samples being marked as unlabeled. By the data sampling processing, 10 groups of experimental data samples are determined, each of which has 100 labeled samples, with the rest unlabeled. The 13 kernels, which are chosen as kernel functions in methods, are asked to be computed on all variables and on each variable. Average accuracy rates are computed for the unlabeled samples, and the detailed results are shown in

Table 1. The eight UCI datasets tests on average accuracy rates (%) with 100 labeled samples.

	Heart	German	Iono	Vote	Pima	Liver	Wdbc	Vehicle
p = 1	82.1	72.0	90.5	94.9	74.8	65.3	94.0	85.4
p = 4/3	78.6	72.0	90.1	91.2	74.3	66.5	91.7	90.9
p = 2	80.8	71.7	92.0	92.4	74.2	66.4	94.9	90.2
p = 3	81.3	72.3	92.4	93.1	74.7	68.8	94.2	90.1
p = 4	81.7	73.7	91.8	92.4	75.0	66.1	92.1	87.8
p = 8	81.8	74.3	92.8	92.8	75.5	67.5	94.1	89.1
p = 10	82.0	72.9	91.2	91.2	76.1	66.2	94.2	89.0
LapSVM	81.8	71.8	84.6	91.3	80.0	68.6	85.2	82.2
TSVM	81.3	68.7	82.9	93.1	73.8	65.3	92.4	83.6
harmonic	78.8	68.1	85.4	92.1	71.5	63.9	93.9	83.2
LDS	81.4	66.3	89.1	90.1	70.0	64.7	94.4	81.7
SVM	81.4	71.2	93.0	92.6	73.1	64.7	91.7	81.2
GMKL	81.9	73.8	92.9	93.7	76.6	67.5	94.3	90.1

. The top accuracies are in bold.

In Table 1, the bold number means the highest accuracy result. From Table 1, the proposed manifold regularized p-norm MKL classifier is efficient and competent. We can observe that there are six out of eight results that best existed in the proposed method. As for german, liver, wdbc and vehicle datasets, non-sparse kernel combinations (p = 4/3, 2, 3, 8) likely preserved complementary geometric information captured by the graph Laplacian. Conversely, sparse MKL underperformed, suggesting that discarding kernel contributions degrades performance when manifold structures are critical.

4.3. Benchmark Datasets

The benchmark datasets we used are named digit1, g241c, g241n, COIL, BCI and USPS. The aim of building the benchmark datasets was to judge the effect and efficiency of the algorithms in a way that was as neutral as possible. There are 12 groups and each has 100 labeled samples and the rest unlabeled. The experimental tests were conducted on each dataset and the average accuracy rates of the unlabeled samples were reported as in

Table 2. Benchmark dataset tests on average accuracy rates (%) with 100 labeled samples.

	Digit	USPS	COIL	g241c	g241n	BCI
p = 1	94.9	91.8	88.1	77.8	74.1	60.4
p = 4/3	93.5	87.9	85.9	79.6	74.4	60.0
p = 2	94.4	86.1	85.0	79.7	74.1	58.7
p = 3	94.6	85.8	85.1	79.8	74.4	60.3
p = 4	94.3	85.7	85.1	79.8	74.5	60.3
p = 8	94.4	85.4	85.2	78.6	74.4	62.6
p = 10	94.5	85.5	85.1	77.5	74.3	60.7
LapSVM	94.8	82.1	86.2	55.4	57.6	60.7
TSVM	92.3	86.5	80.5	79.1	74.1	60.5
harmonic	92.9	91.4	82.1	57.8	58.9	53.1
LDS	93.3	75.0	68.7	71.1	75.8	58.2
SVM	92.9	85.5	79.4	74.2	72.4	56.4
GMKL	94.6	90.1	86.2	78.4	75.6	60.3

.

The thirteen kernels, which are kernel functions in methods, are asked to be computed on all variables. As the number of samples is larger than that of the eight UCI datasets, this experiment can validate the application ability of the proposed model on medium or large datasets. The detailed results are in Table 2. The top accuracies are in bold.

In Table 2, the bold number means the highest accuracy result. From Table 2, we see that the top accuracy rates mostly appear in the proposed classifier, and that there are five results that best existed in the proposed method. Further, we need to point out that the benchmark datasets are preprocessed with adding noises by relative scholars, and that the noises would affect the geometrical structure of data, so as to degrade the efficiency of manifold regularization to some extent. This indicates that moderate non-sparsity balances kernel diversity and regularization, mitigating noise effects. This point also leads us to research more about the anti-noise robust model of the proposed classifier. In addition, the variability in optimal p-values across datasets underscores the importance of tuning p based on data characteristics. In the process of experiments, the author found a trend that higher p-value may stabilize performance in noisy settings. These observations align with the theoretical advantage of p-norm flexibility in adapting to diverse data distributions.

4.4. Impact of P-Norm Choice

The choice of p-norm significantly influences the experiments by controlling the sparsity of kernel weight allocation. Smaller p-values enforce sparsity, selecting only a subset of kernels, while larger p-values distribute weights more evenly across kernels. For instance, on the german dataset (Table 1), p = 8 achieves 74.3% accuracy, outperforming p = 1 (72.0%), suggesting that non-sparse combinations better leverage complementary kernel information. Conversely, in the heart dataset, p = 1 slightly outperforms higher p-values, indicating that sparse selection may suffice when a few kernels dominate.

The trade-off between sparsity and diversity is evident in noisy benchmarks like g241c dataset (Table 2), where p = 3 or 4 achieves an accuracy, outperforming both sparse and highly non-sparse configurations. This implies that moderate p-values balance noise resilience and kernel utilization. Overall, the experiments validate that p-norm adaptation is critical: low p-values risk discarding useful kernels, while excessively high p-values may dilute discriminative power.

5. Conclusions

In this study, we proposed a manifold regularized p-norm multiple kernel classifier to be used in a semi-supervised classification scenario. The proposed model adopted a convex framework, i.e., manifold regularization, to further extend the p-norm multiple kernel classifier to semi-supervised situation. By integrating the non-sparse p-norm multiple kernel learning framework with graph-based manifold regularization, our method bridges the gap between kernel-based supervised learning and semi-supervised paradigms. The convex formulation of manifold regularization leverages both labeled and unlabeled data through a spectral graph embedding, enforcing smoothness of the classifier over the intrinsic data geometry. In a word, we combined the flexibility and capability of multiple kernels with a graph-based regularization term, which is based on graph spectrum theory, developing the local relationships of all the data samples. Theoretical guarantees were established via local Rademacher complexity analysis, demonstrating robust generalization bounds under semi-supervised conditions. From experiments, we observed that the proposed model has good generalization and can deal with many kinds of datasets.

Through studying and testing, the paper finds some limitations of the proposed method to guide future directions. The graph Laplacian construction incurs a relatively high computational complexity, limiting scalability for ultra-large datasets. While approximation techniques such as sampling could mitigate this limitation. The best performance relies on tuning and selecting the p-norm exponent, kernel combination weights and regularization coefficients. Automated hyperparameter optimization remains an open challenge for semi-supervised MKL. Future extensions could incorporate stochastic graph approximations for scalability, develop adaptive p-norm scheduling strategies or integrate contrastive learning principles to enhance low-label robustness.