Vertical Federated XGBoost with Privacy Preservation via Secure Multiparty Computation

Abstract

Gradient Boosted Decision Trees (GBDTs) are popular for their strong predictive performance. However, in domains like finance and healthcare, data are often distributed across organizations, making collaborative model training challenging due to privacy concerns. Vertical federated learning (VFL) enables such collaboration when data are split by features, but many existing methods focus on protecting raw data while exposing sensitive model information, such as gradients and Hessians—especially to the label-owning party. Techniques like Homomorphic Encryption and Secret Sharing help, but often rely on trusted or privileged parties and may still leak intermediate statistics. To address this, we propose MPC-XGB, a privacy-preserving framework for training XGBoost under VFL with an honest-but-curious threat model. It uses secure three-party computation with Replicated Secret Sharing, distributing data across non-colluding servers and performing all computations on shares. This ensures that raw data, labels, and model statistics remain hidden, while supporting both secure training and prediction. Experiments show that MPC-XGB achieves strong performance (0.93 accuracy, 0.82 AUC), comparable to that of existing methods, with improved privacy guarantees.

1. Introduction

A Gradient Boosted Decision Tree (GBDT) is a well-established machine learning (ML) technique widely adopted for its strong predictive performance, scalability, and ability to handle diverse data types with minimal pre-processing. However, as with any ML model, their performance benefits from access to large and diverse training data, often achievable only through extensive data collection. This reliance on data introduces serious privacy concerns, especially when dealing with sensitive user information (e.g., finance and healthcare). Due to privacy regulations such as the General Data Protection Regulation (GDPR) [1], the Consumer Privacy Bill of Rights (CPBR) [2] in the United States, and the Privacy Act 1988 in Australia [3], there is a growing need to secure data-driven techniques using privacy-preserving principles. This article is an extended version of our previously published short paper at IEEE BigData 2025 [4]. Compared to the earlier work, this paper provides a comprehensive description of the proposed algorithms, a detailed threat model and security analysis, and an expanded experimental evaluation.

Federated learning (FL) [5,6] offers a decentralized solution by enabling multiple parties to collaboratively train ML models without exchanging raw data. This approach has gained popularity in privacy-sensitive domains where data are naturally distributed across multiple organizations, for example, financial institutions jointly detecting fraudulent transactions while each party holds a portion of user records or transaction details, or healthcare providers improving personalized treatment recommendations by leveraging complementary patient data without exposing sensitive health records. Despite its advantages, FL still raises significant privacy concerns, as model updates can inadvertently leak sensitive information [7,8,9,10].

Within FL, Horizontal federated learning (HFL) partitions data by samples while sharing the same feature space [6]. HFL has been extensively studied with privacy-enhancing techniques such as Homomorphic Encryption (HE) [11,12,13], Differential Privacy (DP) [14,15,16], and Multi-Party Computation (MPC) [17,18]. These privacy-preserving mechanisms are used along with HFL to protect model updates during collaborative training [5,19,20,21,22]. In contrast, Vertical federated learning (VFL) partitions data by features across parties, typically with a single label-holding party. While many HFL techniques exist, VFL remains comparatively underexplored. Notable VFL works include HE-based SecureBoost [23], additive secret sharing schemes [24], hybrid HE+DP approaches [25], and recent secure gradient boosting variants [23,24,26,27,28,29,30,31,32].

In VFL, one Active Party (AP) holds the data features along with labels, while one or more Passive Parties (PPs) hold additional features [33]. Unlike HFL, where each party can train a submodel locally, VFL requires cross-party interaction during training, making it particularly susceptible to privacy leakage, especially towards the AP, which performs label-dependent computations [23,33]. Existing VFL frameworks typically rely on either (i) direct communication between the AP and PPs, in which intermediate values such as first and second-order derivatives of the training loss (gradients and Hessians) are computed and transmitted, exposing either labels or other model statistics and risking leakage [23,26,27,28,29,30]; or (ii) a single intermediary server that coordinates training but still learns sensitive values [33]. These designs undermine the goal of privacy-preserving federated learning. As illustrated in Figure 1a, a conventional VFL setup relies on direct communication between the AP, PP, and a central coordinator, where label-dependent gradients and Hessians from the AP and feature-derived statistics from the PP may be exchanged during training. Such exposure of intermediate model statistics can reveal sensitive information about labels or feature distributions to other entities involved in the training process.

For decision trees in VFL, SecureBoost [23], a HE-based vertical XGBoost method, hides raw features, but the AP remains privileged because it decrypts passive-side encrypted split statistics derived from the AP’s label-dependent gradients/Hessians. An additive SS scheme [27] allows each party to reconstruct the other’s inputs because each holds one original value and one share of the secret. A hybrid HE+DP approach for fraud detection in VFL [25] permits the AP to decrypt sensitive features from the PP for tree construction. As noted in the recent survey on tree-based VFL [34], the area has expanded beyond early HE-based designs to include more scalable, system-oriented, and privacy-enhanced variants such as SecureBoost+, Pivot, and OpBoost [35,36,37]. These studies show that tree-based VFL is increasingly practical, but they also highlight recurring trade-offs among privacy, efficiency, and trust assumptions. In particular, existing methods often still rely on revealing some intermediate statistics, introducing a trusted or semi-trusted coordinator, or sharing sensitive information among parties such as PP feature information sharing with AP. This leaves room for stronger end-to-end protection of labels, features, gradients, and Hessians during VFL tree training [34]. Some secure tree-learning frameworks further expand this design space. For example, SecureXGB proposes a secure and efficient multi-party protocol for vertical federated XGBoost, SiGBDT develops a large-scale vertically partitioned GBDT training framework using function secret sharing, and Guard-GBDT improves efficiency through approximations tailored to privacy-preserving GBDT training on vertical data [38,39,40]. These systems reinforce that recent progress has focused strongly on efficiency and scalable secure training while still leaving important design choices around leakage, trust assumptions, and protection of intermediate statistics.

Unlike prior VFL tree-training methods that protect selected components of the pipeline while still exposing sensitive intermediate statistics to a privileged party [23,25,27,34], our goal is to remove such privileged visibility from the training architecture itself. The conceptual novelty of our proposed method, MPC-XGB, lies in combining vertical XGBoost training with three-party replicated secret sharing in a way that moves all label-dependent split evaluation for passive features into MPC so that raw features, labels, per-instance gradients, and Hessians are not revealed to either training party or to any single server.

Table 1. Privacy preservation across VFL methods. ✓: protected, ✗: revealed.

Model	Labels (AP)	Grad/Hess (AP)	Grad/Hess (PP)	Features (AP)	Features (PP)
SecureBoost [23]	✓	✓	✗	✓	✓
Additive SS [27]	✓	✓	✓	✗	✗
Hybrid FL [25]	✓	✓	✗	✓	✗
Our Work	✓	✓	✓	✓	✓

highlights the key distinction between prior secure VFL tree-training methods and MPC-XGB in terms of protection of sensitive information. HE-based methods such as SecureBoost mainly protect raw feature values during cross-party computation, yet still leave the AP in a privileged position because passive-side split evaluation depends on label-derived statistics that are revealed, transferred, or inferable during training. Hybrid designs that combine HE with DP or related approximations improve efficiency and scalability, but typically do so by allowing for controlled disclosure, relying on a semi-trusted coordinator, or perturbing intermediate outputs in ways that trade privacy for utility. Additive secret-sharing approaches reduce direct plaintext exchange, but selected values may still be reconstructed by one or more training parties; so, confidentiality is not preserved end-to-end for all intermediate statistics. In contrast, MPC-XGB protects all five categories shown in Table 1, namely labels, AP-side gradients and Hessians, PP-side gradients and Hessians, AP features, and PP features, by redesigning the training architecture so that no party acts as a privileged recipient of label-dependent or feature-derived information. Instead, sensitive split evaluation is carried out entirely on secret shares across three non-colluding servers, and only the minimal split outcome required for tree growth is revealed, yielding stronger end-to-end privacy guarantees while preserving XGBoost functionality under VFL.

• Our Framework:

To address the privacy limitations of existing VFL tree-training architectures shown in Figure 1a, we propose MPC-XGB, a three-server framework for privacy-preserving XGBoost [41] under vertical federated learning, as illustrated in Figure 1b. The key architectural idea is to remove the privileged role typically held by the label-owning party or an intermediary coordinator during split evaluation. Instead, the AP and PP secret-share their sensitive inputs with three non-colluding servers: the AP secret-shares the label-dependent gradients and Hessians, while the PP secret-shares binarized feature indicators derived from its local threshold candidates. The three servers then jointly evaluate the passive-party split gains using MPC and return only the best passive split information required for tree construction. Under the semi-honest, non-colluding assumption, raw features, labels, per-instance gradients, and Hessians remain protected throughout training such that:

• No single party or server can reconstruct any private data or intermediate values;
• The protocol is secure under a semi-honest threat model, assuming no two servers collude.

Here, all participating parties act as clients, collaborating through three non-colluding servers that securely perform computations necessary for training decision trees. The AP secret-shares gradients and Hessians, PPs secret-share binarized features, and all sensitive computations occur exclusively on secret shares. Compared to our IEEE BigData short paper [4], this journal version presents a detailed algorithmic specifications and threat model, and an in-depth security discussion, formally reasoning about privacy guarantees under the assumed adversarial setting.

• Our Contributions:
  • We present MPC-XGB, a privacy-preserving protocol and system architecture for vertical XGBoost training that removes the need for any privileged training party to access another party’s sensitive intermediate statistics.
  • We show how label-dependent quantities and passive-side split evaluation can be carried out entirely on secret shares using three-party replicated secret sharing, thereby protecting raw features, labels, gradients, and Hessians end-to-end during training.
  • We provide a detailed end-to-end algorithmic description of the proposed framework, along with a clearly defined threat model, extensive security analysis and leakage profile, demonstrating correctness and confidentiality under a semi-honest adversarial assumption with non-colluding servers.
  • We substantially extend the experimental evaluation by adding comparisons with additional baseline methods and evaluating the framework on an additional dataset, resulting in more comprehensive performance analysis compared to our earlier work.

2. Technical Preliminaries

This section outlines the core technical foundations of our work, including an overview of XGBoost, secure three-party computation, and a baseline VFL-XGBoost system without additional privacy enhancements.

2.1. XGBoost

XGBoost builds an ensemble of decision trees over T boosting rounds, where each new tree is added to improve the current predictions [41,42]. For binary classification, the initial prediction is commonly set using the log-odds of the positive class:

$${\widehat{y}}_i^{\left(0\right)}=log\left({\displaystyle \frac{\mu }{1-\mu }}\right),$$

(1)

where $\mu$ is the mean of the binary labels. The current prediction is converted to a probability using the sigmoid function:

$$p_i={\displaystyle \frac{1}{1+e^{-{\widehat{y}}_i}}},$$

(2)

and the first- and second-order derivatives of the logistic loss are:

$$g_i=p_i-y_i,h_i=p_i(1-p_i).$$

(3)

At each node, XGBoost evaluates candidate splits by partitioning the instance set I into left and right subsets, $I_L$ and $I_R$, and selecting the split that maximizes the gain:

$${\mathcal{L}}_{\mathrm{split}}={\displaystyle \frac{1}{2}}\left({\displaystyle \frac{{\left({\sum }_{i\in I_L}{g}_i\right)}^2}{{\sum }_{i\in I_L}{h}_i+\lambda }}+{\displaystyle \frac{{\left({\sum }_{i\in I_R}{g}_i\right)}^2}{{\sum }_{i\in I_R}{h}_i+\lambda }}-{\displaystyle \frac{{\left({\sum }_{i\in I}{g}_i\right)}^2}{{\sum }_{i\in I}{h}_i+\lambda }}\right)-\gamma ,$$

(4)

where $\lambda$ is the leaf regularization parameter and $\gamma$ penalizes new splits. If no beneficial split is found, the node becomes a leaf with weight:

$$w_j=-{\displaystyle \frac{{\sum }_{i\in I_j}{g}_i}{{\sum }_{i\in I_j}{h}_i+\lambda }},$$

(5)

where $I_j$ is the set of instances in leaf j.

After a tree is built, predictions are updated as follows:

$${\widehat{y}}_i^{\left(t\right)}={\widehat{y}}_i^{(t-1)}+\eta w_{j_t\left(i\right)}^{\left(t\right)},$$

(6)

where $\eta$ is the learning rate and $w_{j_t\left(i\right)}^{\left(t\right)}$ is the weight of the leaf reached by instance i in tree t. Repeating this process for T rounds yields the final prediction:

$${\widehat{y}}_i^{\left(T\right)}={\widehat{y}}_i^{\left(0\right)}+\eta \sum _{t=1}^T{w}_{j_t\left(i\right)}^{\left(t\right)}.$$

(7)

• Approximate Split-Finding:

To reduce the cost of evaluating all possible thresholds, XGBoost typically uses approximate split-finding based on quantiles. For each feature $f_k$, a set of candidate thresholds ${\mathsf{\Theta }}_k=\{{\theta }_{k1},{\theta }_{k2},\dots \}$ is constructed. Gradient and Hessian statistics are then aggregated within quantile bins:

$$G_{kv}=\sum _{i:{\theta }_{k,v-1}\le x_{ik}<{\theta }_{kv}}{g}_i,H_{kv}=\sum _{i:{\theta }_{k,v-1}\le x_{ik}<{\theta }_{kv}}{h}_i,$$

(8)

which enables efficient evaluation of split gains in Equation (4).

2.2. Secure Three-Party Computation with Replicated Secret Sharing

We employ a three-party computation protocol [43], secure against semi-honest adversaries under an honest majority over the ring ${\mathbb{Z}}_{2^{\mathsf{\ell }}}$ with a $(2,3)$ replicated secret sharing (RSS) in which each party holds two ring elements. The protocol allows multiple parties to jointly evaluate arithmetic circuits over private inputs without revealing any information beyond the final output.

• Replicated secret sharing:

Let $v\in {\mathbb{Z}}_{2^{\mathsf{\ell }}}$. Sample $x_1,x_2,x_3\in {\mathbb{Z}}_{2^{\mathsf{\ell }}}$ such that $x_1+x_2+x_3\equiv 0(mod{2}^{\mathsf{\ell }})$ and define:

$$\begin{array}{cccc}\hfill a_1& =x_3-v,a_2\hfill & \hfill =x_1-v,a_3& =x_2-v(mod{2}^{\mathsf{\ell }}).\hfill \end{array}$$

(9)

Party $P_i$ receives the pair $(x_i,a_i)$. Any two parties can reconstruct v (e.g., $v=x_1-a_2=x_2-a_3=x_3-a_1$).

• Addition:

Given $(x_i,a_i)$ sharing $v_1$ and $(y_i,b_i)$ sharing $v_2$, each party computes locally (no communication):

$$\begin{array}{c}\hfill z_i=x_i+y_i,c_i=a_i+b_i(mod{2}^{\mathsf{\ell }}).\end{array}$$

(10)

The pairs $(z_i,c_i)$ form a valid sharing of $v_1+v_2$ since $c_i=(x_{i\ominus 1}+y_{i\ominus 1})-(v_1+v_2)$.

• Multiplication:

Let $(x_i,a_i)$ share $v_1$ and $(y_i,b_i)$ share $v_2$. Assume parties $P_1,P_2,P_3$ hold correlated randomness $\alpha ,\beta ,\gamma \in {\mathbb{Z}}_{2^{\mathsf{\ell }}}$ such that $\alpha +\beta +\gamma \equiv 0(mod{2}^{\mathsf{\ell }})$. Write ${\rho }_1=\alpha ,{\rho }_2=\beta ,{\rho }_3=\gamma$ and define the cyclic index $i\oplus 1:=(imod3)+1$. The product $v_1{v}_2$ is computed in one round as follows:

• Compute masked values and send (one element each):

  $$r_i={\displaystyle \frac{a_i{b}_i-x_i{y}_i+{\rho }_i}{3}},$$

  (11)

  send $r_i$ from $P_i$ to $P_{i\oplus 1}$($i\in \{1,2,3\}$), where division by 3 is well-defined since $gcd(3,2^{\mathsf{\ell }})=1$.
• Locally convert to a $(2,3)$-sharing of $v_1{v}_2$:

  $$(z_i,c_i)=\left(r_{i\oplus 2}-r_i,-2r_{i\oplus 2}-r_i\right)(mod{2}^{\mathsf{\ell }}),$$

  (12)

  for $i\in \{1,2,3\}$.

The resulting pairs $(z_i,c_i)$ constitute a correct $(2,3)$-RSS of $v_1{v}_2$. In prior privacy-preserving VFL research, some methods have adopted additive secret sharing to protect intermediate values such as gradients, Hessians, or split statistics during training [27]. In additive secret sharing, a secret is decomposed into shares whose sum reconstructs the value, which makes it simple and effective for basic secure aggregation. However, in two-party VFL settings, this approach has important limitations. Since the participating parties already possess their own local information and jointly hold the shares needed for reconstruction, sensitive intermediate statistics may become recoverable during protocol execution, creating a potential privacy risk. In addition, secure multiplication under additive secret sharing generally requires extra interaction rounds or additional preprocessing, which can introduce substantial communication and computational overhead for tree-based training, where many multiplications and comparisons are needed. By contrast, RSS in a three-party honest-majority setting provides stronger separation of knowledge, as no single party holds enough information to reconstruct the secret, while also enabling more efficient secure arithmetic. These properties make RSS better suited for our VFL XGBoost framework, where secure and repeated split evaluation is a core requirement.

3. System and Threat Model

This section formalizes our privacy-preserving VFL framework for training XGBoost under secure three-party computation. The system follows a client–server architecture where the AP and PP act as clients and three non-colluding servers run $(2,3)$ replicated secret sharing over ${\mathbb{Z}}_{2^{\mathsf{\ell }}}$. All sensitive information (raw features, labels, per-instance gradients/Hessians) is secret-shared and evaluated as arithmetic circuits, and each server observes only two shares and never learns any sensitive plaintext value. We adopt an honest-but-curious (semi-honest) adversarial model with at most one corrupted computer server and non-colluding parties. The design overcomes dominant VFL leakage issues e.g., label leakage from gradient exchange and inference of PP feature data, by ensuring that intermediate, label-dependent statistics remain on shares and are never reconstructed during training or prediction. Consequently, no raw features, labels or gradients are disclosed to any party during training and inference.

3.1. System Architecture

The system comprises three roles, as shown in Figure 1b. The datasets and notation follow the preliminaries (Equations (13)–(15)).

• Active party (AP): Holds $D_A$ (Equation (14)) for n aligned instances. At each node with instance set I, the AP computes $(g_i,h_i)$ and evaluates AP-side candidate splits for quantile-based candidate thresholds per feature as in the preliminaries’ approximate split-finding, and coordinates overall training.
• Passive party (PP): Holds $D_P$ (Equation (15)) with $m_2$ features (no labels). It forms quantile-based candidate thresholds per feature and produces bin indicators (per feature, per threshold) for secure gain evaluation at the servers.
• Secure servers: Three non-colluding, semi-honest servers ($S_1,S_2,S_3$) running $(2,3)$ replicated secret sharing over ${\mathbb{Z}}_{2^{\mathsf{\ell }}}$. Given secret shares of AP’s $(g_i,h_i)$ and PP’s bin indicators, they compute split gains for all passive features and thresholds.

Notation

Let $F_A={\left\{f_{A_u}\right\}}_{u=1}^{m_1}$ and $F_P={\left\{f_{P_v}\right\}}_{v=1}^{m_2}$ denote AP and PP feature sets, respectively. Each feature $f_{A_u}$ (resp. $f_{P_v}$) has a finite candidate-threshold set ${\mathsf{\Theta }}_{A_u}$ (resp. ${\mathsf{\Theta }}_{P_v}$), as introduced in the preliminaries. At a tree node with instance set $I\subseteq \{1,\dots ,n\}$, a candidate $(f_{A_u},\theta )$ or $(f_{P_v},\theta )$ with $\theta \in {\mathsf{\Theta }}_{A_u}$ or $\theta \in {\mathsf{\Theta }}_{P_v}$ induces a partition $I\to (I_L,I_R)$.

3.2. Threat Model

We consider a semi-honest (honest-but-curious) adversarial model in which all entities follow the prescribed protocol but may attempt to infer additional information from their local views. The system comprises the Active Party ($\mathcal{A}$), one or more Passive Parties (${\mathcal{P}}_1,\dots ,{\mathcal{P}}_k$), and three non-colluding servers (${\mathcal{S}}_1,{\mathcal{S}}_2,{\mathcal{S}}_3$) that execute three-party RSS over the ring ${\mathbb{Z}}_{2^{\mathsf{\ell }}}$. Samples are entity-aligned across parties with vertically partitioned features, and only $\mathcal{A}$ holds labels.

We assume an adversary may corrupt at most one party (either $\mathcal{A}$ or some ${\mathcal{P}}_i$) or one server during protocol execution. We also discuss the case where one party colludes with one server. Our privacy guarantees hold under the honest-majority, non-colluding assumption of three-party RSS.

3.2.1. Independent Corruption

Server corruption: Under three-party RSS with honest majority ($t=1$), the view of any single server is information-theoretically private and does not reveal raw features, labels, gradients, Hessians, or other plaintext intermediate values.

Party corruption: A corrupted $\mathcal{A}$ may observe the final trained model and the intentionally revealed passive split outputs used during training; a corrupted ${\mathcal{P}}_i$ may observe only its own inputs and protocol messages. However, raw features of the other parties, labels, per-instance gradients/Hessians, and unselected candidate statistics are never revealed in plaintext and remain protected beyond the defined leakage profile.

3.2.2. Collusion Between One Party and One Server

We also consider collusion between a single party and a single server. In this case, the joint view remains insufficient to reconstruct another party’s private inputs or hidden intermediate statistics because one server alone does not hold enough information to reconstruct RSS-shared values, and plaintext values are never revealed at the servers.

3.2.3. Threat Boundaries

Our guarantees do not extend beyond the above corruption threshold. In particular, if two servers collude, the privacy guarantee of the current RSS setting no longer holds, since two parties together can reconstruct shared secrets. Likewise, collusion between $\mathcal{A}$ and ${\mathcal{P}}_i$ trivially combines labels and all vertically partitioned features, revealing the full dataset. Further, malicious adversaries that deviate from the protocol, such as by sending malformed shares or manipulating intermediate computations, are outside the scope of the present semi-honest analysis.

The semi-honest, honest-majority setting is a standard starting point in secure multi-party computation and has been widely adopted in prior three-party computation literature [43,44,45]. In our setting, the assumption of non-colluding servers is motivated by deployment across independent administrative domains. Extending MPC-XGB to maliciously secure protocols, such as variants with consistency checks, MAC-based protections, or other active-security mechanisms, is an important direction for future work.

4. Proposed Method: MPC-XGB

This section formalizes our privacy-preserving VFL framework under secure three-party computation. We introduce a baseline VFL with no additional privacy first, and then we move to our proposed privacy preserving framework.

4.1. VFL-XGBoost (No-Privacy Baseline)

This subsection formalizes a non-private baseline for training XGBoost under VFL. The system comprises the AP, that holds labels and local features, and one or more PPs that hold data features only. This baseline serves as the comparison point for our subsequent privacy-preserving three-party protocol. Let the training dataset be:

$$D={\left\{\left(x^{\left(i\right)},y_i\right)\right\}}_{i=1}^n,y_i\in \{0,1\}.$$

(13)

The AP holds:

$$D_A={\left\{\left(f_{A_1}^{\left(i\right)},f_{A_2}^{\left(i\right)},\dots ,f_{A_{m_1}}^{\left(i\right)},y_i\right)\right\}}_{i=1}^n,$$

(14)

For ease of notation, we describe the case with a single PP; the multi-PP case repeats the PP-side steps in parallel. Denote the PP’s dataset as follows:

$$D_P={\left\{\left(f_{P_1}^{\left(i\right)},f_{P_2}^{\left(i\right)},\dots ,f_{P_{m_2}}^{\left(i\right)}\right)\right\}}_{i=1}^n.$$

(15)

i.e., $m_2$ local features but no labels.

Feature notation: Let $F_A=\{f_{A_1},\dots ,f_{A_{m_1}}\}$ and $F_P=\{f_{P_1},\dots ,f_{P_{m_2}}\}$ denote the feature sets owned by the AP and PP, respectively. For instance i, write $F_A^{\left(i\right)}=(f_{A_1}^{\left(i\right)},\dots ,f_{A_{m_1}}^{\left(i\right)})$ and $F_P^{\left(i\right)}=(f_{P_1}^{\left(i\right)},\dots ,f_{P_{m_2}}^{\left(i\right)})$, so that $x^{\left(i\right)}=(F_A^{\left(i\right)},F_P^{\left(i\right)})$. All parties share the same aligned index set $\{1,\dots ,n\}$.

4.1.1. Baseline Training Flow

The steps involved in the training of VFL XGBoost are given below:

• Node statistics at the AP: At a current node with instance set I, the AP computes probabilities and first/second derivatives ${\{p_i,g_i,h_i\}}_{i\in I}$ using Equations (2) and (3).
• Share label-dependent statistics: The AP transmits ${\left\{(g_i,h_i)\right\}}_{i\in I}$ to the PP.
• PP-side split search: Using the received $\{g_i,h_i\}$ for I, the PP evaluates candidate splits over its local features (using quantile thresholds per feature), computes the split gain via Equation (4), and returns its best split to the AP (feature ID, threshold and gain).
• AP-side split search (in parallel): The AP evaluates candidate splits over its own features (using quantile thresholds per feature) by calculating the split gain in Equation (4).
• Selection of overall best split: The AP compares the best PP candidate with the best AP candidate and chooses the higher gain split. The node’s instance set I is partitioned into $I_L$ and $I_R$ accordingly. If the PP’s split is chosen, the AP requests $I_L$/$I_R$ from the PP.
• Recursive process: Repeat Steps 1–5 independently on each child node until the round’s stopping criteria are met (e.g., maximum depth, minimum samples, or no positive gain), yielding one tree. Compute leaf weight given by Equation (5).
• Boosting Iterations: After the tree is built, the AP updates the predictions ${\widehat{y}}^{\left(t\right)}$ as in Equation (6), then recomputes $p_i,g_i,h_i$ for the next round from the updated predictions.

4.1.2. Limitations of Non-Private VFL-XGBoost

Although the raw features or labels are not shared directly with other parties in this approach, this baseline exposes sensitive information, i.e., sharing $\{g_i,h_i\}$ reveals label-dependent statistics to the PP. For common losses (e.g., logistic), $g_i$ and $h_i$ correlate with $y_i$ and the AP’s current predictions. Repeated exposure at every node and round enables the PP to infer labels with high confidence. The next section introduces our privacy-preserving protocol, which mitigates these vulnerabilities by introducing three-party computation.

4.2. MPC-XGB

Figure 2 summarizes one iteration. At each node with instance set I, the AP computes $(p_i,g_i,h_i)$ using Equations (2) and (3) and evaluates AP-side candidates with the split gain ${\mathcal{L}}_{\mathrm{split}}$ (Equation (4)). In parallel, the PP forms quantile thresholds ${\mathsf{\Theta }}_{P_v}$ for each feature $f_{P_v}$, binarizes values according to the thresholds, and secret-shares these along with the AP’s secret-shared $(g_i,h_i)$ to the three servers. The servers aggregate per-threshold $(G,H)$ (Equation (8)) and compute ${\mathcal{L}}_{\mathrm{split}}$ for all passive candidates, revealing only the argmax tuple (gain, feature index, threshold) to the AP. The AP compares this passive argmax with its best local candidate and selects the global maximizer of ${\mathcal{L}}_{\mathrm{split}}$. The selected split partitions I into $(I_L,I_R)$ and the process of finding best splits recurses until stopping criteria hold. Leaf weights are then set via Equation (5), the tree is added to the ensemble, and predictions are updated using Equation (6). The process repeats for T boosting rounds.

4.3. Objective Formulation of MPC-XGB

At a node with instance set I, let ${\mathcal{C}}_A\left(I\right)$ and ${\mathcal{C}}_P\left(I\right)$ denote the candidate splits formed from the active-party and passive-party features, respectively, using their corresponding quantile thresholds. The node-level split selection is:

$$c^{\star }=arg\underset{c\in {\mathcal{C}}_A\left(I\right)\cup {\mathcal{C}}_P\left(I\right)}{max}{\mathcal{L}}_{\mathrm{split}}\left(c\right),$$

where ${\mathcal{L}}_{\mathrm{split}}$ is the XGBoost split gain in Equation (4).

MPC-XGB preserves this objective while decomposing its evaluation according to feature ownership. The AP computes the best split over ${\mathcal{C}}_A\left(I\right)$ locally using its own features and label-dependent derivatives. In parallel, the three MPC servers securely compute the best split over ${\mathcal{C}}_P\left(I\right)$ from secret shares of the AP’s gradients and Hessians and the PP’s binarized feature indicators. Only the best passive split tuple $\langle {\mathcal{L}}_{\mathrm{split}}^{P\star },v^{\star },{\theta }^{\star }\rangle$ is revealed to the AP, which then compares it with the best active split and selects the overall maximizer.

Therefore, MPC-XGB preserves the same split-selection rule as non-private VFL-XGBoost while shifting passive-side split evaluation from plaintext computation to replicated-secret-shared MPC. The step-by-step training flow is discussed below:

• Step 1: Local Gradient/Hessian Computation (AP) and Feature Binarization (PP)
  • AP:

    –

    For each $i\in I$, compute $(p_i,g_i,h_i)$ via Equations (2) and (3).

    –

    For each active feature $f_{A_u}$, form quantile-based thresholds ${\mathsf{\Theta }}_{A_u}$ and find split gain for AP-side candidates with ${\mathcal{L}}_{\mathrm{split}}$ (Equation (4)).

  • PP:

    –

    For each passive feature $f_{P_v}$, obtain candidate thresholds ${\mathsf{\Theta }}_{P_v}$ using the quantile-based method.

    –

    For each threshold $\theta \in {\mathsf{\Theta }}_{P_v}$, binarize feature values as follows:

    $${\tilde{f}}_{P_v}(i,\theta )=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}{f}_{P_v}^{\left(i\right)}\ge \theta ,\hfill \\ 0,\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

    where $f_{P_v}^{\left(i\right)}$ is the value of feature $f_{P_v}$ for instance i. This binarized feature encodes whether instance i would be placed in right or left branch if split at $\theta$.

• Step 2: Secret Sharing to Three Servers
  • AP: shares ${\left\{g_i\right\}}_{i\in I}$ and ${\left\{h_i\right\}}_{i\in I}$ for the current node’s instance set I.
  • PP: for each feature $f_{P_v}$ and each threshold $\theta \in {\mathsf{\Theta }}_{P_v}$, shares:

    $$\left(v,\theta ,{\left\{{\tilde{f}}_{P_v}(i,\theta )\right\}}_{i\in I}\right).$$
  • Shares are computed as defined in Section 2.2 (replicated secret sharing over ${\mathbb{Z}}_{2^{\mathsf{\ell }}}$).
• Step 3: Secure Split Evaluation (MPC Servers)
  • From shares, compute the sums $G={\sum }_{i\in I}{g}_i$ and $H={\sum }_{i\in I}{h}_i$.
  • For each passive candidate $(v,\theta )$, compute left/right sums of G and H.
  • Evaluate the split objective ${\mathcal{L}}_{\mathrm{split}}(v,\theta )$ using Equation (4), and reveal only:

    $$\left({\mathcal{L}}_{\mathrm{split}}^{\star },v^{\star },{\theta }^{\star }\right)$$

    where ${\mathcal{L}}_{\mathrm{split}}^{\star }={max}_{v,\theta }{\mathcal{L}}_{\mathrm{split}}(v,\theta )$ denotes the maximum split gain (max split gain) and $(v^{\star },{\theta }^{\star })$ is the corresponding feature id and threshold value.
• Step 4: Best Global Split Selection (AP)
  • AP compares the best local candidate from Step 1, $(u^{\star },{\theta }_A^{\star },{\mathcal{L}}_{\mathrm{split}}^{A\star })$, with the best passive candidate from Step 3, $(v^{\star },{\theta }^{\star },{\mathcal{L}}_{\mathrm{split}}^{P\star })$.
  • Choose the higher gain:

    $$\mathrm{Active}\mathrm{if}{\mathcal{L}}_{\mathrm{split}}^{A\star }\ge {\mathcal{L}}_{\mathrm{split}}^{P\star };\mathrm{otherwise},\mathrm{Passive}.$$
  • Insert the node in tree based on the best global split.
• Step 5: Node partitioning and Recursion
  • Partition the current instance set I according to the best split in Step 4:

    –

    Active split: $I_L=\{i\in I:f_{A_{u^{\star }}}^{\left(i\right)}\le {\theta }_A^{\star }\}$, $I_R=I\setminus I_L$. The AP partitions locally and shares $I_L$ with the PP.

    –

    Passive split: $I_L=\{i\in I:f_{P_{v^{\star }}}^{\left(i\right)}\le {\theta }^{\star }\}$, $I_R=I\setminus I_L$. The PP partitions and returns $I_L$ to the AP.

  • For each child node with instance set $I^{\prime }\in \{I_L,I_R\}$, recursively compute maximum gain until a stopping criterion holds (i.e., $max{\mathcal{L}}_{\mathrm{split}}\le 0$, depth limit, or $\left|I\right|<\min \text{\_}\mathrm{sample}$), create a leaf and assign its weight via Equation (5).
• Step 6: Boosting Iterations
  • After the tree t is finalized, update predictions using Equation (6).
  • Add the tree to the ensemble and proceed to the next round; recompute $(p_i,g_i,h_i)$ via Equations (2) and (3).
  • Repeat Steps 1–5 until T trees are built.

4.4. Boosting Round Initialization and Gradient Sharing

Algorithm 1 presents the overall training loop for our MPC-XGB framework. The procedure follows the principle of gradient boosting, where T decision trees are constructed sequentially, each correcting the residual errors of the previous trees.

Algorithm 1 Privacy-Preserving VFL-XGBoost Training

1: Input: Active-party dataset $D_A$ of size $n\times m_1$ with features $F_A={\left\{f_{A_u}\right\}}_{u=1}^{m_1}$ and labels ${\left\{y_i\right\}}_{i=1}^n$ Passive-party dataset $D_P$ of size $n\times m_2$ with features $F_P={\left\{f_{P_v}\right\}}_{v=1}^{m_2}$ (Equations (14) and (15))   2: Output: Ensemble of T decision trees   3: for $t=1$ to T do                      ▷ Build T trees   4:     At Active Party:   5:     for $i=1$ to n do   6:         if $t==1$ then   7:            Initialize prediction ${\widehat{y}}_i^{\left(0\right)}$ via Equation (1)   8:         else   9:            Use predictions ${\widehat{y}}_i^{(t-1)}$ calculated from the previous round using Equation (6) 10:         end if 11:         Compute $p_i$, $g_i$, $h_i$ using Equations (2) and (3) 12:     end for 13:     Secret-share ${\left\{g_i\right\}}_{i=1}^n$ and ${\left\{h_i\right\}}_{i=1}^n$ to servers $S_1,S_2,S_3$ using RSS as defined in the Secure Three-Party Computation subsection (Equation (9)) 14:     Initialize empty tree $Tre{e}^{\left(t\right)}$; set root instance set $I=\{1,\dots ,n\}$ 15:     $Tre{e}^{\left(t\right)}\leftarrow \mathtt{Tree}\_\mathtt{Construction}(I,G^{\left(t\right)},H^{\left(t\right)},Tre{e}^{\left(t\right)})$           ▷ Algorithm 2 16:     Add $Tre{e}^{\left(t\right)}$ to the ensemble 17:     Update predictions for all i using Equation (6) 18: end for 19: return Ensemble of T trees

At the beginning of each boosting round, the AP maintains prediction scores ${\widehat{y}}_i^{\left(t\right)}$ for every instance. $i\in \{1,\dots ,n\}$ initializes ${\widehat{y}}^{\left(0\right)}$ by Equation (1), derives $(p_i,g_i,h_i)$ via Equations (2) and (3), and secret-shares $\{g_i,h_i\}$ to the servers (RSS as in Section 2.2). The root node starts with $I=\{1,\dots ,n\}$. The recursive tree-building procedure (Tree_Construction) is then invoked, which partitions instances, assigns optimal leaf weights (Equation (5)), and evaluates splits using Equation (4). After the tree is finalized, it is added to the ensemble, and the AP updates its predictions using the new leaf weights. This iterative process continues for T rounds, gradually refining predictions and yielding a final ensemble of decision trees trained in a privacy-preserving manner.

4.5. Recursive Tree Construction and Secure Split Selection

Algorithm 2 outlines the recursive tree construction procedure and split selection across both parties.The AP evaluates gains on $F_A$ locally (Equation (4)), and the PP binarizes $F_P$ against ${\mathsf{\Theta }}_{P_v}$ and secret-shares binarized features with three servers.

The servers perform secure gain computation for each passive feature using the secret-shared gradients and Hessians. Only the best-performing feature-threshold pair is revealed to the AP. If the chosen split belongs to the AP, it performs the instance space partitioning and shares $I_L$ with PP. Otherwise, PP partitions the data and shares $I_L$ with AP.

Each recursive call of split selection processes a smaller subset of instances, and the process continues until stopping criteria are met, i.e, non-positive gain, maximum depth reached, or $\left|I\right|<\mathtt{min}\text{\_}\mathtt{sample}$. At that point, a leaf weight is computed using the Equation (5).

Algorithm 2 Tree_Construction(I, $G^{\left(t\right)}$, $H^{\left(t\right)}$, $Tre{e}^{\left(t\right)}$)

Input: Instance space of current node I $G^{\left(t\right)}={\left\{g_i^{\left(t\right)}\right\}}_{i\in I}$ and $H^{\left(t\right)}={\left\{h_i^{\left(t\right)}\right\}}_{i\in I}$ from AP Feature sets $F_A={\left\{f_{A_u}\right\}}_{u=1}^{m_1}$, $F_P={\left\{f_{P_v}\right\}}_{v=1}^{m_2}$ Output: Constructed $Tre{e}^{\left(t\right)}$ with privacy preservation   1: for each party $c\in \{A,P\}$ do   2:     for each feature $f\in F_c$ do   3:         Form candidate thresholds ${\mathsf{\Theta }}_{A_u},{\mathsf{\Theta }}_{P_v}$   4:     end for   5:     if $c=A$ then                ▷ Active Party local search   6:         for each $f=f_{A_u}\in F_A$ do   7:            for each $\theta \in {\mathsf{\Theta }}_{A_u}$ do   8:                Compute ${\mathcal{L}}_{\mathrm{split}}^{\left(A\right)}(u,\theta )$ via Equation (4)   9:            end for 10:         end for 11:     else                         ▷ Passive Party 12:         for each $f=f_{P_v}\in F_P$ do 13:            for each $\theta \in {\mathsf{\Theta }}_{P_v}$ do 14:                Binarize feature values ${\left\{{\tilde{f}}_{P_v}(i,\theta )\right\}}_{i\in I}$ 15:                Secret-share ${\left\{{\tilde{f}}_{P_v}(i,\theta )\right\}}_{i\in I}$, feature id v, and $\theta$ to $S_1,S_2,S_3$ using RSS (Equation (9)) 16:            end for 17:         end for 18:     end if 19: end for 20: At Servers: GainComputationServer()            ▷ Algorithm 3 21: At Active Party: 22: $\mathit{node}\leftarrow argmax\left\{\langle {\mathcal{L}}_{\mathrm{split}}^{A\star },f^{A\star },{\theta }^{A\star }\rangle ,\langle {\mathcal{L}}_{\mathrm{split}}^{P\star },f^{P\star },{\theta }^{P\star }\rangle \right\}$ 23: if node meets stopping criteria then 24:     Set leaf weight w via Equation (5); insert $\langle \mathit{leaf},w\rangle$ into $Tre{e}^{\left(t\right)}$; return 25: else 26:     Insert internal split $\langle f^{*},{\theta }^{*}\rangle$ to $Tre{e}^{\left(t\right)}$ 27:     Partition I into $I_L$ and $I_R$ according to $\langle f^{*},{\theta }^{*}\rangle$ 28:     Tree_Construction($I_L$, $G^{\left(t\right)}$, $H^{\left(t\right)}$, $Tre{e}^{\left(t\right)}$) 29:     Tree_Construction($I_R$, $G^{\left(t\right)}$, $H^{\left(t\right)}$, $Tre{e}^{\left(t\right)}$) 30: end if

4.6. Secure Gain Computation on MPC Servers

Algorithm 3 details how the MPC servers collaboratively compute split gains over secret shares without revealing private values. Given shares of $\{g_i,h_i\}$ and $\left\{{\tilde{f}}_{P_v}(i,\theta )\right\}$, servers compute $(G_L,H_L)$ and $(G_R,H_R)$, derive $(G,H)$, and evaluate ${\mathcal{L}}_{\mathrm{split}}$ entirely over shares. The computation is conducted entirely over secret shares, ensuring that no server learns any private values. Only $\langle {\mathcal{L}}_{\mathrm{split}}^{P\star },v^{\star },{\theta }^{\star }\rangle$ is opened to the AP; all unselected candidates remain hidden.

To maintain correctness over finite fields, all arithmetic operations including addition, multiplications and divisions are carefully implemented using secret-sharing protocols, as defined in Section 2.2.

4.7. MPC-XGB Inference

After training is complete, inference for new samples proceeds using the trained ensemble. Algorithm 4 outlines how inference is performed locally at the AP without revealing or accessing raw passive features. The AP performs all prediction steps by evaluating AP splits locally and following recorded PP splits using the stored $(v,\theta )$ captured at training time (no PP interaction or raw $F_P$ is required at inference). For each node in a tree, if the split condition is based on an feature’s by AP, the AP makes the decision directly. If it is based on PP’s feature, the AP uses the stored feature ID and corresponding threshold to simulate the decision, as no raw values are needed at inference time. The per-tree leaf weights are accumulated with learning rate per Equation (6) and converted to probabilities via Equation (2). This approach ensures that no communication or online interaction is required with the PP during inference. As all splits are recorded using feature IDs and corresponding thresholds, the AP can traverse each tree locally without compromising privacy.

Algorithm 3 GainComputationServer()

Inputs at $S_1,S_2,S_3$: RSS shares (Equation (9)) of ${\left\{g_i^{\left(t\right)}\right\}}_{i\in I}$ and ${\left\{h_i^{\left(t\right)}\right\}}_{i\in I}$ from AP for each passive feature $f_{P_v}$ and threshold $\theta \in {\mathsf{\Theta }}_{P_v}$, shares of ${\left\{{\tilde{f}}_{P_v}(i,\theta )\right\}}_{i\in I}$ from PP Output: Reveal only $\langle {\mathcal{L}}_{\mathrm{split}}^{P\star },v^{\star },{\theta }^{\star }\rangle$ to AP   1: for each $v\in \{1,\dots ,m_2\}$ do   2:     for each $\theta \in {\mathsf{\Theta }}_{P_v}$ do   3:         Using shared multiplication (Equations (11) and (12)) and local addition (Equation (10)), compute:   4:         $G_L\leftarrow {\sum }_{i\in I}\left(1-{\tilde{f}}_{P_v}(i,\theta )\right)·g_i^{\left(t\right)}$   5:         $H_L\leftarrow {\sum }_{i\in I}\left(1-{\tilde{f}}_{P_v}(i,\theta )\right)·h_i^{\left(t\right)}$   6:         $G_R\leftarrow {\sum }_{i\in I}{\tilde{f}}_{P_v}(i,\theta )·g_i^{\left(t\right)}$   7:         $H_R\leftarrow {\sum }_{i\in I}{\tilde{f}}_{P_v}(i,\theta )·h_i^{\left(t\right)}$   8:         $G\leftarrow G_L+G_R$,    $H\leftarrow H_L+H_R$   9:         Compute ${\mathcal{L}}_{\mathrm{split}}(v,\theta )$ via Equation (4) using $(G_L,H_L)$, $(G_R,H_R)$, and $(G,H)$ (all over shares) 10:     end for 11: end for 12: $\langle {\mathcal{L}}_{\mathrm{split}}^{P\star },v^{\star },{\theta }^{\star }\rangle \leftarrow arg{max}_{v,\theta }{\mathcal{L}}_{\mathrm{split}}(v,\theta )$ 13: Reveal only $\langle {\mathcal{L}}_{\mathrm{split}}^{P\star },v^{\star },{\theta }^{\star }\rangle$ to the Active Party

Algorithm 4 Privacy-Preserving Prediction for a New Sample

1: Input: Trained ensemble ${\left\{Tre{e}^{\left(t\right)}\right\}}_{t=1}^T$ New sample $\mathbf{x}$ with features $(x_1,x_2,\dots ,x_n)$   2: Output: Predicted probability p for the sample   3: for $t=1$ to T do   4:     Start at the root node of tree t held by the AP   5:     while $node$ is not a leaf do   6:         if $node$ split uses $f_{A_u}$ at threshold $\theta$ then   7:            if $x<\theta$ then   8:                $node\leftarrow node.\mathrm{L}\mathrm{E}\mathrm{F}\mathrm{T}$   9:            else 10:                $node\leftarrow node.\mathrm{R}\mathrm{I}\mathrm{G}\mathrm{H}\mathrm{T}$ 11:            end if 12:         else                  ▷ split uses passive feature $f_{P_v}$ 13:            Follow stored $(v,\theta )$ branch recorded at training time (no raw $F_P$ access) 14:         end if 15:     end while 16:     Let j be the reached leaf; update $\widehat{y}\leftarrow \widehat{y}+\eta w_j^{\left(t\right)}$       ▷ Equation (6) 17: end for 18: $p\leftarrow$ sigmoid$\left(\widehat{y}\right)$ per Equation (2) 19: return  p

5. Security Analysis

In this section, we analyze the security of MPC-XGB under the semi-honest model. Our protocol combines RSS and MPC to prevent any party or server from accessing other parties’ private inputs or hidden intermediate values beyond the explicitly defined leakage. We summarize the guarantees in terms of confidentiality, leakage profile, and simulation-based security.

5.1. Confidentiality of Inputs and Intermediate Values

Raw AP/PP features, labels, per-instance gradients $\left(g_i\right)$, per-instance Hessians $\left(h_i\right)$, and plaintext intermediate statistics are never revealed during training. All sensitive values are secret-shared, and the servers evaluate bin comparisons, aggregate $(G,H)$ values, and compute split gains without reconstructing those quantities in plaintext. In particular, unselected candidate gains, unselected thresholds, and candidate-specific gradient/Hessian aggregates remain hidden throughout the protocol. Passive parties do not receive labels or gradients/Hessians in plaintext, and the AP does not receive raw passive features.

5.2. Leakage Profile

We now make explicit the information intentionally revealed by MPC-XGB under the assumed threat model.

• Per-node leakage during training:

At each node, only the best passive split tuple

$$\langle {\mathcal{L}}_{\mathrm{split}}^{P\star },v^{\star },{\theta }^{\star }\rangle$$

is revealed to the Active Party for comparison with its locally computed best active split. No other passive candidate statistics are opened. In particular, all non-maximal passive gains, feature-threshold candidates, and their associated aggregated gradient/Hessian values remain hidden. In our implementation, the current node instance space $\left|I\right|$ is also revealed for synchronization. This count is therefore part of the defined leakage profile.

• Per-tree leakage:

Across all nodes within one tree, the AP observes the sequence of selected passive split tuples and the corresponding node sizes for those nodes where passive candidates are evaluated. This reveals the final tree structure and split decisions that are necessary outputs of training, but does not reveal raw passive features, labels, per-instance gradients/Hessians, or unselected candidate statistics.

• Cumulative leakage across boosting rounds:

Across multiple trees, the repeated revelation of selected passive split tuples and node sizes may expose coarse distributional information, such as repeated selection patterns over passive feature IDs. However, this leakage is limited to the selected outputs of training and does not reveal raw feature values, feature names, labels, per-instance gradients/Hessians, or the full set of candidate gains considered at each node.

• Inference-time leakage:

At inference time, no new gradients, Hessians, labels, or passive candidate statistics are generated or revealed. The AP uses the final trained model, including the recorded split metadata, to traverse the ensemble. Therefore, inference does not introduce additional protocol leakage beyond the final model already disclosed at the end of training.

5.3. Simulation-Based Security and Final Output Disclosure

We target simulation-based security in the semi-honest model with respect to an ideal training functionality ${\mathcal{F}}_{\mathrm{Train}}$ that outputs exactly the leakage profile above and the final model. For each potentially corrupted $\mathcal{P}\in \{\mathcal{A},{\mathcal{P}}_1,\dots ,{\mathcal{P}}_k,{\mathcal{S}}_1,{\mathcal{S}}_2,{\mathcal{S}}_3\}$, there exists a PPT simulator ${\mathrm{Sim}}_{\mathcal{P}}$ such that ${\mathrm{View}}_{\mathcal{P}}{\approx }_{ϵ}{\mathrm{Sim}}_{\mathcal{P}}\left({\mathrm{output}}_{\mathcal{P}}\right)$ with negligible $ϵ$ in the security parameter $\kappa$. Robustness against one server corruption and one party + one server collusion follows from RSS privacy against one corrupt participant; servers exchange only additive shares. At the training end, only the model structure (feature indices, thresholds, leaf weights) is revealed to $\mathcal{A}$; inference can be run locally without further interaction, preserving the same privacy guarantees post-training. Together, these guarantees provide end-to-end privacy preservation throughout the entire VFL-XGBoost training pipeline, across multiple parties and repeated computation rounds.

6. Performance Evaluation

In this section, we evaluate the proposed MPC-XGB framework on two credit-related datasets to assess its predictive performance under vertically partitioned training and compare the performance with baselines.

Dataset 1: We conducted experiments using the Credit Scoring dataset [46], which contains 150,000 instances and 10 financial attributes per user. The task is to predict whether a user is at risk of experiencing severe financial distress. For the VFL setup, we partitioned features vertically: the AP held five attributes with labels, and the PP held the remaining five attributes without labels. This dataset represents a realistic financial risk prediction scenario in which sensitive user information is naturally distributed across parties. It therefore provides a suitable testbed for assessing whether privacy-preserving VFL can retain strong predictive performance under secure training.

Dataset 2: In addition, we evaluated our framework on a second credit scoring dataset [47], which is correlated to the task of predicting whether a user would make credit card payments on time. This dataset consists of 30,000 instances and 25 attributes in total. For the VFL setting, we vertically partitioned the features such that the AP held 14 attributes along with labels, while the PP held the remaining 12 attributes without labels. Compared with Dataset 1, this dataset contains a larger feature space and a different prediction target, allowing us to evaluate the consistency of the proposed method across distinct credit-related tasks. This also helps demonstrate that the framework is not limited to a single dataset configuration or feature split.

Both datasets are highly imbalanced; so, we report threshold-agnostic metrics (ROC-AUC and PR-AUC) and threshold-aware metrics (Precision, Recall, F1) at a validation-selected operating point (threshold chosen to maximize F1). Robustness is probed via per-tree row subsampling. Specifically, at the beginning of each boosting round, a random subset of the training instances is sampled without replacement according to row_sample, and the gradients, Hessians, and candidate split statistics for that tree are computed only on the sampled subset. A new subset is drawn independently for each tree; so, the ensemble is trained on multiple overlapping but non-identical views of the data. Since class imbalance can make accuracy alone potentially misleading, these metrics provide a more informative view of ranking quality as well as minority-class detection performance. We compare four configurations: (i) Non-Private VFL-XGBoost (baseline without privacy), (ii) SecureBoost [23], (iii) Hybrid FL [25], and (iv) MPC-XGB (our privacy-preserving VFL algorithm).

Experimental Setup:

Hardware and Software Environment: All experiments were conducted on a machine equipped with a 4th generation Intel Xeon Sapphire Rapids processor, 4 vCPUs (8 threads), and 64 GiB RAM. The experiments were executed on a single machine, where both the data-owner scripts and the MP-SPDZ-based secure computation backend were run. Our implementation used the MP-SPDZ framework [48] with the three-party replicated secret sharing backend (ps-rep-ring), consistent with the semi-honest, honest-majority setting considered in this work. All secure computations were implemented in MP-SPDZ over the arithmetic ring ${\mathbb{Z}}_{2^{163}}$, i.e., with ring bit length $\mathsf{\ell }=163$, and fixed-point values were represented using sfix with precision parameters $(f,k)=(12,60)$.

Hyperparameters: For Dataset 1, which contains 150,000 total instances, we used 50,000 instances as the test set. From the remaining 100,000 instances, 10% was used for validation and the rest for training, resulting in 90,000 training instances, 10,000 validation instances, and 50,000 test instances. The same data partition and vertical feature split were used for all compared methods and for Dataset 2. A fixed random seed of 42 was used throughout. The validation set was used to tune the model configuration and to select the operating threshold for threshold-dependent metrics. Based on this setup, XGBoost used a maximum depth of 6 and up to 20 boosting rounds (max_trees). We set $\lambda =2.0$ (lmd), min_sample = 8, and $\gamma =0.3$ (gma). Approximate split finding used 8 quantile bins (quantile), and the learning rate (eta) was set to $0.05$. Threshold-dependent metrics, including Precision, Recall, and F1, were computed on the test set using the operating threshold selected on the validation set, while threshold-independent metrics such as ROC-AUC and PR-AUC were computed directly from test-set prediction scores.

Efficiency-Oriented Design Choices: We employed row subsampling with row_sample values of $0.5$, $0.3$, $0.15$, and $0.05$. For each setting, at every boosting round, only the corresponding fraction of training instances was randomly sampled and used to construct the current tree. The subset was drawn independently for each tree, introducing controlled diversity across boosting rounds while reducing the volume of secure computation required for split evaluation. These settings were selected by us after empirical testing to balance predictive performance with the practical overhead of secure computation, particularly for repeated split evaluation under MPC. In particular, quantile-based split approximation limits the number of candidate thresholds per feature, while per-tree row subsampling reduces the number of instances contributing to secure gradient and Hessian aggregation. Together, these two design choices lower both computation and communication overhead while preserving stable model quality.

Results:

Table 2. Comparison of performance metrics across Non-Private VFL XGBoost, SecureBoost, Hybrid FL, and MPC-XGB.

Metric	Non-Private VFL XGBoost	SecureBoost	Hybrid FL	MPC-XGB
Accuracy	0.9015	0.92	0.94	0.93
Precision	0.37	0.45	0.98	0.4900
Recall	0.56	0.23	0.75	0.2500
F1	0.47	0.31	0.81	0.3300
AUC	0.87	0.7913	0.66	0.8157

presents the overall model performance across four configurations. We used non-private VFL-XGBoost, SecureBoost and Hybrid FL as primary baselines because they are structurally closest to our setting: all operate in a vertical tree-learning scenario with an active label-holding party and passive feature-holding party, allowing controlled comparison of privacy–utility trade-offs under similar training workflows. MPC-XGB outperforms SecureBoost across most metrics and approaches the non-private baseline on accuracy, while its ROC-AUC is lower than that of non-private VFL-XGBoost but higher than that of SecureBoost. Notably, MPC-XGB shows higher precision and lower recall than the non-private baseline. We attribute this difference to three factors under severe imbalance: (1) In our three-party MPC setup, all real-valued operations use fixed-point arithmetic with limited precision, which can perturb gain values and split selection. (2) Our use of per-tree row subsampling, where each tree is trained on an independently drawn subset of instances, introduces additional stochasticity into the ensemble. Under severe class imbalance, this may reduce the frequency with which borderline positive samples appear in the training subset of a given tree, slightly lowering recall while often improving precision by yielding more conservative split selection. (3) The validation-selected operating point (F1-max) tends to favor precision over recall. Compared to Hybrid FL, although Hybrid FL achieves high accuracy and F1-score, its ROC-AUC is substantially lower. Given the highly imbalanced nature of the evaluated datasets, ROC-AUC is a more reliable and primary metric, as it is threshold-independent and better reflects ranking quality under class imbalance.

ROC Analysis: Figure 3 shows ROC, Precision–Recall, and calibration plots for three methods. MPC-XGB attains ROC-AUC $=0.8157$ and PR-AUC $=0.3308$; the non-private baseline leads with ROC-AUC $=0.87$ and PR-AUC $=0.3788$, while SecureBoost is lower on ROC (AUC $=0.7913$) and slightly higher on PR-AUC (=0.3341). MPC-XGB tracks SecureBoost closely, with an advantage on ROC and a small deficit on PR-AUC. The precision–recall curve illustrates model sensitivity to the minority class, highlighting the drop in precision at lower thresholds common in datasets with rare positive events. The calibration plot shows that predicted probabilities are generally well-aligned with actual observed frequencies, though the model tends to be under confident for mid-range predicted probabilities (0.3–0.7), which could be addressed with further calibration tuning. The dashed line in the ROC curve represents random classifier performance, while in the calibration curve it indicates perfect calibration.

Scalability: To examine the system’s behavior under reduced sample sizes,

Table 3. Impact of data size on model performance for MPC-XGB on Dataset 1.

Metric	50%	30%	15%	5%
Accuracy	0.9323	0.9317	0.9171	0.9100
Precision	0.4936	0.4901	0.4015	0.3696
Recall	0.3177	0.2530	0.3848	0.4752
F1	0.4136	0.3345	0.3852	0.4158
AUC	0.8109	0.7958	0.8119	0.8002

shows performance with decreasing data fractions on the first credit scoring dataset. We find that model performance remains broadly stable as the available data are reduced. In particular, the 5% setting remains close to the 50% setting on the main evaluation metrics, with the F1 score slightly improving from $0.4136$ to $0.4158$ and AUC remaining close ($0.8109$ vs. $0.8002$). This indicates that, under our row-subsampled training setting, using only 5% of the data can still preserve competitive predictive performance while substantially reducing the cost of secure computation. We therefore report the communication and runtime costs for the 5% data setting, as it provides a better practical trade-off between efficiency and utility. We further evaluate scalability on the second credit scoring dataset (30,000 instances) by progressively reducing the available training samples.

Table 4. Impact of data size on model performance for MPC-XGB on Dataset 2.

Metric	50%	30%	15%	5%
Accuracy	0.84	0.82	0.80	0.7950
Precision	0.50	0.57	0.5512	0.5445
Recall	0.47	0.36	0.3250	0.3133
F1	0.49	0.441	0.409	0.3970
AUC	0.70	0.69	0.6823	0.6618

reports the corresponding results, showing consistent trends where MPC-XGB maintains stable predictive performance across varying data fractions, further validating the robustness of the proposed framework across datasets of different sizes.

To evaluate the stability of MPC-XGB under aggressive row subsampling, we repeated the 5% sampling setting three times using different random seeds. The results remained consistent across runs, where the model achieved Accuracy $=0.9100\pm 0.0054$ (mean ± standard deviation), precision $=0.3696\pm 0.0281$, recall $=0.4752\pm 0.0417$, F1 $=0.4158\pm 0.0226$, and AUC $=0.8002\pm 0.0073$. Overall, the results suggest that even under aggressive subsampling setting, the model retains reasonably consistent predictive behaviour while substantially reducing the secure computation required during training.

We further evaluate scalability on the second credit scoring dataset (30,000 instances) by progressively reducing the available training samples. Table 4 reports the corresponding results, showing consistent trends where MPC-XGB maintains stable predictive performance across varying data fractions, further validating the robustness of the proposed framework across datasets of different sizes.

Computation cost and efficiency: For the 5% data setting, the end-to-end training time for MPC-XGB was approximately $48,325$ s ($\approx$13.42 h). The offline phase, which pre-generates correlated randomness and preprocessing material required for secure operations, consumed approximately $28,359$ s and involved about $1.2$ TB of communication across roughly $3.22$ million protocol rounds. The online phase, which performs the actual secure evaluation of split gains during tree construction, required approximately $19,966$ s and transferred about $33.7$ GB of data over approximately $111.17$ million rounds.

Across the entire training process, the MPC protocol executed approximately $14.49$ billion secure multiplications and opened more than $50.37$ million intermediate values required for protocol coordination. In addition, communication between the data owners (AP and PP) and the MPC servers accounted for approximately $984.7$ MB over around $1.09$ million rounds. For reference, the runtime for one MPC-XGB decision tree under the same 5% data setting was approximately 2416 s ($\approx$40.3 min). These costs arise primarily from the repeated secure evaluation of candidate splits at each node of the boosted decision trees, where gradient and Hessian statistics must be aggregated securely across parties.

Because secure computation significantly increases computational and communication overhead compared to plaintext training, we adopt two design choices to keep the cost manageable. First, we employ quantile-based split selection, which limits the number of candidate thresholds evaluated for each feature and thus reduces the number of secure operations required during split evaluation. Second, we apply row subsampling, which reduces the number of instances processed at each boosting round, thereby lowering the number of MPC rounds required during training. These optimizations reduce the overall online computation cost while maintaining stable predictive performance. Although row subsampling can slightly reduce recall under severe class imbalance, it improves training efficiency and helps maintain stable model accuracy within the MPC setting.

Runtime comparison: To provide a direct runtime comparison under matched conditions, we also ran SecureBoost using the same dataset setting, feature partition, and hyperparameter configuration as the 5% MPC-XGB experiment. Under these same settings, SecureBoost completed training in approximately 1993 s (≈33.2 min), whereas MPC-XGB required approximately $48,325$ s (≈13.42 h) for end-to-end training. This higher training cost is expected, since MPC-XGB performs split evaluation through three-party replicated secret sharing and secure fixed-point computation rather than relying on lighter protected or partially revealed training procedures. In return, MPC-XGB offers stronger privacy guarantees than SecureBoost by protecting not only raw features but also labels and intermediate training statistics such as gradients and Hessians during split evaluation. Although the training phase is more expensive, this cost is typically incurred only once during model construction. In contrast, inference in MPC-XGB is efficient, as prediction is carried out locally at the AP using the trained model without requiring repeated MPC interaction. Thus, the additional cost is concentrated in training, while deployment-time prediction remains fast and practical.

Overall, MPC-XGB achieves a strong privacy–utility trade-off. The proposed framework improves precision and accuracy compared to SecureBoost and Hybrid FL while remaining competitive with non-private VFL-XGBoost. At the same time, MPC-XGB provides stronger end-to-end privacy guarantees by ensuring that labels, features, and intermediate training statistics remain protected throughout the training process.

7. Conclusions and Future Work

We presented MPC-XGB, a privacy-preserving XGBoost framework for vertical federated learning using a secure three-party protocol. The system keeps both features and labels private throughout training and prediction, and removes the asymmetry that otherwise advantages the label-holding party (AP) by executing label-dependent computations inside MPC. On a credit-scoring dataset, MPC-XGB delivers predictive performance comparable to that of non-private VFL-XGBoost and improves over SecureBoost while providing strictly stronger privacy guarantees. Compared to prior methods, our approach supports full tree learning end-to-end with no reliance on a partially trusted intermediary server. The system also remains robust on reduced data subsets, proving its applicability in practical federated settings.

Limitation: Training is time- and bandwidth-intensive; to manage time cost, we employed quantile-based split finding and per-tree row subsampling, which keep online rounds manageable in MPC.

Future work: We will focus on reducing the computational and communication cost of the framework through more efficient protocol design and system-level optimization via batching and packing of operations, adaptive fixed-point precision, and improved offline/online scheduling. We also plan to extend the framework to stronger threat models, particularly malicious adversaries, to provide stronger security guarantees in more challenging deployment settings.