SplitML: A Unified Privacy-Preserving Architecture for Federated Split-Learning in Heterogeneous Environments

Abstract

While Federated Learning (FL) and Split Learning (SL) aim to uphold data confidentiality by localized training, they remain susceptible to adversarial threats such as model poisoning and sophisticated inference attacks. To mitigate these vulnerabilities, we propose $\mathbf{SplitML}$, a secure and privacy-preserving framework for Federated Split Learning (FSL). By integrating $IND-CP{A}^D$ secure Fully Homomorphic Encryption (FHE) with Differential Privacy (DP), $\mathbf{SplitML}$ establishes a defense-in-depth strategy that minimizes information leakage and thwarts reconstructive inference attempts. The framework accommodates heterogeneous model architectures by allowing clients to collaboratively train only the common top layers while keeping their bottom layers exclusive to each participant. This partitioning strategy ensures that the layers closest to the sensitive input data are never exposed to the centralized server. During the training phase, participants utilize multi-key CKKS FHE to facilitate secure weight aggregation, which ensures that no single entity can access individual updates in plaintext. For collaborative inference, clients exchange activations protected by single-key CKKS FHE to achieve a consensus derived from Total Labels (TL) or Total Predictions (TP). This consensus mechanism enhances decision reliability by aggregating decentralized insights while obfuscating soft-label confidence scores that could be exploited by attackers. Our empirical evaluation demonstrates that $\mathbf{SplitML}$ provides substantial defense against Membership Inference (MI) attacks, reduces temporal training costs compared to standard encrypted FL, and improves inference precision via its consensus mechanism, all while maintaining a negligible impact on federation overhead.

1. Introduction

Machine Learning (ML) is a powerful tool that can solve various problems, yet it raises serious privacy concerns. Indeed, when ML models are trained on sensitive data, there is a risk that this data could be used to identify individuals or infer sensitive information about them. For instance, telecom companies implement advanced ML algorithms based on locally collected data through Security Incidents and Events Management (SIEM) to help determine potential cyber threats, protect their networks and customers’ data, and enhance security and privacy measures. Telecom data is one of the most sensitive data types, as any four location points are enough to uniquely re-identify 90% of individuals [1].

While SIEMs can be hosted on a standalone network device, they can also be deployed through cloud services offered by security service providers. These systems process logs in quasi-real-time but may also support offline log processing. They are responsible for storing, analyzing, and correlating logs. Gathered data from a standalone SIEM can include incomplete or non-representative information, leading to the inaccurate classification of incidents, thus taking improper actions that can induce severe damages [2]. Collaboration between different SIEMs is encouraged to deal with this challenge. The collaborative SIEM system enables companies to quickly identify and respond to potential threats, thereby reducing the impact of security breaches. By collaborating and sharing their expertise, companies also enhance their security posture and foster customer trust. However, this raises many issues. When a client shares its logs with other organizations, this information can be exploited, and the reported incident can be used to attack vulnerable devices. This becomes even more critical if the reported incident involves a widely used device. The use of incident information can lead to the creation of valuable target profiles for security vendors or be sold to competitors as alert reports, directly damaging the company’s brand and reputation. Indeed, distributed ML algorithms, i.e., Federated Learning (FL) [3] and Split Learning (SL) [4], enable the training of a global model on decentralized data stored on multiple client devices without sharing these data with a central entity. (Refer Appendix B.1 and Appendix B.2 for details.) This approach can benefit SIEM scenarios where sensitive proprietary data cannot be shared with peers.

Despite data being resident on the client device, confidentiality and privacy remain at risk. Distributed learning methods, including federated and split learning, provide local obfuscation but are not formal privacy mechanisms, offering no inherent guarantees of privacy. The deployment of collaborative learning in cybersecurity must account for the unique constraints of varied and limited environments, such as those found in IoT-based anomaly detection. As explored in [5], establishing a trustworthy infrastructure requires balancing privacy-preserving mechanisms with the operational challenges of Federated Learning, including node reliability and communication overhead. This work highlights the necessity of secure aggregation to mitigate privacy risks in IoT security, where individual nodes often lack the computational resources for heavy cryptographic operations.

Without these safeguards, even decentralized models remain vulnerable to membership inference and reconstruction attacks that can expose sensitive network topographies or device signatures. By situating $\mathbf{SplitML}$ within this context, we acknowledge the shift toward decentralized architectures that utilize partitioned models to protect sensitive telemetry data while maintaining the efficacy of anomaly detection across heterogeneous and resource-constrained networks. Our framework builds upon these foundational challenges by employing model partitioning to offload sensitive computations from the edge while ensuring that the central aggregation process remains cryptographically secure.

Several attacks on privacy, such as Membership Inference [6,7] and Model Poisoning [8], must be considered. These attacks aim to infer sensitive information about the training data or clients. They exploit the relationship between the updates and the private features on which they were trained. By analyzing the global model, attackers might reconstruct training data or individual contributions, even if gradients are carefully designed. This is possible because the global model embodies aggregate information from participants’ data. SL involves exchanging Intermediate Representations (IR) between participants. Analyzing these IRs, even without access to raw data, may reveal sensitive information hidden within them. An attacker can reconstruct parts of the private training data used to build the model by feeding crafted inputs and observing model outputs. Dishonest users can inject manipulated updates into the training process to steer the model toward incorrect predictions or biased outcomes. These “poisoned” updates influence the global model, affecting everyone. Attackers might tamper with their data before training their local model and then contribute to the poisoned model updates. This way, they can subtly influence the global model without directly injecting malicious updates.

It is necessary to ensure confidentiality, integrity, and privacy while enhancing collaboration. Several solutions are proposed to implement various Privacy Enhancing Technologies (PET), namely (i) privacy-preserving computation, e.g., Fully Homomorphic Encryption (FHE) and secure Multi-Party Computation (MPC), and (ii) Statistical Disclosure Control (SDC) techniques, e.g., Differential Privacy (DP). FHE enables computations on encrypted data without requiring decryption, ensuring confidential analysis, while DP perturbs data to conceal individual characteristics. (Refer Appendix A.1 and Appendix A.2).

These techniques aim to protect the privacy of both the client data and the associated model and prevent inference attacks while enabling practical model training in a distributed manner. However, their implementation in real-world scenarios brings new challenges with malicious or curious adversaries (a curious adversary is a passive entity that observes and learns from a system’s communication without altering it, whereas a malicious adversary is an active entity that can observe, alter, and inject false information into the system), which may collude to derive information regarding other participants. This paper proposes $\mathbf{SplitML}$, a general framework to enhance collaborative yet personalized neural networks. Though $\mathbf{SplitML}$ can be used for any application, we focus on intrusion detection in this paper. In our scheme, the input layer, the output layer, and the learning task are shared across clients with a possible variation in hidden layers. Clients may want a deeper network as depth helps achieve more accurate models for their local distributions or a shallow network to reduce resource usage.

$\mathbf{SplitML}$ supports heterogeneous client models, where the top layers (close to input data) of a client model that extract generic dataset features are shared with other client models. In contrast, the bottom layers (close to output labels) are more specific. The architectural designation of ‘top’ and ‘bottom’ layers in $\mathbf{SplitML}$ is chosen to reflect the flow of authority in a decentralized SIEM environment, where input-proximal layers (top) are standardized for cross-institutional alignment, while output-proximal layers (bottom) are reserved for local classification refinement.

Here, the advantages of model heterogeneity are two-fold: (1) for training, it helps increase model accuracy over non-IID data, and (2) diverse predictions can be obtained for inference through consensus. We assume clients share features and labels while keeping their data and ML models private in a semi-honest threat model. (A semi-honest threat model (also known as “honest-but-curious”) assumes that an adversary will faithfully follow a protocol’s instructions but will attempt to learn as much private information as possible from the messages they observe.) Clients collaborate to train the top layers with the help of a central server and keep their bottom layers private. In $\mathbf{SplitML}$, clients train their models locally on their private data and collaborate to train only a set of generic (top) layers with FL by sharing encrypted weights. Thus, aggregation is performed on shared layers, although they might have been trained on different (overall) topologies.

By abstraction, the use of FL increases the size of the training dataset, so the feature extraction by the first layers will be more accurate. Clients have the same architecture for the output layer but may have different hidden layers to facilitate a personalized model. As such, they refrain from sharing them with other clients and ensure they will not be able to get any information about their dataset.

$\mathbf{SplitML}$ is fundamentally different from Vertical Federated Learning (VFL) in how the data is partitioned: VFL is a feature-partitioned approach, designed for scenarios where multiple clients share the same set of samples (users/entities) but each client possesses a different portion of the feature space (${\mathbf{A}}_1,{\mathbf{A}}_2,\dots$). In contrast, $\mathbf{SplitML}$ is built upon Federated Split Learning, which is a sample-partitioned approach similar to Horizontal Federated Learning (HFL), where clients possess the same feature set ($\mathbf{A}$) but hold different, non-overlapping samples (${\mathbf{D}}_1,{\mathbf{D}}_2,\dots$). This sample partitioning is central to $\mathbf{SplitML}$’s novelty, as it allows clients to use private bottom model layers for local feature extraction while collaboratively aggregating and training the common top layers, an architecture fundamentally different from the feature-split collaboration utilized by VFL.

Fully Homomorphic Encryption (FHE) represents a comprehensive cryptographic solution that facilitates the direct execution of arbitrary computational functions on ciphertexts. Originally conceptualized by Gentry in 2009 [9], FHE enables data owners to utilize untrusted cloud computing services for data analysis while ensuring that sensitive information is never exposed in plaintext [10,11,12,13,14,15,16,17,18,19], while FHE provides robust security, standard schemes frequently encounter limitations regarding substantial computational latency and significant ciphertext expansion. To mitigate these performance constraints, the Cheon-Kim-Kim-Song (CKKS) scheme [20] was introduced in 2017 as a specialized variant of FHE. Unlike traditional exact arithmetic schemes, CKKS is specifically optimized for approximate arithmetic on real and complex numbers. By treating encryption noise as part of the significant figures of the message, CKKS achieves high efficiency in numerical analysis, machine learning training, and statistical modeling. In this work, we employ the OpenFHE library [21,22] to implement a hybrid cryptographic architecture. Specifically, we utilize multi-key CKKS to secure the collaborative weight aggregation during the training phase and single-key CKKS to maintain privacy during the inference process.

Indistinguishability under Chosen-Plaintext Attack ($IND-CPA$) is the minimum security standard for modern Public-Key Cryptography (PKC), formalized by a game where an attacker, given only the Public Key (PK), must fail to distinguish between the ciphertexts of two chosen messages with a success probability better than random chance; this guarantee ensures confidentiality and necessitates the use of probabilistic encryption. However, Li and Micciancio recently showed that the standard $IND-CPA$ model is insufficient for the CKKS FHE scheme in multi-party contexts, demonstrating a key recovery attack that is feasible when decryption results are shared among multiple parties, such as in a threshold FHE setting, thereby requiring a stronger adversarial security model than $IND-CPA$ to maintain the confidentiality of the Secret Key (SK).

$IND-CP{A}^D$ protects against secret-key recovery under shared decryptions; it does not, by itself, prevent all forms of gradient leakage, which is why model partitioning and consensus inference are also required.

To mitigate this, OpenFHE subsequently extended the original CKKS scheme to operate under a stronger adversarial model that permits the sharing of decryption results among multiple parties, choosing a default configuration designed to tolerate a relatively large number of decryption queries involving the same or related ciphertexts. Specifically, OpenFHE’s CKKS implementation utilizes a countermeasure known as noise flooding, which involves adding a large, random Gaussian noise to the ciphertext just before decryption, mathematically achieving the security notion of $IND-CP{A}^D$ (a principle based on Differential Privacy). This method ensures that the statistical noise distribution of the output is independent of the SK, a guarantee that cannot be provided by the intrinsic, often data-dependent, noise generated during homomorphic operations.

Existing approaches for collaborative training, such as FL, SL, and their hybrids like Federated Split Learning (FSL) and SplitFed Learning (SFL), primarily offer privacy through data localization, but remain fundamentally vulnerable to various inference and poisoning attacks. We discuss previous works for privacy-preserving tasks in more detail in appendices and briefly compare our approach $\mathbf{SplitML}$ (Figure 1c) with FL (Figure 1a), SL (Figure 1b), and the combinations of two approaches.

While primitives like SecAgg [23] use secure multi-party computation to protect the model weights during aggregation from the central server, they do not defend against an adversarial client’s ability to extract sensitive training data or exploit shared decryption results in advanced settings like threshold FHE. This limitation highlights a critical gap in existing secure aggregation schemes, which often focus on server-side privacy while overlooking the risks posed by malicious or compromised participants within the federation.

To further clarify the technical positioning of our framework, it is essential to compare $\mathbf{SplitML}$ with recent advancements in heterogeneous training, such as the S2FL (Heterogeneous Split Federated Learning) framework. S2FL [24] is primarily designed to enhance training efficiency and accuracy by partitioning models to accommodate hardware diversity among clients, while S2FL successfully addresses resource-aware splitting and convergence speed, $\mathbf{SplitML}$ extends these concepts by integrating a robust “Privacy-via-Encryption” layer using Multi-key FHE. This cryptographic integration ensures that the weights of the shared layers remain encrypted throughout the entire aggregation cycle, preventing the central server from inspecting the model parameters.

Unlike S2FL, which focuses on the training logistics of a partitioned model, $\mathbf{SplitML}$ introduces a unique collaborative inference phase where consensus is reached through encrypted activations. This allows our framework to not only handle architectural heterogeneity during training but also to provide a secure mechanism for resolving classification ambiguities through a cross-institutional consensus that is absent in standard S2FL implementations. By leveraging the diversity of local bottom layers, $\mathbf{SplitML}$ ensures that the global model benefits from various localized feature extraction strategies without requiring the raw data or private model segments to be exposed to peer institutions.

$\mathbf{SplitML}$ generalizes FSL under the rigorous $IND-CP{A}^D$ security model using a combination of FHE with DP. Unlike traditional perturbation methods that inject calibrated Gaussian noise to satisfy $(ϵ,\delta )$-Differential Privacy, $\mathbf{SplitML}$ adopts a ’Privacy-via-Encryption’ approach. By utilizing the noise flooding techniques inherent in the OpenFHE CKKS implementation, we ensure that the inherent rounding errors of approximate arithmetic obscure the individual contributions. This provides a functional barrier against inference attacks while avoiding the significant accuracy degradation typically observed when adding external noise to satisfy a formal privacy budget.

This approach not only provides robust security against well-known threats like Membership Inference (MI) attacks but also enables novel features, such as supporting heterogeneous client model architectures and utilizing multi-key CKKS for secure aggregation and consensus-based inference. Our experiments show that the (minimum) member-accuracy for the MI attack on the top layers (for any number of shadow models) was about 37%. In contrast, for bottom layers, they were as low as 19%.

Our contributions in this paper can be summarized as follows:

• We formalize FL and SL and present $\mathbf{SplitML}$, a fused FL (for training) and SL (for inference) partitioned between ML model layers to reduce information leakage. The novelty stems from clients collaborating on partial models (instead of full models in FL) to enhance collaboration while reducing privacy risks; while federation helps improve feature extraction, horizontal splitting allows entities to personalize their models concerning their specific environments, thus improving results.
• $\mathbf{SplitML}$ implements multi-key FHE with DP during training to protect against clients colluding with each other or a server colluding with clients under an honest-but-curious assumption. $\mathbf{SplitML}$ reduces time compared to training in silo while upholding privacy with $IND-CP{A}^D$ security.
• We propose a novel privacy-preserving counseling process for inference. An entity can request a consensus by submitting an encrypted classification query using single-key FHE with DP to its peers.
• We empirically show that $\mathbf{SplitML}$ is robust against various threats such as poisoning and inference attacks.

This paper is organized as follows. First we introduce the threat model in Section 2.1 and detail the proposed framework in Section 2.2. Then, Section 3 discusses security threats concerning the identified adversaries, and Section 4 reviews the empirical evidence before concluding in Section 6. We briefly discuss background and related work in the Appendix.

2. Our Framework

The following subsections define the security boundaries and structural layers (throughout this paper, ‘top layers’ refer to input-proximal layers, and ‘bottom layers’ to output-proximal layers) that enable $\mathbf{SplitML}$ to facilitate collaborative learning.

2.1. Threat Model

$\mathbf{SplitML}$ follows the standard client-server model, where trust is established before training starts (e.g., in the log analysis domain, the organizations (clients) may opt out of the corresponding SIEM platform (server) if they host adversarial clients). In an ideal scenario, all $K+1$ participants, including the server and the K clients, act honestly and perform their assigned tasks. For each training round, clients train their local models on their own (unencrypted) data, and the server combines the (encrypted) weights for shared layers. After training, all clients perform inference locally (unencrypted) and do not share any information with their peers or the server. However, a client may want to perform consensus on a subset of data to benefit from heterogeneous models, in which case a client sends (encrypted) smashed data to its peers.

Our system only considers a scenario where the server, while curious, passively observes updates and cannot modify anything. It does not address a more robust scenario where a malicious server actively collaborates with clients and disrupts the training process, while honest-but-curious clients follow the protocol, they could potentially collaborate to learn about specific individuals in the training data (membership inference attack). Malicious clients, however, may deviate from the protocol for harmful purposes and send misleading updates (model poisoning attack) or attempt to extract the entire model from a particular client. Communication (e.g., exchange of weights and activations) is encrypted in $\mathbf{SplitML}$, and our scheme is secure under collusion if the colluding clients $T^{\prime }$ are lower than T clients required for fused decryption (fused decryption in multi-key FHE efficiently combines partial decryption results from multiple parties into a single plaintext) under multi-key FHE ($T=K$ for multi-key FHE and $T\le K$ for threshold-key FHE). Moreover, DP offers plausible protection in case $T-1$ clients collude with the server to uncover the model weights of the target.

In contrast to conventional Federated Learning (FL), where the central server maintains full visibility of the global model architecture, $\mathbf{SplitML}$ adopts the partitioning principles of Split Learning (SL) to strictly limit server-side exposure. A defining security enhancement in $\mathbf{SplitML}$ is the preservation of data confidentiality during the training phase, wherein the server is systematically denied access to plaintext smashed data. Upon completion of collaborative training, the framework offers flexible execution paths: clients may perform inference locally to ensure maximum autonomy, or alternatively, engage in collaborative inference. In the latter mode, the framework shifts to a decentralized paradigm where clients delegate encrypted smashed data to self-selected peers via single-key FHE. This peer-to-peer consensus mechanism effectively eliminates the server’s involvement in processing intermediate results, further mitigating the risk of centralized data leakage.

2.2. Proposed Architecture

Our proposed solution $\mathbf{SplitML}$ (Definition 1) is a Federated Split Learning (FSL) approach where each client shares output labels (classes) L and input attributes (features) A but may have different ML models (hidden layers), except for the top common layers and the output layer. Clients train all their models locally on their private data and send the weights for the shared layers encrypted with an FHE scheme to a federated server S for each training round r. The server makes the averaging in the encrypted domain from the participants during that round and updates the clients.

Definition 1.

$\mathbf{SplitML}$ is a privacy-preserving, secure collaborative scheme for training and inference of a partially shared ML model deploying FL using an FHE scheme with $K>1$ clients and an FL server S. A client local model $M_k,k\in \{1,\dots ,K\}$ has total $n+t_k$ layers, where $n\ge 1$ are the shared top layers up to the ’cut/split layer’ q and the rest of the $t_k\ge 1$ are private (bottom) layers including a common output layer. For each training round $r\in \{1,\dots ,R\}$, participants $1<m\le K$ train $M_k$ with their private (iid or non-iid) dataset $D_k$ with common attributes A and common labels L. After each training round, the participants send their model weights encrypted with FHE for n layers ($[1,\dots ,q]$) to S, where the server aggregates the weights with some averaging algorithm and sends the encrypted updates to participants of the next round. Training continues till all $M_k$ achieve an acceptable accuracy or fixed rounds R. The clients can further collaborate to form a consensus using FHE during inference for the model outputs with low confidence close to the classification boundary for some threshold $\lambda >0$.

A primary advantage of $\mathbf{SplitML}$ is realized during the inference phase, where participants leverage model diversity to enhance classification reliability. This is particularly valuable in scenarios where a network domain administrator identifies specific log records for which the local model yields low confidence scores, suggesting a high probability of false positives or negatives. To resolve these ambiguities, the administrator reprocesses the suspicious logs and transmits the encrypted activations from the cut (The cut layer serves as the structural boundary between the shared common layers and the localized private model segments.) layer to peer administrators. Upon receiving these encrypted activations ($En{c}_{PK}(\nabla q)$), peer administrators execute a forward pass through their respective model segments, which may possess unique architectural variations or have been refined on different data distributions. The resulting encrypted outputs are returned to the requester, who performs decryption using a single-key FHE secret key ($De{c}_{SK}$). A final classification is then established through a consensus mechanism based on either the majority of Total Labels (TL) or the highest aggregate Total Prediction (TP) scores. This architecture allows $\mathbf{SplitML}$ to operate in both a multi-institutional capacity, by aggregating data from various entities during training, and a cross-institutional capacity, by utilizing diverse models for inference. To maintain end-to-end security, the framework employs a hybrid cryptographic approach: encrypted layer weights are used for secure aggregations during the collaborative training phase, while encrypted activations facilitate the inference consensus. Consequently, $\mathbf{SplitML}$ effectively integrates the collaborative weight updates of Federated Learning for model development with the partitioned, privacy-preserving execution of Split Learning for inference.

• $\mathbf{SplitML}$ can FL with a parameter $\beta$ over model layers, where $\beta$ controls the proportion of layers to collaborate on. Thus, FL is realized when $k\in K$ clients collaborate to train all layers of the global ML model M; hence, $\beta =1, \left|M\right|=n, t=0$. A value of $\beta =0$ indicates no collaboration, thus $|M_k|=t_k, n=0$.
• Transfer Learning (TL) [25,26,27,28,29] is realized for an architecture (e.g., Convolutional Neural Networks-Artificial Neural Networks (CNN-ANN) model) where K distinct clients collaborate to train the first n (e.g., convolution) layers for feature extraction. For inference after training, clients retrain the rest of the $t_k$ (e.g., Fully Connected (FC)) layers of their ML model $M_k$ till convergence on their private data without updating the first n layers (effectively freezing the feature extracting CNN layers).

We compare $\mathbf{SplitML}$ with existing approaches regarding resource requirements in

Table 1. Comparison of computation and communication costs per client in different schemes. Assuming $\left|n\right|<\left|M\right|$ (usually cardinality follows: $\left|q\right|<\left|n\right|<\left|M\right|$), $\mathbf{SplitML}$ can help save bandwidth over FL. Here, $\alpha \left|M\right|$ denotes the fraction of ML parameters of model M with a client.

Method	Computation	Communication
FL [3]	$D_k\left|M\right|$	$2\left|M\right|$
SL [4]	$D_k\alpha \left|M\right|$	$2\left|q\right|$
SFL [30]	$D_k\alpha \left|M\right|$	$2\left|q\right|+2\alpha \left|M\right|$
$\mathbf{SplitML}$	$D_k\left|M_k\right|$	$2n$

. For each client k in $\mathbf{SplitML}$, the computation cost varies based on the specific local model segment $M_k$ rather than the fixed cost associated with a global model M in standard FL. This flexibility allows nodes with disparate hardware capabilities to participate in the federation by adjusting the depth or complexity of their private bottom layers.

Furthermore, $\mathbf{SplitML}$ achieves significant bandwidth savings compared to FL, as clients only transmit model updates for the top (shared) layers n rather than the full parameter set where $\left|M\right|>n$. Unlike SplitFed Learning (SFL) [30], our framework supports clients with entirely different model architectures and encrypts all shared information, which, to our knowledge, represents a simultaneous first in the literature. This combination of architectural heterogeneity and cryptographic confidentiality ensures that the system remains both scalable and secure against honest-but-curious aggregators. More details about SFL and its integration with our approach are described in Appendix B.3.

2.3. Key Generation

Key generation is an offline setup phase in $\mathbf{SplitML}$ before training. We use the OpenFHE [31] library to implement these procedures.

2.3.1. Public and Secret Keys

Multi-key FHE uses multiple encryption keys, one for each party. This makes it more difficult for any one party to decrypt the ciphertext, even if they are malicious. For a more secure (under collusion), multi-key approach, all K clients participating in the training generate their Public-Private key pairs $P{K}_k,S{K}_k,k\in \{1,\dots ,K\}$ in sequence (Refer to Algorithm 1 (For simplicity, we denote various OpenFHE API for key generation as a variation of $KeyGen$ function.)). A shared Public Key $PK=P{K}_K$ is generated using each $P{K}_k$.

Algorithm 1 Public and Secret Keys Generation

Input: Each client $C_k$ performs $KeyGen\left(\right)$ iteratively Output: Public and private keypairs $P{K}_k,S{K}_k$ for each client $C_k$   1: $P{K}_1,S{K}_1\leftarrow KeyGen\left(\right)$   2: for each client k in $\{2,\dots ,K\}$ do   3:       $P{K}_k,S{K}_k\leftarrow KeyGen\left(P{K}_{k-1}\right)$   4: end for   5: $PK=P{K}_K$

2.3.2. Evaluation Key for Addition

Similarly, each operation-specific key, such as $E{K}_{Add}$ for addition, $E{K}_{Mult}$ for multiplication, and $E{K}_{Dec}$ for fused decryption, is generated from secret shares of all the K clients (or some clients $T<K$ as per threshold). Generating the evaluation key for addition is a two-pass process, as shown in Algorithm 2. In the first iteration, all clients generate their addition keys $E{K}_{Ad{d}_k}$ using their secret and public keys $S{K}_k\mathrm{and}P{K}_k$. In the second iteration, the final shared addition key $E{K}_{Add}$ is calculated from client keys $E{K}_{Ad{d}_k}\mathrm{and}P{K}_k$.

Algorithm 2 Evaluation Keys Generation for Addition

Input: Keypairs $P{K}_k,S{K}_k$ for each client $C_k$ Output: Evaluation key for Addition $E{K}_{Add}$   1: $E{K}_{Ad{d}_1}\leftarrow KeyGen\left(S{K}_1\right)$   2: for each client k in $\{2,\dots ,K\}$ do   3:       $E{K}_{Ad{d}_k}\leftarrow KeyGen(E{K}_{Ad{d}_1},S{K}_k,P{K}_k)$   4: end for   5: $E{K}_{Ad{d}_{temp}}\leftarrow KeyGen\left(E{K}_{Ad{d}_1}\right)$   6: for each client k in $\{2,\dots ,K\}$ do   7:       $E{K}_{Ad{d}_{temp}}\leftarrow KeyGen(E{K}_{Ad{d}_k},E{K}_{Ad{d}_{temp}},P{K}_k)$   8: end for   9: $E{K}_{Add}=E{K}_{Ad{d}_{temp}}$

2.3.3. Evaluation Key for Multiplication

The evaluation key for multiplication $E{K}_{Mult}$ is generated in four iterations, as shown in Algorithm 3. The first two iterations are similar to the process described for the additive evaluation key in Algorithm 2. In the first pass, all clients K generate their local multiplication keys $E{K}_{Mul{t}_k},k\in \{1,\dots ,K\}$ using their respective secret keys $S{K}_k$. During the second pass, clients calculate a temporary collective component $E{K}_{Mul{t}_{temp}}$ by combining their local keys $E{K}_{Mul{t}_k}$ with the collective public key $PK$. In the third pass, clients generate specialized shares $E{K}_{Mul{t}_{tem{p}_k}}$ using the temporary collective key, their individual secret keys $S{K}_k$, and the collective public key $PK$. This iterative process ensures that no single participant can reconstruct the full evaluation key or the secret keys of other parties. Finally, all $E{K}_{Mul{t}_{tem{p}_k}}$ shares are fused at the aggregator to yield the final evaluation key $E{K}_{Mult}$, which enables the homomorphic multiplication of ciphertexts encrypted under different keys. This four-step synchronization is essential for maintaining the security of the multi-party CKKS scheme, as it allows for secure relinearization without compromising the confidentiality of the underlying secret parameters.

2.4. Training Phase

$\mathbf{SplitML}$ requires that all K clients share common data attributes A, output labels L, model hyperparameters-batch size $bs$ and learning rate $lr$, output (last) layer, and the model structure up to the first (top) n layers before the training begins. Clients can have a variable architecture for hidden layers, except the output layer, where the total (bottom) layers of a client are $t_k\ge 1$, $k\in \{1,\dots ,K\}$, including the last layer, and the total layers of a client model are ${\left|M\right|}_k=n+t_k\ge 2$. We briefly justify using multi-key (or threshold (Multi-key FHE requires all participants to be online and share their partial decryptions to evaluate the fused decryption, which may be infeasible for organizations distributed over the internet. Threshold-FHE eases this by requiring T out of K, $T\le K$ partial decryptions to calculate fused decryption.)). FHE over single-key FHE in collaborative training.

Algorithm 3 Evaluation Keys Generation for Multiplication

Input: Keypairs $P{K}_k,S{K}_k$ for each client $C_k$ Output: Evaluation key for Multiplication $E{K}_{Mult}$   1: $E{K}_{Mul{t}_1}\leftarrow KeyGen\left(S{K}_1\right)$   2: for each client k in $\{2,\dots ,K\}$ do   3:       $E{K}_{Mul{t}_k}\leftarrow KeyGen(E{K}_{Mul{t}_1},S{K}_k)$   4: end for   5: $E{K}_{Mul{t}_{temp}}\leftarrow KeyGen\left(E{K}_{Mul{t}_1}\right)$   6: for each client k in $\{2,\dots ,K\}$ do   7:       $E{K}_{Mul{t}_{temp}}\leftarrow KeyGen(E{K}_{Mul{t}_{temp}},E{K}_{Mul{t}_k},P{K}_k)$   8: end for   9: for each client k in $\{K,\dots ,1\}$ do 10:       $E{K}_{Mul{t}_{tem{p}_k}}\leftarrow KeyGen(S{K}_k,E{K}_{Mul{t}_{temp}},P{K}_K)$ 11: end for 12: $E{K}_{Mul{t}_{temp}}\leftarrow KeyGen(E{K}_{Mul{t}_{tem{p}_K}},E{K}_{Mul{t}_{tem{p}_{K-1}}})$ 13: for each client k in $\{K-2,\dots ,1\}$ do 14:       $E{K}_{Mul{t}_{temp}}\leftarrow KeyGen(E{K}_{Mul{t}_{tem{p}_k}},E{K}_{Mul{t}_{temp}})$ 15: end for 16: $E{K}_{Mult}=E{K}_{Mul{t}_{temp}}$

2.4.1. Single-Key FHE

In the single-key FHE training (Figure 2a), one of the clients is chosen (at random) to generate the homomorphic encryption parameters: Public Key $PK$ to encrypt client model updates, Secret Key $SK$ to decrypt averaged weights, and Evaluation Key $EK$ to perform averaging in the cipher domain. $PK$ and $SK$ are shared with all clients and $EK$ is shared with the FL server. All clients K share their encrypted weights under $PK$ for shared layers to server S, and S averages the weights using $EK$.

Clients can decrypt this result using shared secret $SK$ and update their models. However, single-key FHE is insecure in a multiparty setting because it allows any party with the key to decrypt the entire ciphertext. This means that if one party is malicious, they can collude with the other parties to decrypt the ciphertext and learn the secret data. To address this security vulnerability, multi-key FHE was proposed.

2.4.2. Multi-Key FHE

The multi-key (MK-FHE) training procedure is detailed in Algorithm 4 and Figure 2c. For every training round $r\in \{1,\dots ,R\}$, all clients K (alternatively, several participants $m,T\le m\le K$ are chosen in threshold-FHE) perform forward and backward propagation on their entire models $M_k$ on their private data $D_k$ and share the encrypted model updates with a common public key $PK$ for the first n shared layers to the central server S.

S uses FedAvg [3] (or other federated averaging algorithms) to calculate the global model weights and share them with all clients. To calculate the averaged weights in the encrypted domain, S uses evaluation keys for addition $E{K}_{Add}$ and multiplication $E{K}_{Mult}$ to add all encrypted weights received from the clients and multiply with $1/K$ to average. After receiving the encrypted result, clients partially decrypt the results with their secret keys $S{K}_k$, and S generates fused decryption from partial descriptions using an evaluation key for fused decryption $E{K}_{Dec}$ to get the final result in plaintext. Clients update the weights of top layers accordingly before the next training round begins. Training continues till all the clients achieve their target accuracy or the maximum limit of rounds R.

Algorithm 4 Training

Input: $D_k[A,L],bs,lr,PK,S{K}_k,E{K}_{Add},E{K}_{Mult},E{K}_{Dec}$ Output: Trained models $M_k$ for each client $C_k$   1: for each round r in $\{1,\dots ,R\}$ do   2:       for each client k in $\{1,\dots ,K\}$ do   3:             Client $C_k$ trains model $M_k(D_k[A,L],bs,lr)$   4:       end for   5:       Server S encodes a vector $V_{Mult}$ with values $1/K$   6:       Server S encrypts $V_{Mult}$ with $PK$   7:       for each shared layer in $\{1,\dots ,n\}$ do   8:             for each client k in $\{1,\dots ,K\}$ do   9:                   Client $C_k$ encrypts layer weights with $PK$ 10:             end for 11:             Server S adds encrypted vectors to $V_{Add}$ with $E{K}_{Add}$ 12:             Server S multiplies $V_{Add}$ to $V_{Mult}$ with $E{K}_{Mult}$ 13:             for each client k in $\{1,\dots ,K\}$ do 14:                   Client $C_k$ partially decrypts $V_{Mult}$ with $S{K}_k$ 15:             end for 16:             Server S generates fused decryption using $E{K}_{Dec}$ 17:             for each client k in $\{1,\dots ,K\}$ do 18:                   Client $C_k$ sets layer weights from fused decryption 19:             end for 20:         end for 21: end for

2.5. Inference Phase

After the training, all clients should have converged models with their target accuracy. The top n layers have the same architecture and weights across clients, facilitating transfer learning and consensus. We propose a novel consensus approach in the encrypted domain as detailed in Algorithm 5 and Figure 2b. For prediction, any client $C_j$ can choose to perform a consensus in the encrypted domain for the samples $D_j^{\prime }$ close to the classification boundary or the range with high false positives/negative occurrences (After federated training concludes, a client may observe the output range for false predictions and choose to perform consensus for the samples falling in this range to improve the inference accuracy. For $Sigmoid$ as $f\left(z\right)$, we choose $\lambda =0.05$ for boundaries $0.05$ (label 0) and $0.95$ (label 1) by simulation. Since all the false predictions were from ranges $[0.00,0.10]$ and $[0.90,1.00]$. For instance, clients may use the $Sigmoid$ activation function $f\left(z\right)\in [0,1]$ for the output layer for a binary log classification where 0 indicates a ‘normal’ scenario and 1 is an ‘anomaly’. A client $C_j$ may choose a $\lambda =0.10$ and collect samples $D_j^{\prime }$ for which $f\left(z\right)\in [0.40,0.60]$, as the classification boundary is drawn at $0.50$. First, a client $C_j$ generates single-key FHE keys $P{K}_j^{\prime },S{K}_j^{\prime },E{K}_j^{\prime }$ (different from multi-key FHE used in training) and shares $E{K}_j^{\prime }$ with consensus peers. $C_j$ calculates forward activations up to cut layer q for $D_j^{\prime }$ and encrypts forward activations $\nabla q_{{D^{\prime }}_j}$ using $P{K}^{\prime }$ for these samples to chosen $m^{\prime }-1$ peers. Peers participating in the consensus receive $E{K}^{\prime }$ and $\nabla q_{{D^{\prime }}_j}$ and send either the encrypted predicted label or values after performing homomorphic calculations on their bottom layers. The client decrypts these results using its secret key $S{K}^{\prime }$ and chooses a label based on the majority vote. We propose two variants for consensus results: total labels (TL) or prediction scores (TP):

• The (voting) clients send a classification label (TL), and the consensus is done on a label majority.
• The (voting) clients send a result of the final activation function (TP), which is summed up, and the label is chosen if the summation is higher than some required threshold.

In ensemble learning, majority voting (hard voting) and soft voting combine predictions from multiple models. TL represents majority voting, and TP represents soft voting. For a majority vote, each model “votes” for a class, and the most popular choice wins. It is simple but ignores confidence levels. Soft voting is more nuanced, considering each model’s “certainty” by averaging their predicted probabilities for each class. This can be more accurate, especially when models disagree slightly or deal with imbalanced data.

Consider a consensus setup with a $Sigmoid$ activation with $f\left(z\right)\in [0,1]$ for $m^{\prime }=10$ peers, with $f\left(z\right)=0$ representing a “normal” class and $f\left(z\right)=1$ as an “anomaly” for a binary log classification. In a TL consensus, a sample is considered abnormal if a majority, e.g., 6 out of 10 participants ($50\%$ or more), classify a sample as 1. Meanwhile, all the predicted values are summed up for a TP consensus. A sample may be considered anomalous if the result exceeds some chosen threshold, e.g., $5.1$ for 10 participants, given that the classification boundary is drawn at $f\left(z\right)=0.5$ (for $Sigmoid$) and $5.1>(0.5\ast 10)$.

Algorithm 5 Inference

Input: Data subset ${D^{\prime }}_j$ from client $C_j$ Output: Class labels or predictions scores from $m^{\prime }$ clients   1: Client $C_j$ creates a subset ${D^{\prime }}_j\subseteq D_j$ of local data   2: Client $C_j$ generates $P{K}_j^{\prime },S{K}_j^{\prime },E{K}_j^{\prime }\leftarrow KeyGen\left(\right)$   3: Client $C_j$ shares $E{K}_j^{\prime }$ with other $m^{\prime }$ consensus clients   4: Client $C_j$ generates activations $\nabla q_{{D^{\prime }}_j}$ for ${D^{\prime }}_j$ from cut layer q   5: Client $C_j$ encrypts activations $\nabla q_{{D^{\prime }}_j}$ with $P{K}_j^{\prime }$   6: for each consensus client $h\in \{1,\dots ,m^{\prime }\}$ do   7:       Client $C_h$ receives encrypted $\nabla q_{{D^{\prime }}_j}$ from $C_j$   8:       Client $C_h$ performs calculation on $\nabla q_{{D^{\prime }}_j}$ with $E{K}_j^{\prime }$   9:       Client $C_h$ sends encrypted result back to Client $C_j$ 10:       Client $C_j$ decrypts results received from $C_h$ with $S{K}_j^{\prime }$ 11: end for 12: Client $C_j$ considers a majority based on decrypted labels or prediction values received from consensus clients

2.6. Differential Privacy

Multi-Key FHE is vulnerable to collusion and shared model updates (training) and cut layer activations (inference) can reveal substantial information about the local datasets in a federated setting. We use Differential Privacy (DP) to protect the privacy of honest clients. DP can be applied to local model updates before aggregation at each training round (or to activations during inference). This application helps mitigate inversion and inference attacks with minimal impact on model utility. However, DP may not prevent Extraction attacks or reduce the severity of the privacy violations that extraction enables [32]. Model Extraction attacks can be mitigated using techniques such as model compression, obfuscation, and watermarking [33] and security measures in the deployment environment. $\mathbf{SplitML}$ reduces privacy leakage under the honest-but-curious model with collusion by cryptographic guarantees of $IND-CP{A}^D$ secure FHE. Li and Micciancio [34] showed that approximate FHE schemes such as CKKS [20] can leak information about the secret key. In some scenarios, the $IND-CPA$ model may not be sufficient for the CKKS scheme because a decryption result can be used to perform a key recovery attack. CKKS decryptions give direct access to the secret key given a ciphertext and decryption since the user gets $\widetilde{\mathbf{ct}}=(\tilde{\mathbf{a}}\tilde{\mathbf{s}}+\tilde{\mathbf{m}}+\tilde{\mathbf{e}},-\tilde{\mathbf{a}})$ and its decryption is $\tilde{\mathbf{m}}+\tilde{\mathbf{e}}$. As a solution [35] we employ decryption given a CKKS ciphertext $\widetilde{\mathbf{ct}}=(\widetilde{{\mathbf{c}}_{\mathbf{0}}},\widetilde{{\mathbf{c}}_{\mathbf{1}}})$ as a randomized procedure, $\mathbf{Dec}\left(\widetilde{\mathbf{ct}}\right):\mathrm{Sample}\tilde{\mathbf{z}}\leftarrow {\tilde{\mathbf{D}}}_{\tilde{\mathbf{R}},\sigma }.\mathrm{Return}\widetilde{{\mathbf{c}}_{\mathbf{0}}}+\widetilde{{\mathbf{c}}_{\mathbf{1}}}\tilde{\mathbf{s}}+\tilde{\mathbf{z}}\left(\mathbf{mod}\tilde{\mathbf{q}}\right)$, where ${\tilde{\mathbf{D}}}_{\tilde{\mathbf{R}},\sigma }$ is a discrete Gaussian over the polynomial ring, and $\mathbf{\sigma }$ is a standard deviation. For $\tilde{\mathbf{s}}>\mathbf{0}$ bits of statistical security, $\mathbf{\sigma }=\sqrt{\mathbf{12}\tau }{\mathbf{2}}^{\tilde{\mathbf{s}}/\mathbf{2}}\widetilde{\mathbf{ct}}.\tilde{\mathbf{t}}$, where, $\tau$ is the number of adversarial queries expected and $\widetilde{\mathbf{ct}}.\tilde{\mathbf{t}}$ is the ciphertext error estimate.

This attack applies to the setting where multiple parties must share decryption results, e.g., in the multi-key (or threshold) FHE setting. By default, OpenFHE [31] chooses a configuration to prevent passive attacks where many decryption queries of the same or related ciphertexts can be tolerated (The lower bound for the tolerated number of such decryption queries is ${\tilde{\mathbf{N}}}_{\tilde{\mathbf{d}}}=\mathbf{128}$). For more robust adversarial models, the number of shared decryptions of the same or related ciphertexts can be increased at the cost of precision.

A recent investigation [36] into using homomorphic encryption’s intrinsic noise growth for DP found that this noise is highly dependent on the input messages, leading to potential privacy leakage. This case study showed that while a relaxed precision parameter could achieve a reasonable privacy budget ($ϵ<0.5$) over 50 iterations when message dependence was ignored, accounting for this dependence dramatically increased the leakage, resulting in a much worse privacy budget ($ϵ\approx 2$) over the same iterations. To provide robust, localized protection for honest clients against collusion between curious clients and the server, $\mathbf{SplitML}$ employs a two-fold privacy-enhancing mechanism during encryption that is designed to persist because the noise is locally generated by each client, not collaboratively or centrally added. (We do not manually inject additional DP noise into the system; instead, we rely on two inherent features of the CKKS scheme for DP guarantees: the default OpenFHE configuration for noise flooding and the intrinsic noise generated during CKKS operations. Alternatively, for FHE schemes like BGV or BFV, achieving DP would typically involve adding extra noise directly to the model updates before they are encrypted.) This dual protection relies on: (1) the intrinsic errors inherent to the CKKS encryption scheme, and (2) the extra noise added via noise flooding through the default OpenFHE configuration, which is the mechanism used to achieve the stronger $IND-CP{A}^D$ security against key-recovery attacks.

3. Security Analysis

The following results analyze the efficacy of $\mathbf{SplitML}$ in mitigating both model poisoning and inference-based privacy leaks.

3.1. Model Poisoning Attacks

The integrity of decentralized learning is frequently challenged by Byzantine behaviors, wherein compromised participants submit erroneous or malicious weight updates. As established by [8], the traditional reliance on arithmetic mean functions for weight aggregation presents a significant vulnerability; a singular Byzantine adversary can disproportionately influence the global parameters, effectively derailing the convergence process. This vulnerability is often exploited through model poisoning strategies, where an attacker deliberately manipulates local model states.

By injecting backdoors or intentionally skewed gradients into the shared architecture, these adversaries can compromise the global model’s reliability or introduce hidden triggers that remain dormant until activated during inference. $\mathbf{SplitML}$ benefits from split learning in the absence of a global model. An adversary may only influence the top (shared) layers for feature extraction and can not significantly impact the outcome of an honest client model, as the bottom (personalized) layers will compensate for the propagated errors of shared layers. The local model will be trained for multiple rounds (epochs) until the desired accuracy is achieved. In the following, we discuss different experiments and analyze the robustness of our scheme against model poisoning attacks with different settings.

First, we experimented on three small Neural Network (NN) models on a binary log classification problem. All three clients collaborate to update weights for the first layer of 4 neurons with ReLU activation. Clients have the last (output) layer with two neurons and $Softmax$ activation in common, where one neuron corresponds to the ‘normal’ and another to the ‘anomaly’ class. Model-1 has a hidden layer of 2 neurons with $ReLU$ activation, model-2 does not have any hidden layers, and model-3 has two hidden layers with two neurons each; the prior hidden layer has $ReLU$, and the later hidden layer has $Sigmoid$ activation. We have used modified Loghub [37] HDFS_1 labeled data from Logpai; refer to Section 4.1 for details. We experimented with different configurations of $\mathbf{SplitML}$ and measured the trained model’s accuracy (Figure 3a,b, and

Table 2. Accuracy metrics for clients during training under varying settings. M-1, 2, 3 refers to model-1, 2, 3; TA is Training Accuracy; TL is Training Loss; VA is Validation Accuracy; VL is Validation Loss. S1 is heterogeneous models of $\mathbf{SplitML}$ in honest setting; S2 is heterogeneous models in poisonous setting; S3 is homogeneous models of $\mathbf{SplitML}$ in poisonous setting, and FL is poisonous setting.

Type	S1			S2			S3			FL
	M1	M2	M3	M1	M2	M3	M1	M2	M3	M1	M2	M3
TA	0.9655	0.9583	1.0000	0.9655	0.8404	1.0000	0.9655	0.5345	1.0000	0.9655	0.5345	0.9994
TL	0.1465	0.1441	0.0000	0.1501	0.5115	0.0000	0.1501	0.6908	0.0000	0.1553	0.6910	0.0151
VA	0.9664	0.9590	1.0000	0.9664	0.8980	1.0000	0.9664	0.5338	1.0000	0.9664	0.5338	1.0000
VL	0.1433	0.1438	0.0000	0.1473	0.4348	0.0000	0.1472	0.6910	0.0000	0.1472	0.6909	0.0008

).

In the first scenario, S1, all three clients have heterogeneous models, as described earlier. They perform honestly and collaborate on a shared input layer with four neurons. We achieved 96.64% validation accuracy for model-1, 95.90% accuracy for model-2, and 100% accuracy for model-3. We use this as a benchmark and compare model accuracy for each client in malicious settings S2 and S3, where clients send poisonous weights instead of correct layer weights. In S2 ($\mathbf{SplitML}$ in adversarial setting with heterogenous models) and S3 ($\mathbf{SplitML}$ in malicious setting with homogeneous models), only model-2 is honest, and the majority of clients (model-1 and model-3) are poisonous. We chose a considerable (poisonous) update value of 999 for the experiments compared to values in the $[-2,2]$ in an honest setting. S2 achieved a remarkable 89.80% (only dropped $\approx 6\%$) accuracy with a malicious majority. In S3, we repeat the malicious majority setting of S2, with all three clients having the same 4-layer architecture as model-3. We observed similar accuracy levels for poisonous clients 1 and 3, honest client 2 suffered heavily while only achieving 53.38% (over 40% loss) accuracy, close to random guessing. We repeated this experiment in an FL setting, where clients collaborate on all layers instead of the top layers in S3. We observed similar accuracy metrics for FL as S3.

To ensure the robustness of our findings, we validated the framework by conducting experiments across five distinct model architectures. In this configuration, all five clients collaboratively train the initial two layers, which function as the common top layers. The primary input layer consists of five neurons, followed by a second layer of four neurons, both of which utilize the Rectified Linear Unit ($ReLU$) activation function. The architecture culminates in a shared output layer featuring a single neuron with a $Sigmoid$ activation function to facilitate binary classification.

The internal composition of these models is intentionally varied to simulate hardware and task heterogeneity among participants. Specifically, Model 1 incorporates an additional private hidden layer of two neurons using $ReLU$ activation situated prior to the output. Model 2 and Model 4 similarly include a hidden layer, consisting of four and three neurons, respectively, both utilizing $ReLU$ activations. In contrast, Model 3 represents a streamlined architecture without any supplementary hidden layers. Model 5 is configured with a two-layer private sub-network, where both layers contain two neurons and utilize $ReLU$ activation functions. This deliberate variation in depth and width across the local segments of the models demonstrates the capacity of $\mathbf{SplitML}$ to manage architectural divergence. By maintaining fixed common layers for aggregation while allowing local layers to fluctuate, the framework proves its utility in real-world scenarios where clients possess different computational constraints or domain-specific requirements.

In an honest setting, all five client models, M1 to M5, send correct updates. Under the malicious setting, only clients 2 and 4 are honest, and a majority (3 out of 5) models 1, 3, and 5 send poisonous updates (vector of 999 instead of honest values in the range $[-4,4]$). Since FL architecture collaborates on all the layers, we observed a poor performance (

Table 3. Accuracy metrics for five clients in a training phase. S1 refers to heterogeneous models in honest $\mathbf{SplitML}$ setting, S2 is heterogeneous models in poisonous $\mathbf{SplitML}$ setting, S3 is homogeneous models in poisonous $\mathbf{SplitML}$, and FL is performed under the poisonous setting.

Arch	Training Loss					Validation Accuracy
	M1	M2	M3	M4	M5	M1	M2	M3	M4	M5
S1	0.1352	0.1564	0.2390	0.0000	0.0000	0.9691	0.9623	0.9163	1.0000	1.0000
S2	0.1378	0.1568	5241.3701	0.0000	0.0000	0.9691	0.9623	0.5658	1.0000	1.0000
S3	0.1378	0.1568	0.6839	0.0000	0.0000	0.9691	0.9623	0.5658	1.0000	1.0000
FL	8.48 × 1015	1.03 × 1016	1.19 × 1017	3.20 × 1017	3.43 × 1017	0.9691	0.9623	0.5658	0.0000	0.0000

) as expected with very high losses. In FL, only models 1 and 2 managed to get 96% accuracy, while model 3 observed 56% accuracy, where its performance dropped by 35% compared to the honest setting. Moreover, models 4 and 5 in FL reported 0% accuracy.

FL leaks information (under malicious setting) as the reported validation accuracy for these five models corresponds to their data distribution. Since the poisonous updates classifies everything as anomalous, validation accuracy is inversely proportional to the dataset size with normal class. Model-1 had 3.08% normal samples, Model-2 had 3.67%, Model-3 had 43.23%, and Model-4 and Model-5 had 100% normal samples (thus 0% accuracy).

$\mathbf{SplitML}$ performed exceptionally well, with the same model accuracy in a poisonous setting as honest. Only model-3 performed poorly with 56% accuracy, as it has an output layer right after the two shared layers and did not have any additional hidden layers to compensate for the propagated poisonous values and adjust the weights in the deep hidden layers, while we presented empirical evidence of inherent robustness to poisoning attacks due to our architecture, we do not offer active mitigations to prevent poisonous updates (refer Appendix C.1), as measures based on the similarity of model updates would not be helpful if the majority of the clients are malicious.

3.2. Inference Attacks

The following subsections provide a detailed discussion of membership inference and model inversion attacks. These evaluations are critical for quantifying the “Privacy-via-Encryption” efficacy of $\mathbf{SplitML}$, as they represent the most common methods used by adversaries to extract sensitive training data from shared model updates. In our analysis, we demonstrate how the architectural partitioning of $\mathbf{SplitML}$ and the intrinsic noise flooding of the CKKS scheme successfully obscure these sensitive signals, preventing an honest-but-curious server from executing these privacy-compromising techniques.

3.2.1. Membership Inference

Membership Inference (MI) attacks, as formalized by [38], aim to jeopardize data privacy by ascertaining whether a specific record was utilized during the training phase. These privacy violations typically manifest when sensitive artifacts, such as model weights or activations, are exchanged between participating entities. A black-box MI attack leverages shadow models (Figure 4) and label-based queries to approximate a target model’s behavior. However, the architectural nuances of $\mathbf{SplitML}$ necessitate a more granular evaluation. Given the partitioned nature of the framework, we investigate adversarial vectors targeting both the (i) input-proximal top layers and (ii) the output-proximal bottom components. This dual-layered analysis accounts for the unique information leakage risks inherent in the split-model paradigm, where intermediate smashed data and gradient updates serve as potential signals for membership disclosure.

• First, we attack (input) datasets and their gradients from the split layer to determine their membership.
• We develop another attack model to infer membership from labels or predictions given the gradients from the cut layer.

We attacked a CNN model similar to [39] on the MNIST [40] dataset. The target model has 4 CNN and 2 ANN layers, with the first convolution layer with 3 × 3 kernel and $ReLU$ activation. It is followed by a max pooling layer with a 2 × 2 kernel. The third convolution layer has a 3 × 3 kernel with a $ReLU$ activation followed by a fourth max pooling layer of 2 × 2. The fifth layer is a dense layer with 128 neurons with $ReLU$ activation, followed by a 10-neuron output layer with $Softmax$ activation. We refer to this full model with six layers as architecture A (Figure 5).

Architecture-B is this full model A’s first 4 CNN layers (top split), and C is the last 2 ANN layers (bottom split). We then create more models, keeping the top layers (as in A) the same and measuring MI attack accuracy with shadow models on different bottom layers after the split. Architecture-D has 3 ANN layers: 128 neurons with $ReLU$, 64 with $ReLU$, and 10 with $Softmax$. E has the same number of (ANN) layers (2) as in A with 64 neurons instead of 128 in the first dense layer, keeping the same output layer. F has 2 layers as in A but 256 neurons rather than 128. In G, we remove the 128-neuron layer and keep the output layer. Finally, H has 128 neurons with $TanH$ and 10 with $Sigmoid$ for the output layer. Due to the intrinsics of the shadow models, the developed attack model will have good accuracy either on MI or non-MI. An attacker may develop two attack models complementing each other for high-confidence results.

Further, we empirically observed (

Table 4. Member-accuracy for different shadow models.

Shadows	Architecture	Median	Minimum	Maximum
	Full model	0.4374	0.4302	0.6816
1	Top layers	0.5072	0.5006	0.5100
	Bottom layers	0.3970	0.3046	0.7036
	Full model	0.5464	0.1764	0.7552
2	Top layers	0.5064	0.4186	0.6028
	Bottom layers	0.5072	0.3994	0.7042
	Full model	0.6306	0.3482	0.8566
3	Top layers	0.5170	0.4154	0.6088
	Bottom layers	0.4958	0.3030	0.6030
	Full model	0.6750	0.4644	0.8170
4	Top layers	0.4994	0.4080	0.5838
	Bottom layers	0.4778	0.3004	0.6784
	Full model	0.7130	0.4420	0.8260
5	Top layers	0.5015	0.4542	0.6166
	Bottom layers	0.4881	0.2892	0.6196
	Full model	0.7610	0.4730	0.8984
6	Top layers	0.5058	0.4352	0.5674
	Bottom layers	0.3934	0.3792	0.5972
	Full model	0.7572	0.7324	0.8428
7	Top layers	0.4766	0.4566	0.5402
	Bottom layers	0.5054	0.5016	0.5856
	Full model	0.7546	0.5564	0.7862
8	Top layers	0.5242	0.3714	0.5594
	Bottom layers	0.4052	0.3768	0.8056
	Full model	0.8356	0.8176	0.8530
9	Top layers	0.4482	0.4374	0.5168
	Bottom layers	0.5008	0.3904	0.6004
	Full model	0.7886	0.5324	0.8270
10	Top layers	0.5206	0.4376	0.5776
	Bottom layers	0.4020	0.1910	0.8900

) that the top layers, on average, leak more information than the bottom layers for MI. However, on average, splitting helps reduce attack accuracy on partial models compared to performing this attack on a complete model. Though accuracy was observed around $50\%$ for average cases, which is analogous to random guessing, a well-trained model can improve attack performance; while using a Laplacian noise for DP is expected, we suggest adding extra Gaussian noise for encryption to reduce the attack accuracy and achieve $IND-CP{A}^D$ security for approximate FHE. Scaling parameter $\Delta$ can be adjusted for better precision for this extra noise.

3.2.2. Model Inversion

Model inversion attacks seek to reconstruct the defining characteristics of a hidden input by analyzing the corresponding model outputs. As demonstrated in [41], this technique can be used to synthesize a representative input—such as a facial prototype—that maximizes the confidence score for a specific class. It is essential to distinguish this from Membership Inference (MI); while MI aims to identify specific training records, model inversion merely generates a generalized average of features that characterize an entire category. Consequently, the resulting artifacts are often semantically disconnected from any single actual data point. In the context of log anomaly detection, the efficacy of inversion attacks is significantly diminished. Unlike image processing, where feature averaging yields recognizable patterns, the discrete nature of text logs renders “average” representations semantically incoherent. $\mathbf{SplitML}$ further mitigates these risks through a dual-defense strategy. First, the integration of multi-key FHE ensures that reconstructive attacks are only theoretically plausible in a highly specific, colluding adversary model. Second, the injection of Gaussian noise during the encryption process introduces statistical perturbations that mask sensitive feature correlations. A detailed discussion of contemporary countermeasures against such threats is provided in Appendix C.

3.3. Model Extraction Attacks

Model Extraction (ME) attacks represent a significant threat to the confidentiality of machine learning assets, where an adversary aims to reconstruct a high-fidelity replica of a target model through strategic labeling queries [32], while Federated Learning (FL) primarily prioritizes data localization, it remains inherently susceptible to Intellectual Property (IP) theft. In contrast, Split Learning (SL) architectures offer structural protection; by partitioning the model, they eliminate the possibility of a direct, wholesale download of the global parameters. $\mathbf{SplitML}$ further hardens this defense-by-design, as individual participants lack visibility into the complete neural architectures of their peers.

Despite these structural advantages, extraction remains a theoretical risk through indirect observation. Prior research by Li et al. [42] has demonstrated that malicious entities in SL environments can exploit gradient information to facilitate model theft. However, $\mathbf{SplitML}$ mitigates this specific vector by ensuring that peers are restricted to shared weight updates rather than raw gradient data during training. Furthermore, even in scenarios where adversaries attempt to reverse-engineer functionality via abundant inference queries [43], $\mathbf{SplitML}$ implements proactive countermeasures. Specifically, we advocate for a consensus mechanism based on Total Labels (TL) rather than high-precision Total Predictions (TP). By obfuscating the soft-label probability vectors and providing only the final categorical output, the framework drastically reduces the signal-to-noise ratio available to an attacker. This strategy thwarts the development of accurate surrogate models, which are often precursors to transferable adversarial attacks [44,45] or hardware-level threats like bit-flip attacks [46]. A detailed discussion on restricting query information and modifying return values is further explored in the work of Jagielski et al. [47].

4. Experimental Analysis

By benchmarking our framework against standard distributed learning paradigms, we demonstrate how architectural partitioning and encrypted aggregation affect the overall efficacy of the system in a multi-institutional SIEM context.

4.1. Dataset

Log anomaly datasets typically exhibit significant class imbalance, often being heavily skewed toward either “normal” or “anomalous” instances. To prevent the machine learning model from achieving misleadingly high accuracy through majority-class bias, we implemented a balanced sampling strategy. To validate the equilibrium of our processed dataset (

Table 5. Return-1 model shows our dataset’s balanced distribution of two classes.

Type	Accuracy	Precision	Recall	F1-Score
Full (100%)	0.4999	0.4999	1.0000	0.6666
Test (20%)	0.5016	0.5016	1.0000	0.6681

), we employed a “Return-1 Model” baseline, which consistently predicts the “anomalous” class for every input. This baseline yielded an accuracy of 49.99% and a recall of 100%, confirming that the labels are evenly distributed. Under this convention, label-0 represents the normal class while label-1 signifies an anomaly.

The primary data source for this study is the Loghub HDFS_1 repository provided by Logpai [37]. This 1.47 GB dataset comprises logs generated by Hadoop-based MapReduce jobs executed across more than 200 Amazon EC2 nodes over a duration of 38.7 h. Expert labeling was performed by Hadoop domain specialists. Out of the 11,175,629 total log entries, approximately 2.58% (288,250) were identified as anomalous.

To convert these unstructured logs into a structured format suitable for analysis, we utilized the Drain log parser [48], though the specific mechanics of the textual parsing are omitted here for conciseness. For our experimental framework, we curated a balanced subset consisting of 576,499 entries, characterized by seven features distributed uniformly across both classes. This dataset was subsequently partitioned into non-overlapping, independent and identically distributed (i.i.d.) subsets totaling 397,366 observations distributed among three distinct clients. Specifically, Client-1 was allocated 69,455 normal and 66,780 anomalous samples; Client-2 received 93,032 normal and 73,055 anomalous samples; and Client-3 was assigned 43,666 normal and 51,378 anomalous samples. For each client, the data was bifurcated into a training set comprising 80% of the local observations and a testing set containing the remaining 20%.

4.2. Results

We present experimental results about ML attacks in Section 3. The computations were performed on a 2.4 GHz MacBook Pro with a Quad-Core Intel Core i5 processor and 8 GB of 2133 MHz LPDDR3 memory. We used Python 3.11 [49] with sklearn APIs [50] for binary classifiers. We compared the performance based on the following measures: Precision, Recall, Accuracy, and F1-Score. We created three small NNs, depicting three clients (Client-1, 2, 3) participating in our scheme for both training and inference.

The models share the first two Fully Connected (FC) layers, with the first layer having five neurons, 40 parameters, and $ReLU$ activation and the second layer having four neurons, 24 parameters, and $ReLU$ activation. For simplicity, all the models have a common output layer with a single neuron and $Sigmoid$ activation function. Model-1 has a third layer with two neurons, ten parameters, and $ReLU$ activation. Model-2 only has the three layers described earlier. In contrast, Model-3 has two additional FC layers, a third layer with three neurons, 15 parameters, and $ReLU$ activation, and a fourth layer with two neurons, eight parameters, and $ReLU$ activation.

All the clients train their models with plaintext data and only encrypt the weights to the FL server after one round is complete. For each round, all clients train their models for one epoch in parallel. In the first round, clients initialize their model weights with random values, and for the subsequent rounds, the weights for the shared layers are set to the values provided by the server. The server calculates the results in each round using an FL algorithm (like FedAvg) in the encrypted domain and the homomorphic $EK$. Empirically, we observed that we reduced the epochs required to converge the model on average by half. As shown in Figure 6, for a batch size of 64 and learning rates of 0.05 and 0.10, the standalone training (S_Acc, S_Loss denotes Standalone Accuracy/Loss) required six epochs to converge whereas collaborative learning (C_Acc, C_Loss denotes Collaborative Accuracy and Loss) achieved higher accuracy in only three epochs.

We further experimented with inference using the earlier trained models. For $Sigmoid$ activation function $f\left(z\right)\in [0,1]$, we set boundary threshold $\lambda =0.05$. We performed prediction consensus using both the predicted label TL (Section 2.5) and prediction value TP (Section 2.5) approach for the samples that fall in the prediction range of $[0.45,0.55]$. We set $TL\ge 2$ and $TP\ge 1.5$ to choose label-1 for three clients in a consensus.

We observed that TL performed better when most samples were classified as normal (prediction scores close to 0), and TP worked better when most were anomalous (scores close to 1). For example, in an observation for $lr=0.05$ with 88 normal and 0 anomaly samples, Model-1 achieved 18.18%, Model-2 achieved 73.86%, Model-3 achieved 00.00%, TL achieved 75.00%, and TP achieved 31.82% accuracy. TL outperformed all models. In another observation for $lr=0.10$ (

Table 6. Consensus results with $lr=0.10$.

Type	Accuracy	Precision	Recall	F1-Score
Model-1	0.8770	0.8749	1.0000	0.9333
Model-2	0.7366	0.9319	0.7485	0.8302
Model-3	0.8604	0.8604	1.0000	0.9249
TL	0.7366	0.9319	0.7485	0.8302
TP	0.8877	0.8845	1.0000	0.9387

) with 353 normal and 2175 anomalies, Model-1 had 87.70%, Model-2 had 73.66%, Model-3 had 86.04%, TL had 73.66%, and TP recorded the highest of 88.77% accuracy.

In practice, a client may want to perform a consensus for the predictions most likely to be False Positives (FP) and False Negatives (FN). From our experiments, we observed that for our three clients with $Sigmoid$ output, all the false predictions were from $[0.00,0.10]$ and $[0.90,1.00]$. For one observation for $lr=0.05$ with 1388 FN and 23663 FP collected from all three clients, Model-1 achieved 5.54%, Model-2 achieved 41.11%, Model-3 achieved 5.54%, TL achieved 5.54%, and TP achieved 20.80% accuracy. For another observation for $lr=0.10$ with 3166 FN and 29 FP, Model-1 achieved 0.69%, Model-2 achieved 26.29%, Model-3 achieved 26.26%, TL achieved 32.99% and TP achieved 21.60% accuracy.

Key generation involves interaction between parties and is done before the training process begins. We observed ∼5–6 s processing time (zero network communication overhead, as all the clients were simulated locally on a single machine) to generate all private and shared keys. Federation overhead was ∼1.2–1.5 s of total training time of ∼25–30 s per epoch.

5. Discussion

The experimental results and security analysis presented in this study validate $\mathbf{SplitML}$ as a robust framework for decentralized SIEM environments. By partitioning the model into shared top layers and private bottom layers, the system successfully balances global collaborative learning with the need for local model heterogeneity. This architectural split ensures that participants can maintain unique local configurations while still benefiting from a collective global intelligence.

The complexity of the multi-key CKKS generation and the consensus inference procedure can be understood through a simple authority model. In the key-generation phase, participants collaboratively create a public evaluation key to ensure that no single party ever possesses the full secret key. This is similar to a multi-signature vault where all institutional participants must provide their partial shares to unlock the final fused result. Similarly, the consensus inference phase serves as a decentralized second opinion. When a client’s local private layers produce an ambiguous classification for a suspicious log entry, it sends an encrypted smashed data packet, which is an intermediate activation, to its peers. The peers process this through their shared layers and return an encrypted vote. The final decision is decrypted only after aggregation. This process ensures that neither participants nor servers ever see the raw telemetry or the individual local predictions of others.

5.1. Differential Privacy and Noise Flooding

A critical distinction must be made regarding the nature of the privacy guarantees provided by $\mathbf{SplitML}$, while our approach utilizes noise flooding inherent in the OpenFHE CKKS implementation to achieve $IND-CP{A}^D$ security, this should be distinguished from formal $(ϵ,\delta )$-Differential Privacy (DP). Traditional DP provides a mathematical bound on the probability of identifying a specific individual’s contribution by injecting calibrated noise. In contrast, $\mathbf{SplitML}$ relies on a Privacy-via-Encryption paradigm.

The noise flooding mechanism ensures that the decryption results do not leak the secret key or sensitive intermediate values in a multi-party setting, while this effectively thwarts reconstructive inference attacks by obscuring individual gradients and activations, it does not currently provide a formal DP privacy budget accounting. This cryptographic protection is robust against the honest-but-curious server, but it does not technically satisfy the information-theoretic definition of DP unless specific $(ϵ,\delta )$ parameters are formally integrated into the noise generation process.

5.2. Large Language Model (LLM) Security

As distributed learning moves toward more complex architectures, the security of Large Language Models (LLMs) becomes a paramount concern. Recent surveys, such as Zhou et al. [51], have highlighted significant backdoor threats in LLMs. In these scenarios, malicious triggers can be embedded during the training or fine-tuning phase to bypass security filters. Furthermore, the deployment of LLMs in sensitive environments introduces the risk of fingerprinting attacks. In these cases, an adversary optimizes queries, sometimes using reinforcement learning, to identify and exploit model identities uniquely [52].

The $\mathbf{SplitML}$ architecture offers a potential defense-in-depth strategy for these emerging LLM threats. By keeping the output-proximal bottom layers, such as specialized classification heads or instruction-tuning layers, private to each institution, the framework can neutralize universal backdoor triggers that rely on global end-to-end weight manipulation [51]. Additionally, because gradients and activations are encrypted using multi-key FHE, the framework provides a cryptographic barrier against fingerprinting attempts. Protecting model updates in this manner prevents the intermodel discrepancy signals that offensive fingerprinting tools use to discriminate between models [52]. Integrating these LLM-specific defenses is a vital direction for the next generation of FSL.

6. Conclusions

This paper presents the $\mathbf{SplitML}$ framework, which is a unified architecture designed to resolve the critical privacy–utility trade-offs in distributed machine learning. By merging the structural advantages of FL and SL, $\mathbf{SplitML}$ enables model heterogeneity through a partitioned hierarchy. This involves sharing standardized top layers for collaborative feature extraction while keeping bottom layers private for localized classification.

The integration of MK-FHE ensures that sensitive activations and gradients remain encrypted during both training and inference. This provides a robust defense against reconstructive inference attacks. Furthermore, a collaborative consensus process facilitates high-precision global inference without exposing raw client telemetry. The evaluation of federation costs demonstrates that the multi-key FHE overhead is concentrated in the initial setup. Once the shared evaluation keys are established, the per-epoch training latency is dominated by local computation rather than cryptographic operations. This positioning of the security tax at the initialization phase makes $\mathbf{SplitML}$ a practical choice for SIEM operators who require long-term and continuous collaborative monitoring across heterogeneous network infrastructures.

7. Future Work

Despite these advancements, the current framework operates under a semi-honest or honest-but-curious threat model; while the architecture reduces the blast radius of poisoning through localized layer refinement, it lacks active cryptographic or statistical defenses to filter malicious updates in real-time. Furthermore, the reliance on a central aggregator assumes a passive adversary; it does not account for an actively malicious server capable of tampering with global weights or misrouting encrypted queries.

Future research will prioritize extending the security perimeter to include actively malicious servers by transitioning toward decentralized trust models. We propose deploying digital signature verification, multi-server distributed aggregation using threshold secret sharing, or a peer-to-peer (P2P) decentralized topology to eliminate the central aggregator as a single point of failure. These architectures distribute the flow of authority to ensure no single compromised entity can manipulate the global model or recover private data.

Additionally, we intend to integrate active Byzantine-robust aggregation techniques, such as Krum, Trimmed Mean, or Median-based filtering, to prune malicious gradients before they statistically contaminate the shared weights. We emphasize that SplitML is complementary to these robust aggregation methods; while our architectural partitioning limits the scope of an attack to the shared layers, these statistical filters would provide the necessary active defense to maintain the integrity of those shared parameters.

Finally, future iterations will formally incorporate fairness-aware constraints into the optimization objective. This will help to prevent bias and ensure equitable model performance across diverse and heterogeneous participants.