A New Lossless Compression Paradigm for Federated Learning: A Quantile-Based Framework for Bandwidth Efficiency Without Accuracy Degradation

Abstract

Federated Learning (FL) is a machine learning technique that preserves data privacy and security by training models directly on decentralized edge network devices. This generates substantial communication overhead due to the repeated exchange of model updates across numerous edge network devices. Quantization has tackled this challenge by reducing communication overhead and computational costs by quantizing model updates. Although selecting the most suitable quantization level to balance communication efficiency and model accuracy is challenging, failing to achieve this balance results in excessive compression, leading to accuracy degradation due to the lossy nature of the quantization technique. This challenge was tackled in this paper via a Quantile-based lossless compression method named Pcodec, which implements lossless compression in the FL context. Pcodec is a Quantile-based lossless compression algorithm designed for numerical data that utilizes mode identification with delta encoding and binning, where binning groups similar values into entropy-coded bins and stores the exact offset within each bin, thus achieving high compression ratios and efficient processing speed. Using MNIST and CIFAR-10 datasets and models such as CNN and ResNet18, we demonstrate that Pcodec achieves up to 58.19% size reduction with no accuracy loss compared to standard quantization methods. The experiments showed that the proposed Quantile-based compression approach in FL reduces up to 2.81× the communication overhead between each server and edge network device while maintaining the accuracy. In comparison to quantization, the Quantile approach reduced the communication overhead by 2.74×, tackling the main challenge of FL context by reducing communication overhead with a remarkably high compression ratio while maintaining the model’s accuracy.

1. Introduction

FL [1,2,3,4] represents a groundbreaking approach in machine learning where models are trained directly on decentralized edge network devices instead of centralizing data in one location, thereby ensuring strong privacy and data security. This approach is essential in several sectors where data privacy is incredibly important, like finance, healthcare, the Internet of Things, and autonomous vehicles. One of the primary challenges in FL is the massive communication overhead incurred when aggregating model updates from numerous edge network devices. This overhead can lead to delays in model convergence and increased operational costs. These effects collectively point to a clear need to reduce communication overhead in FL to enhance the efficiency of model training and minimize the resource consumption on participating devices.

Several approaches have been put forth to address this challenge by using compression to reduce the volume of the model updates, and then communication overhead is frequently reduced [5,6,7]. This has been accomplished by common techniques such as quantization, which minimizes the model size by decreasing the precision of weights [8,9]. Typically, it converts floating-point data into fixed-point or integer representations. It decreases the model updates and transmits them using just a few numbers of bits [10,11,12,13,14,15,16], knowledge distillation, in which a smaller “student” model learns from a larger “teacher” model, enabling the use of more compact models in FL systems [17,18], and pruning, which removes less significant connections from the model, producing sparser models that require less data transfer [19,20,21,22,23]. To further reduce the model size, techniques such as federated dropout and gradient sparsification are used, selectively zeroing out less important values, wherein just a portion of the model’s parameters are updated or transmitted [24,25,26,27,28].

Despite their effectiveness in reducing communication overhead, these compression techniques carry significant drawbacks. The lossy nature of knowledge distillation, sparsification, pruning, and quantization introduces a fundamental trade-off between compression efficiency and model accuracy in FL. These methods invariably result in some information loss since they have simplified model structures, eliminated gradient information, removed connections, and shrunk the precision of weights. Throughout multiple FL rounds, these accumulated weight distortions may compound and lead to gradual degradation in the model’s accuracy [25,29,30,31]. To address these limitations, the authors aim to use a lossless compression method with a high compression ratio, namely quantile-based lossless compression.

This paper proposes a novel usage of an open-source quantile-based lossless compression method, namely Pcodec [32]. It is crafted as a solution for compressing numerical sequences losslessly. It achieves high compression ratios ranging from 29% to 94% while maintaining data accuracy, ensuring no information is lost [33].

Unlike other lossy compression techniques, this approach ensures a precise data representation while providing efficient compression and decompression. As a result, it is suitable for applications requiring accuracy and performance. It supports various data types, including columnar data, time series, and low-bandwidth communication.

This study investigates the use of an existing open-source quantile-based lossless compression framework known as Pcodec within the FL context. To the best of our knowledge, this work is among the first to explore the potential of Pcodec as a communication efficiency tool in FL settings. In contrast to the existing literature that has primarily relied on lossy compression methods, this work focuses on a lossless approach that makes sure to preserve the integrity of model updates. This distinction is particularly crucial in FL environments, where model convergence may be compromised by even small compressive errors that propagate and accumulate over communication cycles. This study assesses the practical effectiveness of Pcodec by evaluating it in a unified experimental framework with the dual objective of reducing communication overhead and maintaining the model accuracy.

This work aims to test whether Pcodec can reduce communication overhead in FL without degrading model accuracy. In addition, the study examines how Pcodec impacts data transmission from edge networks to the central server in FL and whether it maintains model performance across different benchmarks. To address these research questions, the study utilizes Pcodec across benchmarks and compares its results with existing compression methods. The main contributions in this paper are listed as follows:

1.

Applies the open-source quantile-based lossless compression framework (Pcodec) to compress deep network updates in the context of FL.

2.

Demonstrates through comprehensive experiments that the Pcodec algorithm can be efficiently employed with various convolutional deep networks and achieves a high compression ratio while maintaining competitive model accuracy under a unified evaluation setup.

The rest of this paper is structured as follows, following an overview of the current model compression techniques and quantile techniques in other domains in Section 2. Section 3 clarifies the Quantile lossless compression approach along with the proposed FL algorithm. In Section 4, comprehensive explications of the experimental setup are provided, which includes a description of the baseline methods, implementation details, and evaluation metrics used to assess the Pcodec technique. The results are then discussed and compared with the baseline methods. Finally, Section 5 presents the conclusion and future work.

2. Related Work

2.1. Model Compression Techniques

FL has attracted significant attention in recent years since it protects privacy by enabling collaborative learning among multiple clients without requiring the sharing of raw data. Nevertheless, despite these privacy protections, FL remains vulnerable to attacks that compromise training data, which requires sophisticated privacy-preserving strategies. Techniques such as private grouping of teacher ensembles (PATE), secure multi-party computation (SMPC), and homomorphic encryption (HE) have been commonly used to enhance privacy and protect model updates. The authors in [34] helped FL systems perform more consistently via implementing various combinations of FL with HE, SMPC, and PATE, and tested models against several attack scenarios, including poisoning, backdoor, model inversion, and man-in-the-middle attacks. However, communication overhead is still considered a significant challenge in the FL context, prompting researchers to explore model compression techniques to address this challenge and reduce the massive data exchanged during the training process.

Some recent works tackle these compression techniques. In [35], the model updates were compressed through a vector compression approach depending on the random lattice quantization. Quantized deep networks within general FL and updated quantization parameters concurrently with the updated of deep networks during the FL process using stochastic gradient descent (SGD) were presented in [36]. The major bottleneck in FL is the excessive communication overhead caused by the frequent exchange of model updates, addressed in [37,38] via incorporating both sparsification and quantization techniques. To minimize communication overhead and accelerate the inference process, a model compression technique known as knowledge distillation (KD) was incorporated into the framework [39]. In [40], knowledge distillation was utilized to compress local models and minimized the communication overhead caused by vehicles’ high mobility and limited bandwidth in Unmanned Aerial Vehicle-assisted Vehicular Edge Computing.

In comparison to centralized machine learning, FL has distinct challenges in handling the communication overhead caused by the frequent exchange of model updates and in protecting data privacy. To tackle these challenges, the authors in [35] introduced a technique that jointly performed lossy compression with privacy improvements in the FL context without a great effect on the value of the trained model. This technique quantizes data to the desired bit rate concurrently while maintaining the aimed level of privacy.

Deep network compression employing data is not feasible in the FL context due to missing data on the centralized server. Deep network compression is a crucial operation because of its inference possibility even on low-capacity networks. To tackle this challenge, the authors in [36] introduced a novel quantization technique that decreased floating-point computations every second in the deep network without compromising the user data in FL. This approach learned quantization and deep networks while making sure that the system was secure, where data and computations were distributed across various devices. The authors presented noticeable results that proved this strategy is feasible in the most typical deep neural network with convolutional layers by emulating it in several deep neural networks with convolutional layers. Nevertheless, this method reduces the precision of the data; it still performed as accurately as the standard higher-precision method when tested.

In [37], sparsification and quantization techniques tackled the communication overhead caused by homomorphic encryption in FL. While encryption ensures data privacy in FL systems by masking local gradient updates, it substantially increases data transfer due to the extra ciphertext size, leading to communication overhead and transmission inefficiencies. The authors in [37] mitigated this challenge by applying gradient sparsification to transmit only the most essential gradient values, then applying quantization to further compress those values, and finally, the Lion optimizer is employed to assist low-error gradient quantization.

In [38], the authors addressed communication costs in the FL challenge, which arise extremely due to the transmission of full precision of weights. This challenge becomes significant with the rising use of edge devices, where data sharing leads to high communication overhead. The authors in [38] addressed this challenge by first applying heuristic sparsification to select only the most important model updates. then, applying ternary quantization to further reduce accuracy loss. In addition, to better identify important parameters, a dual-threshold approach adjusted sparsity levels. Finally, residual updates were refined via weighted voting.

FL facilitates the sharing of machine learning knowledge among numerous agents due to its distributed learning structure. Unfortunately, in the case of mobile agents, learning performance is strongly impacted by the agents’ mobility. Furthermore, a significant communication overhead could occur via frequent local model exchanges. The authors in [39] introduced a framework to enhance the learning performance and speed up the training process by developing a mobility-aware node-selecting strategy and modeling the impact of the agent’s mobility on the quality of communication and the correlation of data. Their framework incorporated a model compression technique, knowledge distillation, to minimize communication overhead and speed up the inference process.

In unmanned aerial vehicle-assisted vehicular edge computing, the vehicles’ high mobility and limited bandwidth lead to a considerable communication overhead in FL systems [40]. This challenge arises from frequent data exchanges and large model updates, increasing transmission costs and latency. The authors in [40] tackled the challenge by utilizing a knowledge distillation approach. In this approach, each vehicle maintained a teacher model and a student model. During each training cycle, the parameters of the teacher model were updated via knowledge distillation. On the other hand, the student model’s parameters were sent for global aggregation to the edge server. By compressing the original model, this technique effectively reduced communication overhead, enabling faster model transmission.

Overall, most existing approaches mainly focus on reducing the communication overhead in FL by using lossy compression techniques such as quantization, sparsification, knowledge distillation, and pruning. These techniques are effective in lowering the communication overhead but still often lead to approximation errors that accumulate over the rounds of communication and affect the model accuracy. This limitation underscores the necessity for alternative compression strategies that can alleviate communication overhead while maintaining the integrity of model updates.

2.2. Quantile Methods in Other Domains

Quantile methods have been extensively used across various domains due to how well they work in summarizing data distributions. These methods are applied in fields such as finance, where they pave the way for making well-informed decisions and strategic plans [41,42,43], and help assess risk by measuring value-at-risk [44,45,46]; machine learning, where they offer a closer look and nuanced understanding of the data, helping to model prediction uncertainty, variability, and distributional features [47,48,49]; and FL, where they reduce encoding and decoding overhead by compressing the intermediate model [50]. Quantile-based techniques are also crucial in healthcare, environmental science, manufacturing, and sustainable development domains; with some other algorithms, they ensure a robust analysis that can be applied to enhance decision-making and promote innovation [51]. These diverse applications highlight the ability of quantile-based methods to efficiently represent the distribution of data and motivate their possible utilization in FL to enhance communication efficiency.

2.3. Gap Analysis

Most current compression techniques in the FL context reveal a critical gap in achieving high compression ratios without degrading model accuracy, limiting their overall effectiveness. Even though the quantile methods have been used in various domains, their use in FL focuses on approximate or lossy representations, which still make a difference in the accuracy due to their lossy nature. Consequently, no studies have investigated quantile-based lossless compression for FL communication. This gap motivates exploring the lossless quantile-based compression method in the context of FL, which can achieve high compression ratios without sacrificing model accuracy.

3. Quantile-Based Compression Method

3.1. Proposed Method Overview

This section presents a high-level overview of the quantile-based compression technique Pcodec. This method conceptually compresses numerical data via dividing it into quantile ranges, with each range represented by a Huffman code based on its frequency in the dataset. The Huffman code identifies the quantile range a value belongs to, while an offset specifies the exact position within that range. If delta encoding is also applied, the technique compresses the differences between consecutive values, furthermore improving compression efficiency. The proposed methodology is depicted in Figure 1.

In Figure 1, the central server initially distributes the global model to each client (e.g., edge device). Each client thereafter trains the model locally on its private data without disclosing any raw data. Upon completion of the training, each client compresses its updated model using the quantile-based lossless compression approach Pcodec. The compressed local models are then transmitted to the central server, where they are decompressed and aggregated to update the global model. The compression methodology internally optimizes encoding selections based on data characteristics to achieve high compression efficiency that approaches the theoretical compression limit of the data distribution, as defined by Shannon entropy. The compressed data are stored in a file with a well-structured format, including a lightweight header, metadata, and numerical data chunks, followed by a termination byte. This structure makes sure that the storage is efficient and the decompression is easy to perform while preserving the integrity of the original data, making this technique perfectly ideal for large-scale numerical datasets. The theoretical relationship among quantile binning, Huffman coding, and Shannon entropy can be substantiated as follows. Huffman coding is an optimal prefix-free coding scheme. Its average code length is theoretically limited to within one bit of the Shannon entropy of the source distribution. In Pcodec, the weight distribution is partitioned into quantile bins of approximately equal probability mass, which maximizes the entropy of the bin distribution and tightens the gap between the Huffman code length and the Shannon entropy lower bound. Furthermore, delta encoding is applied before binning to reduce the inter-sample redundancy, lowering the effective entropy of the sequence before compression. Together, these two stages allow Pcodec to approach the theoretical compression limit more closely than fixed-precision quantization, which makes no use of the empirical weight distribution.

In this work, the above conceptual quantile-based compression is implemented using the Pcodec library. The library offers an adaptive compression framework in which user-defined parameters are not exposed and operations like entropy coding, delta encoding, and binning are handled internally. As a result, quantile levels, encoding thresholds, and entropy coding settings are managed internally, and there is no GCD-based preprocessing. To ensure complete reproducibility, the implementation used in this study is publicly available at [32].

3.2. Mathematical Formulation

3.2.1. Federated Learning

FL follows several steps, At first initializes a deep network by the central server and transmits it to user devices. Second, the users train the deep networks locally without sharing raw data. Third, all user devices send the deep networks to the central server. Finally, the central server aggregates all deep networks trained on user devices and sends the resulting deep network back to the user devices. Equation (1) shows the aggregation formula Federated Averaging (FedAvg).

$$w_{t+1}\leftarrow {\displaystyle \sum _{n=1}^N}({\displaystyle \frac{D_n}{D}})w_{t+1}^n$$

(1)

where $w_{t+1}$ is the global model at round $t+1$, N is the total number of participating clients, $D_n$ is the number of data samples on client n, and $D={\sum }_{n=1}^N{D}_n$ represents the total number of data samples across all clients. Finally, $w_{t+1}^n$ is the model update from client n.

3.2.2. Quantile Lossless Compression-Based Method

Compressing weights using Quantile lossless compression is a way to reduce the byte size to almost the theoretical ideal limit of the Shannon entropy of the distribution for faster inference. In this technique, each number is represented by a Huffman code followed by an offset, where the Huffman code belongs to a defined range [lower, upper] containing the number, and the offset indicates the precise location within that range. The compression process follows steps:

1.

Compute deltas and use them as the values instead of the original values if delta encoding is configured.

2.

Determine unoptimized prefixes by selecting $2^{compression\_level}$ roughly evenly spaced quantiles from the distribution, each is a range with related metadata. It determines the GCD for each range if enabled.

3.

Merge neighboring prefixes to maximize their effectiveness where appropriate.

4.

Each prefix is given a Huffman code according to its respective weights in the data.

5.

Use prefixes to encode deltas.

Equation (2) denotes the concept of lossless compression.

$$f_{decomp}\left(f_{comp}\left(w\right)\right)=w$$

(2)

where applying the decompression function $f_{decomp}$ after the compression function $f_{comp}$ on the original data w results in the exact data w. In other words, the process of compression and decompression is lossless, meaning no information is lost during compression, and the original data can be perfectly recovered.

3.2.3. The Proposed Algorithm

In Algorithm 1, an adaptive implementation of FL is presented including weight compression using the Quantile-lossless compression technique. The technique utilizes weight compression to make sure that the communication is efficient while coordinating a distributed learning process among N clients. The process starts with initializing the neural network weights W on a central server S and distributing them once to all clients. This is followed by a for loop of T communication rounds of federated training, where each round consists of two phases executed sequentially: the client-side local training and the server-side decompression and aggregation phases.

Inside this loop, on the client side, each client $C_i$ receives the current global model weights W from the server and then performs the local training for $\tau$ epochs in its private data set $D_i$ without sharing any raw data. Upon completion of the local training iterations, each client after employs Quantile compression on its updated weights $W_i$ before the transmission process. The compression stage is essential, as it significantly decreases the communication overhead between clients and the central server without sacrificing model accuracy, thereby enhancing the efficiency and scalability of the FL process.

Also inside this loop, on the server side, the server begins receiving the compressed weights from all participating clients and decompresses them. Following decompression, the server aggregates decompressed weights by computing the weighted average, where the weights correspond to the size of the local data set of each client $(D_i/D)$. This ensures that clients with more extensive datasets have a greater impact on the global model update, balancing contributions according to data volume across all T rounds, gradually enhancing the global model while preserving two crucial aspects: data privacy via local training and communication efficiency through the compression mechanism. This approach effectively balances the trade-off between model performance and system efficiency in FL scenarios.

Algorithm 1: The FL algorithm with compressing weights using the quantile lossless compression method.

The time complexity of the proposed FL algorithm with weight compression using the quantile-based compression method can be expressed as the following expression: $O(T·N·\tau ·f\left(D_i\right)+T·N·g\left(W\right)+T·N·h\left(W\right)+T·W)$. In the expression, T refers to the number of communication rounds, where N is the number of participating clients, and $\tau$ represents the number of training epochs each client performs on the local dataset $D_i$. $f\left(D_i\right)$ represents the computational cost of local model training (Algorithm 1, Lines 10–11), which depends on the dataset size and the model architecture. The term $g\left(W\right)$ refers to the cost of applying quantile compression to the model weights (Algorithm 1, Line 12), including a quantile transformation followed by lossless encoding using delta and Huffman coding over W parameters. Similarly, $h\left(W\right)$ denotes to the decompression cost at the server side (Algorithm 1, Line 17), required to reconstruct the compressed weight updates. Finally, the term $O(T·W)$ reflects the time required to aggregate the decompressed weights on the central server (Algorithm 1, Line 18). Thus, the overall time complexity captures the whole workload from both the client and server sides, indicating that it is dependent on the number of clients, communication rounds, and model parameters, in addition to the local training and compression costs.

4. Results and Discussion

4.1. Experimental Setup

In this section, the proposed method is evaluated using two datasets. (1) The MNIST Dataset [52] is a group of 70,000 grayscale images with 28 × 28 pixels that display handwritten digits from 0 to 9. It is divided into 10,000 testing images and 60,000 training images. MNIST is popular for its small size and simplicity, and it remains a standard benchmark for evaluating basic model performance and efficiency. (2) The CIFAR-10 Dataset [53] is a group of 60,000 color images with 32 × 32 pixels, divided into 10 distinct classes such as cars, ships, birds, and airplanes. Each class contains 6000 images, with the dataset split into 10,000 testing images and 50,000 training images. CIFAR-10 is more complex than MNIST due to its higher resolution and diversity, making it a common choice for evaluating image classification models and testing more advanced tasks. The rest of this section is organized as follows. Section 4.1.1 introduces the two baseline methods used for comparison with the proposed approach, the two evaluation metrics are presented in Section 4.1.2, and finally, the implementation details of the experiments are presented in Section 4.1.3.

4.1.1. Baseline Methods

As discussed in Section 2, the JOPEQ (Joint Privacy Enhancement and Quantization) approach [35] tackles the communication overhead challenge in FL via combining lossy compression and privacy enhancement. In our comparison, we emphasize its particular implementation details, where Vector quantization has been used based on a random lattice and produces a distortion miming and additive noise that are afterward utilized to enhance privacy guarantees. Also, analytical findings offered local differential privacy (LDP) assurances and limitations on distortion and convergence, while the empirical research validated how it is effective against privacy attacks.

In addition, the OQFL (Optimized Quantization in Federated Learning) approach [36] tackles the challenge of compressing deep networks in FL without accessing centralized data. OQFL reduced floating point operations per second (FLOPS) by training quantization parameters alongside network weights, assuring both compression and data privacy. Experiments on various convolutional neural networks indicated that OQFL can preserve the accuracy of conventional FL while employing only 4-bit quantization, notably decreasing the computing overhead compared to the standard 32-bit model.

4.1.2. Evaluation Metrics

Two primary metrics are considered to evaluate how well the baseline methods in FL perform: compression ratio and accuracy. The compression ratio measures the effectiveness of the compression technique by measuring the proportional reduction in the size of the original model. When the compression ratio is higher, it shows that the method is more effective with fewer bits required for communication, which is the main factor for reducing communication overhead. The second key metric is accuracy, which aims to measure how the compression technique affects the predictive performance of the model.

4.1.3. Implementation Details

The proposed framework is implemented using the Rust-based quantile compression library q_compress (v0.11.7) within the Pcodec framework, along with the associated internal crate pco (v0.0.0-alpha.0). Default settings are used throughout all experiments. In addition, the baseline quantization methods (JOPEQ [35] and OQFL [36]) were re-implemented within the experimental setup to ensure a consistent comparison across all compared methods.

An in-depth analysis of the Quantile-based lossless compression’s performance in the FL environment is provided by focusing on two key issues: Q1. How does quantile-based compression impact data transmission from edge networks to the central server? Q2. What is the impact of quantile-based lossless compression and decompression on the accuracy of the learned model?

Two comprehensive experiments were performed to investigate these two questions.

1.

The first experiment is conducted using the MNIST and CIFAR-10 datasets in an Independent and Identically Distributed (iid) environment. It is supposed that all data is kept on the user’s devices rather than on the central server. In the PyTorch version 2.10.0 simulation, the cross-entropy function is utilized as the loss function with 0.9 momentum. The fraction of participation of the overall 100 clients is 0.1 with 5 local epochs, 100 batch size, 0.01 local learning rate.

Two models are used in this experiment, VGG16 [54] and ResNet18 [55] for deep convolutional neural networks. These two architectures were selected because they present contrast architectural characteristics. VGG16 is a deep network with many parameters and a simple sequential design and ResNet18 has a compact structure with residual connections and fewer parameters. Finally, both models are commonly used in FL studies, which allows us to compare the study results with previous work, and they are also practical to train repeatedly within our experimental setup. All networks are compressed using a quantile-based compression method with full precision. Compared to the original full-precision network, the compressed network infers substantially faster. Therefore, the compressed networks by quantile-based lossless compression will work more smoothly on local devices with limited computing power. Finally, both neural networks have the same result that the approach maintains the accuracy of the model.

2.

In the second experiment, two different datasets are used to implement the FL frameworks: the MNIST and CIFAR-10 datasets. For the MNIST dataset, the number of participating users in this simulation, presented as (K), is 30 users, each performing local training using SGD with a 0.01 learning rate. All networks are compressed using a quantile-based compression method, and the experiment was evaluated by employing three model architectures: a linear regression model, a multi-layer perceptron (MLP), and a convolutional neural network (CNN). The proposed framework is compared with the baseline [35]. For the baseline method, the lattice dimension is $L=2$ with the compression ratio $R=1$ and privacy budget $ϵ=4$. Subsequently, the FL is applied to the CIFAR-10 dataset, which contains 32 × 32 RGB images, similarly divided into 50,000 training and 10,000 test samples, but distributed among K = 30 users. Each user utilizes local SGD with a learning rate of 0.1. The architecture in this case is a more complex CNN with three convolutional layers followed by four fully-connected layers, integrating ReLU activations, max pooling, and dropout layers before the softmax output layer. While the MNIST experiments validate the scalar encoders, CIFAR-10 focuses on implementing multivariate approaches as baselines, allowing the exploration of how model complexity and data diversity affect FL performance. For the baseline method, two configurations are evaluated, both with $L=2$ and $ϵ=3$, but differing in the compression ratio: the first with $R=2$ and the second with $R=1$.

4.2. Results

Table 1. Compression ratio results comparing our Quantile method with Quantization [35] across three models (Linear Regression, MLP, and CNN) on the MNIST and CIFAR-10 datasets.

Dataset	Compression Method	Model
		Linear	MLP	CNN
MNIST	Quantization [35]	3.6%	−2.3%	0.4%
	Proposed Quantile	68%	55%	53%
CIFAR-10	Quantization [35]	−3.4%	1.8%	1.7%
	Proposed Quantile	63%	86%	86%

Compression ratio shows percentage reduction in file size using (original size − compressed size)/original size × 100.

compares two compression methods, quantization [35] and our quantile-based approach, across the MNIST and CIFAR-10 datasets using three different models: linear regression, MLP, and CNN. The results demonstrate that, while both methods aim to reduce communication overhead, the quantile-based method achieves significantly better performance. It increases the compression ratio by 19 to over 132 times compared to quantization in positive cases and completely eliminates degradation in scenarios where quantization results in negative performance.

Table 2. Compression ratio results for different layers (LSQ, Act, and Conv) in the VGG16 model on MNIST and CIFAR-10 datasets.

Dataset	Compression Method	VGG16 Model Layers
		Linear	Activation	Convolution
MNIST	Quantization [36]	4.2%	3.9%	4.1%
	Proposed Quantile	49%	58%	50%
CIFAR-10	Quantization [36]	3.3%	3.1%	3.7%
	Proposed Quantile	49%	58%	50%

Compression ratio shows percentage reduction in file size using (original size − compressed size)/original size × 100.

compares two compression methods, the quantization-based FL model [36] and the proposed quantile-based FL model, across the MNIST and CIFAR-10 datasets using the VGG16 model, specifically for three layers: the linear layer, the activation layer, and the convolution layer. These findings show that the proposed quantile-based FL approach achieves compression ratios 11 to 18 times higher than quantization across all layers in both datasets. Similarly, in

Table 3. Compression ratio results for different layers (Act, and Conv) in the ResNet18 model on MNIST and CIFAR-10 datasets.

Dataset	Compression Method	ResNet18 Model Layers
		Activation	Convolution
MNIST	Quantization [36]	4.6%	4.6%
	Proposed Quantile	54%	49%
CIFAR-10	Quantization [36]	3.9%	4.6%
	Proposed Quantile	54%	49%

Compression ratio shows percentage reduction in file size using (original size − compressed size)/original size × 100.

, a comparison between the same methods was conducted across the MNIST and CIFAR-10 datasets using the RESNET18 model for the activation and convolution layers. Here, the Quantile-based approach achieves compression ratios 10 to 14 times higher than Quantization across both layers. This demonstrates that the quantile-based method outperforms quantization in terms of compression efficiency.

In

Table 4. Accuracy results for the ResNet18 model with MNIST and CIFAR-10 datasets.

Dataset	Compression Method	Accuracy
		Train	Test
MNIST	Quantization [36]	99.99%	99.36%
	Proposed Quantile	100%	99.2%
CIFAR-10	Quantization [36]	85.29%	83.35%
	Proposed Quantile	88.08%	85.63%

, a comparison between two compression methods is performed on the ResNet18 model and evaluated using the MNIST and CIFAR-10 datasets, where the performance is the comparison criteria. The compression methods examined are the baseline quantization method [36] and our quantile-based compression method. The table presents the training and testing accuracy attained by each method for both datasets. For MNIST, the quantile-based method shows slightly improved accuracy compared to quantization. Similarly, for the CIFAR-10 dataset, the quantile-based method maintains competitive accuracy. These outcomes confirm the effectiveness of the quantile-based compression method in preserving model accuracy without any information loss.

Table 5. Statistical comparison of accuracy for MNIST.

Model	Accuracy (%)		$\mathbf{\Delta }$ (%)	p-Value	Effect Size (r)	Practical Impact
	Quantization [35] Mean ± Std	Proposed Quantile Mean ± Std
CNN	$98.006\pm 0.013$	$98.104\pm 0.013$	$-0.098$	<0.001 ***	$1.00$ (large)	Negligible
MLP	$95.116\pm 0.076$	$95.000\pm 0.000$	$+0.116$	<0.001 ***	$0.80$ (large)	Negligible
Linear	$90.866\pm 0.102$	$90.870\pm 0.000$	$-0.004$	0.014 *	$0.60$ (medium)	Negligible

$\Delta$ = With Quantization − Without Quantization. All models evaluated using Mann–Whitney U test across 10 independent runs. A negative $\Delta$ indicates a marginal accuracy reduction due to quantization; a positive $\Delta$ indicates a marginal gain. All absolute differences are sub-$0.12\%$, indicating no practically meaningful impact. *** p < 0.001, * p < 0.05.

presents the final validation accuracy comparison between the proposed PCodec framework and conventional quantization [35] across all three architectures (Linear Regression, MLP, and CNN) for the MNIST dataset over 10 independent runs. The key finding is that PCodec achieves accuracy statistically equivalent to conventional quantization across all architectures while delivering a substantially higher compression ratio, making it the more efficient choice for bandwidth-constrained federated learning.

Mann–Whitney U tests confirm statistically significant differences in all cases ($p<0.001$ for CNN and MLP; $p=0.014$ for linear regression); however, absolute differences range from only $0.004$ to $0.116$ percentage points, rendering them operationally trivial. For the CNN, PCodec marginally outperforms conventional quantization ($98.104\%\pm 0.013\%$ vs. $98.006\%\pm 0.013\%$, $\Delta =+0.098\%$). For the MLP, the difference favours conventional quantization by an equally negligible margin ($95.116\%\pm 0.076\%$ vs. $95.000\%\pm 0.000\%$, $\Delta =-0.116\%$), while PCodec exhibits perfectly stable convergence across all runs. For the linear classifier, both methods are practically identical ($90.866\%\pm 0.102\%$ vs. $90.870\%\pm 0.000\%$, $\Delta =+0.004\%$).

These results collectively demonstrate that PCodec preserves classification accuracy across all architectures with no meaningful performance penalty relative to conventional quantization. Combined with its superior compression ratio, PCodec presents a more favourable accuracy-compression trade-off, affirming its suitability as an efficient alternative for federated learning.

Figure 2 visually confirms the numerical results presented in Table 5. These results show the accuracy differences between the proposed Pcodec and the conventional quantization [35] across all three architectures (Linear, MLP, and CNN) on the MNIST dataset, with absolute gaps ranging from only 0.004% to 0.116%.

Statistical validation across $n=10$ independent runs confirms that the proposed Quantile method achieves higher classification accuracy than Quantization [35] on CIFAR-10 under both evaluated configurations of quantization, namely $R=2,\epsilon =3$ and $R=1$, $\epsilon =3$ (

Table 6. Classification accuracy on the CIFAR-10 dataset (CNN3 model) over $n=10$ independent runs, reported as Mean ± Std (%). Normality is assessed via Shapiro–Wilk; pairwise comparison uses Mann–Whitney U test. *** p < 0.001.

Config	Method	Final Val Acc (%)	Best Val Acc (%)	Test	p-Value
$R=2,\epsilon =3$	Quantization [35]	$68.70\pm 0.48$	$69.00\pm 0.00$	Mann–Whitney U	$7.5\times {10}^{-5}$***
	Proposed Quantile	$70.00\pm 0.00$	$70.00\pm 0.00$
$R=1,\epsilon =3$	Quantization [35]	$68.00\pm 0.00$	$69.00\pm 0.00$	Mann–Whitney $U^{†}$	$<0.001$***
	Proposed Quantile	$70.00\pm 0.00$	$70.00\pm 0.00$

^† Under R = 1 both groups exhibit zero variance; the rank-biserial correlation r = +1.00 confirms complete stochastic dominance of the Proposed Quantile method over Quantization [35].

, $p<0.001$ in all cases). Under $R=2,\epsilon =3$, the proposed Quantile attains a mean final validation accuracy of $70.00\%$, compared to $68.70\pm 0.48\%$ for Quantization [35]. Under the stricter $R=1,\epsilon =3$ configuration, the gap is maintained, with the proposed Quantile reaching $70.00\%$ versus $68.00\%$ for Quantization.

A particularly revealing observation is the zero standard deviation ($\pm 0.00\%$) of the Proposed Quantile method under both configurations, reflecting the deterministic and perfectly reproducible nature of lossless compression. In contrast, Quantization [35] exhibits non-trivial run-to-run variability (std $=0.48\%$ under $R=2$), attributable to the stochastic noise introduced by its lossy lattice-based mechanism. This behavioral difference provides direct empirical support for the theoretical advantage of Pcodec: because model weights are recovered exactly after each communication round, no compression-induced distortion accumulates across FL rounds, unlike lossy methods whose errors compound over successive aggregations.

The results demonstrated that our proposed method delivers substantial advantages in compression efficiency while maintaining model accuracy comparable to the baseline. The most important difference is that Pcodec is lossless by nature, which means that the compressed data can be decompressed to obtain the exact original model weights, without any loss of data. In contrast, quantization-based methods are lossy by nature and permanently discard numerical precision. These findings directly addressed Q1 by demonstrating that the proposed method reduced communication overhead effectively, thereby speeding up data transmission from edge networks to the central server—and Q2 by demonstrating that the proposed method enhanced the communication efficiency without compromising the accuracy of the learned model. Nevertheless, the technique introduces some overhead in the compression and decompression time; this was not considered a significant limitation, as model training and compression are typically offline tasks, and the additional time did not affect real-time inference or deployment. Furthermore, this approach did not incorporate any privacy-preserving mechanisms. However, this should not affect the original model.

4.3. Study Limitation

One limitation of the present study is the absence of a detailed statistical analysis. Standard deviation, variance, and repeated experimental runs were not reported, which may constrain the confidence in the observed performance differences. In addition, while the proposed Pcodec method excels at compression efficiency, it does not offer any privacy enhancement, unlike approaches such as JOPEQ. This may limit its adoption in privacy-critical federated learning deployments unless combined with additional privacy-preserving techniques. Furthermore, the approach produces some overhead in compression and decompression time compared to the traditional FL. However, this is not considered a substantial constraint due to the offline compression and training process. In future work, we intend to address the statistical limitation by conducting extensive studies, incorporating multiple repetitions and measures of variability, to provide a more robust and reliable evaluation of our method’s performance, as well as exploring the integration of privacy-preserving techniques to make the method suitable for sensitive data scenarios.

5. Conclusions

A quantile-based lossless compression approach, Pcodec, is proposed to compress model updates in the FL context. Experiments demonstrate that Pcodec achieves high compression ratios: for the MNIST dataset, ratios of 68%, 55%, and 53% were achieved using linear regression, MLP, and CNN models, respectively, and in the CIFAR-10 dataset, 63%, 86%, and 86% were achieved. Utilizing the VGG16 model, the compression ratios achieved in three different layers were 49%, 58%, and 50% for both the MNIST and CIFAR-10 datasets. Finally, using the ResNet18 model on the MNIST and CIFAR-10 datasets in two different layers achieved 54%, 49%, 54%, and 49%. Due to its lossless compression nature, Pcodeccompresses weights without affecting the model’s accuracy. Experiments then proved that its results outperformed quantization results using various models on different datasets, achieving communication overhead reduction between server and edge network devices up to 2.81 times without sacrificing the model accuracy. Compared to quantization, the quantile approach reduces communication overhead by 2.74 times. Thus, it addresses the main challenge in the FL context. The approach produces some overhead in compression and decompression time compared to the traditional FL. However, this is not considered a substantial constraint due to the offline compression and training process. In addition, this approach did not incorporate any privacy-preserving mechanisms. However, this should not affect the original model. In future work, quantile-based lossless compression will be evaluated in more environments and on a broader range of datasets, and the integration of privacy-preserving mechanisms will be investigated.