A Comparative Study of Secure Outsourced Matrix Multiplication Based on Homomorphic Encryption

Abstract

Homomorphic encryption (HE) is a promising solution for handling sensitive data in semi-trusted third-party computing environments, as it enables processing of encrypted data. However, applying sophisticated techniques such as machine learning, statistics, and image processing to encrypted data remains a challenge. The computational complexity of some encrypted operations can significantly increase processing time. In this paper, we focus on the analysis of two state-of-the-art HE matrix multiplication algorithms with the best time and space complexities. We show how their performance depends on the libraries and the execution context, considering the standard Cheon–Kim–Kim–Song (CKKS) HE scheme with fixed-point numbers based on the Microsoft SEAL and PALISADE libraries. We show that Windows OS for the SEAL library and Linux OS for the PALISADE library are the best options. In general, PALISADE-Linux outperforms PALISADE-Windows, SEAL-Linux, and SEAL-Windows by 1.28, 1.59, and 1.67 times on average for different matrix sizes, respectively. We derive high-precision extrapolation formulas to estimate the processing time of HE multiplication of larger matrices.

1. Introduction

Third-party services offer a convenient alternative to build, complement, or extend their infrastructures. They can provide convenient data access, unlimited storage, and processing capacities [1]. However, storing and processing sensitive data (e.g., medical records) requires selecting a third-party provider with a high level of data protection. To provide this type of protection, various techniques are used, including homomorphic encryption (HE), which can process encrypted data [2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31]. In addition, homomorphic encryption is part of post-quantum cryptography (PQC), which aims to avoid the risk of attacks by quantum computers. There are several software implementations of PQC methods with hardware acceleration of cryptographic primitives [32,33,34]. In this study, we focus on the software implementation of matrix multiplication using Microsoft libraries SEAL and PALISADE under Windows and Linux operating systems.

HE defines a class of encryption techniques that allow performing mathematical operations on encrypted data, generating results that correspond to the results of operations on the plaintext, without information about the secret key and access to the raw data [35]. Initially, Partially Homomorphic Encryption (PHE) and Somewhat Homomorphic Encryption (SHE) techniques offered only a limited number of operations, only one type, or a predetermined set. This limited its applicability to a small range of problems. Fully Homomorphic Encryption (FHE) brought another breakthrough by allowing unrestricted addition and multiplication operations on encrypted data.

The main idea of FHE can be presented as follows. Let $m_k$ be a plaintext and $ek$ the encryption key. Then, $Enc\left(ek,m_k\right)$ defines the encryption of $m_k$ via HE with $ek$ and the encryption function $Enc$. Thus, FHE operations can be represented as

$$Dec\left(Enc\left(m_1\right)\otimes Enc\left(m_2\right)\right)=m_1\ast m_2,$$

(1)

$$Dec\left(Enc\left(m_1\right)\oplus Enc\left(m_2\right)\right)=m_1+m_2,$$

(2)

where $Dec$ is a decryption function, $\otimes$ defines homomorphic multiplication, and $\oplus$ is homomorphic addition. Thus, these schemes are homomorphic over both operations.

Not all operations can be easily performed in the HE domain. In particular, many non-modular and matrix operations are time consuming [36]. This is especially true for matrix multiplication, a common operation in many algorithms [37,38,39,40,41].

Matrix multiplication is a fundamental operation for data processing. It is used in a wide range of data processing and analysis algorithms, including principal component analysis, linear regression, image processing, and neural networks [42,43,44,45].

Multidimensional packing is a technique used in HE to pack multiple values into a single ciphertext, which can then be homomorphically processed [46]. This technique can be used to perform approximate matrix arithmetic by packing each row of a matrix into a single ciphertext.

To pack a matrix using multidimensional packing, we must first choose a packing factor that determines how many values are packed into each ciphertext. Then, each row of the matrix is divided into blocks of the size of the packing factor and each block is packed into a single ciphertext.

Once the matrix is packed, we can perform approximate matrix arithmetic by homomorphically processing the ciphertexts. For example, to perform a matrix addition, we can add the corresponding ciphertexts element by element. Similarly, we can perform matrix multiplication by first packing the second matrix and then performing a series of homomorphic operations to compute the product.

It is important to note that some approximation errors may occur in multidimensional packing, since the packed values may not be exact representations of the original matrix elements. However, these errors can be controlled by choosing an appropriate packing factor and carefully selecting the parameters of the HE scheme [47].

In this paper, we provide for the first time a detailed analysis of the practical applicability of current HE matrix multiplication algorithms, discuss bottlenecks and further directions for their efficient implementation. We examine the key technical and theoretical aspects that distinguish algorithms and libraries. We then present performance benchmarks and the main use cases.

We analyze efficiency as a function of several factors: algorithm, implementation, programming language, operating system (OS), and HE scheme.

The main contributions of the paper can be summarized as follows.

• We provide a detailed analysis of the state of the art in HE matrix multiplication algorithms with fixed-point numbers.
• We compare implementations of algorithms with the best time and space complexity based on the Microsoft SEAL [48] and PALISADE [49] libraries.
• We evaluate the impact of different operating systems and libraries on their performance.
• We apply curve fitting to derive high-precision extrapolation formulas for homomorphic multiplication of larger matrices.

We consider space complexity in the analysis of the size of axillary cipher matrices, time complexity, and multiplicative depth. The implementations are based on C++ to ensure comparability between the analysis and the Cheon–Kim–Kim–Song (CKKS) method [48]. CKKS is a pioneer in enabling approximate computations over fixed-point numbers, which are critical for machine learning and deep learning applications (see Appendix A).

There are three reasons for the selection of the two FHE libraries. First, they have been continuously developed and adapted to the changing needs in the field. Second, they work with the promising scheme of CKKS and allow the implementations to run on multiple operating systems. Third, for comparability reasons, they offer the possibility of implementing algorithms in a single programming language.

Large cloud service providers (CSPs) such as Amazon EC2 and Google Cloud typically offer computing services that run on Linux and Windows operating systems. Therefore, we analyze the performance of our implementations on both operating systems.

The paper is organized as follows. Section 2 reviews related work in this area. Section 3 describes the current HE libraries and their features: programming language, supporting numbers, HE schemes, etc. Section 4 describes state-of-the-art secure matrix multiplication methods and their comparison. Section 5 presents the experimental analysis. Section 6 presents an extrapolation to estimate the execution time of a homomorphic matrix multiplication of arbitrary size. Section 7 discusses the use of HE to implement privacy-preserving matrix operations. Finally, Section 8 summarizes the main results, their implications, and further research directions.

2. Related Work

2.1. Privacy-Preservation in Deep Learning

Privacy preservation in deep learning includes techniques and methods to ensure that individuals’ sensitive information, such as personal information, financial information, and medical information, is not exposed or compromised during the training or use of deep learning models.

There are several techniques for maintaining privacy in deep learning, including:

Differential Privacy: This is a mathematical framework that adds noise to data before they are processed by a deep learning model. This ensures that the model cannot learn individual-level information, thus preserving the privacy of the data [50].

Federated Learning: This technique allows training a deep learning model on decentralized data. Instead of collecting all the data in a centralized location, the data are kept locally and only the model parameters are shared between devices or nodes [51]. This approach ensures that the data are not exposed and individual privacy is preserved.

Homomorphic Encryption: This is a technique that allows computations to be performed on encrypted data without decrypting them. This technique ensures that the data remain encrypted throughout the computation process, thus preserving the privacy of the data [2,3,4,5,6,7,8,9,10,11].

Secure Multi-Party Computation: This is a technique that allows multiple parties to participate in a computation without revealing their input [52]. This approach ensures that the data remain private during the computation process.

Overall, privacy preservation is critical in deep learning, especially for applications involving sensitive data. The above techniques and methods can be used to ensure that the privacy of individuals is maintained when using deep learning models.

2.2. Matrix Multiplication in Privacy-Preserving Neural Networks

Matrix operations are commonly used in the development of privacy-preserving neural networks (PPNNs) for various tasks such as image recognition, natural language processing, and speech recognition. Some of the key places where matrix operations are used in PPNN are:

Data preparation: Data are typically represented as a matrix in PPNN, where each row of the matrix represents an input example and each column represents a feature of the example.

Weight initialization: In a PPNN, the weights connecting neurons in different layers are typically initialized as random matrices.

Forward propagation: In forward propagation, the input matrix is multiplied by the weight matrix of the first layer, an activation function is applied to the resulting matrix, and the result is passed to the next layer [53].

Backpropagation: In the backpropagation process, the gradients of the loss function are calculated with respect to the weights of the PPNN, which is usually performed using matrix calculus [54].

Gradient descent: The weights of the network are updated using an optimization algorithm such as gradient descent, where the gradients are multiplied by a learning rate and the result is subtracted from the current weights [55].

Convolutional layers: In convolutional neural networks (CNNs), the convolutional operation is performed using matrix multiplication between the input and the filter kernel [56].

Overall, matrix operations are an essential part of neural network development and training, as they enable efficient computation of complex mathematical operations with large amounts of data.

3. Homomorphic Encryption Libraries

Rivest, Adleman, and Dertuzos [57] published the initial efforts to construct a homomorphic cipher in 1978. This work provided an essential theoretical foundation for HE. After several decades, the construction of an efficient homomorphic cipher that can be used in practice is still an open question [58].

The first FHE scheme was proposed by Craig Gentry in 2009 [59]. Several modifications have been made to the existing FHE schemes and new schemes have been proposed. For instance, Martin Van Dijk, Craig Gentry, Shai Halevi, and Vinod Vaikuntanathan [30] developed a simpler FHE approach based on [50]. This FHE uses integer arithmetic instead of ideal lattice calculations.

Some of these schemes are the basis of the Homomorphic Encryption Standard [60]: The Brakerski–Gentry–Vaikuntanathan (BGV) scheme developed in 2011 [6], the scale-invariant Brakerski/Fan-Vercauteren (BFV) and the Nth-degree TRUncated polynomial ring (NTRU)-based López–Tromer–Vaikuntanathan (LTV) schemes, which was developed in 2012 [61,62,63], the Gentry–Sahai–Waters (GSW) method and the YASHE method, which appeared in 2013 [63,64], and CKKS scheme, which was developed in 2017.

Despite the progress in this field, researchers are still trying to solve performance and memory problems to make HE technique mature for real applications [12,13,14,15,65,66,67,68]. This situation also affects matrix operations [33,69,70], especially matrix multiplication for privacy-preserving machine and deep learning applications [36,71,72,73,74].

Advances in theoretical foundations have been followed by several open-source implementations of FHE methods. These libraries provide support for various HE schemes, operations, and data types. The choice of a particular library depends on several factors, and understanding its properties facilitates the choice.

Table 1 provides an overview of the HE libraries. It contains the library name, the number type, the development language, the supported operating system, and the physical resources used, such as the central processing unit (CPU) and the graphics processing unit (GPU).

The Homomorphic Encryption Library (HElib) [69] is an open-source software library developed by IBM in 2013. It supports the integer BGV method with bootstrapping and the CKKS method for approximate value arithmetic.

Simple Encrypted Arithmetic Library (SEAL) [48] is a library developed by Microsoft in 2015. It supports the BFV scheme for working with integers and the CKKS scheme for working with fixed-point numbers. SEAL implements most of the operations associated with HE, including encoding/decoding with single and vector inputs. Homomorphic arithmetic operations are also available. However, the library is not capable of performing operations with matrices.

PALISADE [49] is an open-source lattice crypto software library developed by the New Jersey Institute of Technology (NJIT) in 2017. It implements schemes such as BGV, BFV, CKKS, FHEW, and the TFHE variant that includes bootstrapping.

Homomorphic Encryption for Approximate Numbers (HEAAN) [58] is an open source HE library developed by Seoul National University (SNU). It was developed to implement the CKKS method, and its first version was released in 2016. The multiplication is accelerated by using Fast Fourier Transform (FFT) and Number Theoretic Transform (NTT).

Fastest Homomorphic Encryption in the West (FHEW) [75] is an open-source library developed by the Defense Advanced Research Projects Agency (DARPA) and the University of California (UCSD). It is based on the Fastest Fourier Transform in the West (FFTW) library, which was released in 2017 [76].

TFHE: Fast Fully Homomorphic Encryption over the Torus [77] is an open source HE library released in 2017. It is based on a ring variant of the GSW method. A parallel implementation of the TFHE method called NuFHE was released in 2019 [78]. NuFHE provides GPU acceleration using Compute Unified Device Architecture (CUDA) and Open Computing Language (OpenCL).

Lattigo [79] is a library in Go that implements HE based on Ring Learning With Errors (R-LWE). It was developed in 2019 and supports BFV and CKKS methods. It was developed using the residue number system (RNS).

Λ◦λ (“LOL”, or Lattice Cryptography Library) [80] is a Haskell library released in 2016. It focuses on functional lattice cryptography and implements the BGV scheme. Λ◦λ has the ability to integrate specialized backends (e.g., GPUs).

CUDA Homomorphic Encryption Library (cuHE) [81] is a GPU-accelerated library for HE. It implements the Doröz-Hu-Sunar (DHS) SHE method [82], which is based on the LTV method. This library was released in 2016 and has not been updated.

Concrete [83] is an FHE library created in 2020 that implements the variant of Zama’s TFHE scheme [77]. It was developed using the fast and secure programming language Rust.

cuFHE [84] is a library developed in 2018. It implements the TFHE scheme on CUDA-enabled GPUs. cuFHE reports a 26× speedup in gate-by-gate bootstrapping performance compared to the TFHE library for the CPU version.

cuYASHE [85] is an open-source library developed in 2016 by the University of Campinas. It implements the CUDA-accelerated version of the YASHE process [86]. The authors report a 6- to 35-fold improvement in polynomial multiplication compared to implementations on CPU, GPU, and FPGA.

Node-seal [86] is a version of the Microsoft SEAL library adapted for TypeScript or JavaScript, and was first released in 2019. Node-seal provides the fastest web implementation that works in any server/client configuration.

Python for Homomorphic Encryption Libraries (Pyfhel) [87] provides functionalities of FHE libraries in Python. The current version supports only Microsoft SEAL. The library was developed based on the abstraction for homomorphic encryption libraries (Afhel).

Table 1. Main properties of most common HE libraries.

Library	Numbers with Fixed-Point		Integer Numbers						Language	OS		CPU	GPU	Ref.
	CKKS	TFHE	BFV	BGV	LTV	DHS	FHEW	YASHE		Windows	Linux
Microsoft SEAL	●		●						C++	●	●	●		[48]
PALISADE	●	●	●	●			●		C++	●	●	●		[49]
HEAAN	●								C++		●	●		[58]
cuYASHE								●	C++		●		●	[85]
HElib	●			●					C++		●	●		[69]
FHEW							●		C++		●	●		[75]
TFHE		●							C++		●	●		[77]
NuFHE		●							Python	●	●		●	[78]
Lattigo	●		●						Go	●	●	●		[79]
$\mathsf{\Lambda }◦\mathsf{\lambda }$				●					Haskell	●	●	●	●	[80]
cuHE					●	●			C++	●	●		●	[81]
Concrete		●							Rust		●	●		[83]
cuFHE		●							C++		●		●	[84]
node-seal	●		●						TypeScript	●	●	●		[86]
Pyfhel	●		●						Python	●	●	●		[87]
SEAL-python	●		●						Python	●	●	●		[88]

Table 1. Main properties of most common HE libraries.

Library	Numbers with Fixed-Point		Integer Numbers						Language	OS		CPU	GPU	Ref.
	CKKS	TFHE	BFV	BGV	LTV	DHS	FHEW	YASHE		Windows	Linux
Microsoft SEAL	●		●						C++	●	●	●		[48]
PALISADE	●	●	●	●			●		C++	●	●	●		[49]
HEAAN	●								C++		●	●		[58]
cuYASHE								●	C++		●		●	[85]
HElib	●			●					C++		●	●		[69]
FHEW							●		C++		●	●		[75]
TFHE		●							C++		●	●		[77]
NuFHE		●							Python	●	●		●	[78]
Lattigo	●		●						Go	●	●	●		[79]
$\mathsf{\Lambda }◦\mathsf{\lambda }$				●					Haskell	●	●	●	●	[80]
cuHE					●	●			C++	●	●		●	[81]
Concrete		●							Rust		●	●		[83]
cuFHE		●							C++		●		●	[84]
node-seal	●		●						TypeScript	●	●	●		[86]
Pyfhel	●		●						Python	●	●	●		[87]
SEAL-python	●		●						Python	●	●	●		[88]

SEAL-python [88] is a header-only library that allows the use of Microsoft’s SEAL library in Python. It was developed in 2020 by the Cryptography Research Group at Microsoft. SEAL-python can include source code in a Docker image.

Not all libraries provide support for Windows and Linux operating systems. This is especially important when performing HE operations on a third-party infrastructure, such as a CSP’s infrastructure.

Amazon EC2, one of the leading CSPs, supports six different virtual machine (VM) configurations with Linux operating systems and four configurations when running a Windows operating system. Similarly, Google Cloud offers seven different configurations of VMs running a Linux family operating system and only two configurations running a Windows family operating system.

A library that is compatible with both operating systems increases interoperability and facilitates transfer between CSPs and services (VM types).

Therefore, in our experimental setup, we consider probably the most practical operating systems from the Linux and Windows families.

4. Secure Matrix Multiplication

Homomorphic matrix computation is a fundamental operation for statistical analysis and privacy-preserving machine learning. The algorithms proposed by Halevi and Shoup [69] and Jiang et al. [70] are currently the best state-of-the-art algorithms.

The algorithm of Halevi and Shoup [69] is based on a sequence of matrix-vector multiplications. It encodes each vector of the matrix as plaintext, i.e., a plaintext vector is created. Then, the encrypted matrix is encrypted as a vector of ciphertexts. Finally, the vector of ciphertexts is encrypted into a single ciphertext. The operations are performed with this ciphertext.

In matrix-vector multiplication, the input matrices are encoded in their diagonal representation, i.e., each diagonal is encoded into a ciphertext.

Let a matrix $A$ of size $d\times d$ be given by $a_0,\dots ,a_{d-1}$, where $a_i=\left(A_{0,i},A_{1,i+1},\dots ,A_{d-1,d+i-1}\right)$. Therefore, $a_i\left[j\right]=A_{j,j+i}$. The product $w=vA$, where $v$ is the input vector, can be calculated as $w\leftarrow {\sum }_{i=0}^{n-1}{a}_i\times \left(v⋘i\right)$.

This method requires $d$ rotations, multiplications, and additions. The multiplicative depth is 1.

The algorithm of Jiang et al. [70] is based on the linear transformation of square matrices.

For a matrix $U\in {ℛ}^{n\times n}$, a linear transformation $L:{\mathcal{R}}^n\to {\mathcal{R}}^n$ can be represented as $L:m_e⟼U·m_e$. Thus, the multiplication of a matrix by a vector can be represented by combining the operations of rotation and multiplication by a constant.

For $0\le \ell <n$, $\ell$-th -th diagonal vector $U$ can be determined as

$$\begin{gathered} u_{\ell }=\left(U_{0,\ell }, U_{1,\ell +1}, \dots , U_{n-\ell -1,n-1}, U_{n-\ell ,0}, \dots , U_{n-1,\ell -1}\right)\in {\mathcal{R}}^n \\ U\cdot m_e={\sum }_{0\le \ell <n}\left(u_{\ell }\odot \rho \left(m_e;\ell \right)\right), \end{gathered}$$

(3)

where $\odot$ is a component-wise multiplication between vectors.

$A={\left(A_{i,j}\right)}_{0\le i,j<d}$ is a matrix of size d × d. The permutations of $\sigma ,\tau ,\phi$ and $\psi$ on the set ${\mathcal{R}}^{d\times d}$ are defined as follows.

• $\sigma {\left(A\right)}_{i,j}=A_{i,i+j}$.
• $\tau {\left(A\right)}_{i,j}=A_{i+j,j}$.
• $\phi {\left(A\right)}_{i,j}=A_{i,j+1}$.
• $\psi {\left(A\right)}_{i,j}=A_{i+1,j}$.

The matrix multiplication can be specified by the following formula:

$$A·B={\displaystyle {\sum }_{k=0}^{d-1}}\left({\phi }^k\circ \sigma \left(A\right)\right)\odot \left({\psi }^k\circ \tau \left(B\right)\right),$$

(4)

where $\circ$ denotes the function composition.

The multiplication algorithm requires determining the matrix representations corresponding to permutations: $U^{\sigma }, U^{\tau }, V^k,$ and $W^k$. For $0\le i, j<d, 1\le k<d$, and $0\le \ell <d^2$:

$$U_{d·i+j,\ell }^{\sigma }=\left\{\begin{array}{c}1 \mathrm{if} \ell =d·i+{\left[i+j\right]}_d\\ 0\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\mathrm{otherwise}\hfill \end{array}\right.;$$

(5)

$$U_{d·i+j,\ell }^{\tau }=\left\{\begin{array}{c}1 \mathrm{if} \ell =d·{\left[i+j\right]}_d+j\\ 0\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\mathrm{otherwise}\hfill \end{array}\right.;$$

(6)

$$V_{d·i+j,\ell }^k=\left\{\begin{array}{c}1 \mathrm{if} \ell =d·i+{\left[j+k\right]}_d\\ 0\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\mathrm{otherwise}\hfill \end{array}\right.;$$

(7)

$$W_{d·i+j,\ell }^k=\left\{\begin{array}{c}1 \mathrm{if} \ell =d·{\left[i+k\right]}_d+j\\ 0\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\mathrm{otherwise}\hfill \end{array}\right..$$

(8)

Multiplication of two encrypted matrices $\mathrm{c}\mathrm{t}.A$ (ciphertext $A$) and $\mathrm{c}\mathrm{t}.B$ (ciphertext $B$) is performed as follows.

Step 1.1. Linear transformation $U^{\sigma }$ on the input $\mathrm{c}\mathrm{t}.A$:

$$U^{\sigma }·a=\sum _{-d<k<d}\left(u_k^{\sigma }⨀\rho \left(a;k\right)\right),$$

(9)

where $a=i^{-1}\left(A\right)\in {\mathcal{R}}^n$ is a vector representation of $A$.

This expression can be computed in the HE scheme as

$$\sum _{-d<k<d}\mathrm{C}\mathrm{M}\mathrm{u}\mathrm{l}\mathrm{t}(\mathrm{R}\mathrm{o}\mathrm{t}\left(\mathrm{c}\mathrm{t}.A;k\right);u_k^{\sigma }).$$

(10)

Step 1.2. Linear transformation $U^{\tau }$ on the input $\mathrm{c}\mathrm{t}.B$:

$$U^{\tau }·a=\sum _{0\le k<d}\left(u_{d·k}^{\tau }⨀\rho \left(b;d·k\right)\right),$$

(11)

where $b=i^{-1}\left(B\right)\in {\mathcal{R}}^n$ and $u_{d·k}^{\tau }$ is a diagonal vector of $U^{\tau }$.

In HE, it is represented as follows:

$$\sum _{0\le k<d}\mathrm{C}\mathrm{M}\mathrm{u}\mathrm{l}\mathrm{t}(\mathrm{R}\mathrm{o}\mathrm{t}\left(\mathrm{c}\mathrm{t}.B;d·k\right);u_{d·k}^{\tau }).$$

(12)

Step 2. Homomorphic computation of operations $\sigma \left(A\right)$ and $\tau \left(B\right)$. For $1\le k<d$, the column shifting matrix $V^k$ contains two non-zero diagonal vectors $v_k$ and $v_{k-d}$:

$${\mathit{v}}_k\left[\ell \right]=\left\{\begin{array}{c}1 \mathrm{if} 0\le {\left[\ell \right]}_d<(d-k)\\ 0\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\mathrm{otherwise}\hfill \end{array}\right.,$$

(13)

$${\mathit{v}}_{k-d}\left[\ell \right]=\left\{\begin{array}{c}1 \mathrm{if} (d-k)\le {\left[\ell \right]}_d<d\\ 0\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\mathrm{otherwise}\hfill \end{array}\right..$$

(14)

Adding two ciphertexts $\mathrm{C}\mathrm{M}\mathrm{u}\mathrm{l}\mathrm{t}\left(\mathrm{R}\mathrm{o}\mathrm{t}\left(\mathrm{c}\mathrm{t}.A^{\left(0\right)};k\right);v_k\right)$ and $\mathrm{C}\mathrm{M}\mathrm{u}\mathrm{l}\mathrm{t}\left(\mathrm{R}\mathrm{o}\mathrm{t}\left(\mathrm{c}\mathrm{t}.A^{\left(0\right)};k-d\right);v_{k-d}\right)$ we obtain $\mathrm{c}\mathrm{t}.A^{\left(k\right)}$ and $\mathrm{c}\mathrm{t}.B^k\leftarrow \mathrm{R}\mathrm{o}\mathrm{t}(\mathrm{c}\mathrm{t}.B^{\left(0\right)};d·k)$. This step requires $d$ additions, $2d$ constant multiplications, and $3d$ rotations.

Step 3. At this step, Hadamard multiplication is calculated for the $\mathrm{c}\mathrm{t}.A^{\left(k\right)}$ and $\mathrm{c}\mathrm{t}.B^{\left(k\right)}$ ciphertexts for $0\le k<d$, and we obtain the resulting ciphertext. This step requires $d$ homomorphic additions and multiplications.

Although the method proposed by Jiang et al. [70] requires more depth compared to the method of Halevi and Shoup [69], the algorithm has the lowest time complexity and space complexity (number of ciphertexts) (see

Table 2. Main Properties of matrix multiplication algorithms.

Ref.	Number of Ciphertexts	Complexity	Required Depth	Library				OS
				HElib	HEAAN	Microsoft SEAL	PALISADE	Windows	Linux	Mac OS
[69]	$d$	${O(d}^2)$	1 Mult	●					●
[79]	1	$O\left(d\right)$	1 Mult + 2 CMult		●					●
new	1	$O\left(d\right)$	1 Mult + 2 CMult			●	●	●	●

). These factors are crucial for efficient implementation of privacy-preserving neural networks, especially when processing large amounts of data. Therefore, in our work, we analyze the performance of the method of Jiang et al. [70].

5. Experimental Analysis

In this section all steps of HE matrix multiplication are analyzed: Encoding and encoding of an input matrix, encoding of precomputed auxiliary matrices, execution of matrix multiplication, decoding, and decoding of the result.

The creation of $U^{\sigma }, U^{\tau }, V^k,$ $W^k$, and the diagonal functions is not the subject of our experiments, since these operations are not related to HE and are performed identically in both libraries.

The implementation uses the CKKS scheme with Microsoft SEAL 3.5.6 and PALISADE v1.10.6 libraries on Ubuntu 20.04 and Windows 10 Home Edition. The hardware configuration consists of a CPU Intel Core i5-8250U 1.60 GHz, RAM DDR4 8 GB 1196 MHz and SSD 512 GB. The average time was measured by running the algorithms 1000 times on each platform.

We adopt the security settings specified in the HE standard for both libraries [63], see

Table 3. Security parameters.

Parameter	Security Level	$\mathit{N}$	$\mathbf{log} \mathit{Q}$	$\mathit{\omega }$
Value	128	8,192	220	3.2

.

The results are presented in two-dimensional graphs, with the abscissa indicating the size $d\in \left\{2,3,\dots ,19\right\}$ of the square matrices and the ordinate indicating the time in seconds (sec.).

Each plot shows four curves illustrating the library-operating system combinations labeled SEAL-Linux, SEAL-Windows, PALISADE-Linux, and PALISADE-Windows, see

Table 4. Notations.

Notation	Meaning
SEAL-Linux	Implementation of the algorithm using the SEAL library compiled in OS Linux Ubuntu
SEAL-Windows	Implementation of the algorithm using the SEAL library compiled in OS Windows 10
PALISADE- Linux	Implementation of the algorithm using the PALISADE Library compiled in OS Linux Ubuntu
PALISADE-Windows	Implementation of the algorithm using the PALISADE Library compiled in OS Windows 10
T, s	Time to complete the operation/algorithm in seconds.
d	The matrix order
α	Degradation over PALISADE-Linux (times)

and

Table 5. Implementation characteristics with CKKS scheme.

Name	Library		OS
	SEAL	PALISADE	Windows	Linux
SEAL-Linux	●			●
SEAL-Windows	●		●
PALISADE- Linux		●		●
PALISADE-Windows		●	●

.

The performance degradation is defined as the ratio of the execution time of all combinations to PALISADE-Linux.

5.1. Encoding Time

Figure 1 shows the encoding time of the $U^{\sigma }$ matrix. It can be seen that the most efficient implementation is SEAL-Windows and the worst is SEAL-Linux (see Figure 1a). On average, it takes 0.94 times the time required by PALISADE-Linux, with the best result of 0.85 for the 10 × 10 matrix (Figure 1b).

In the worst case, this implementation outperforms PALISADE-Linux by a factor of 1.15 for a matrix size of 11 × 11.

The results of encoding the matrix $U^{\tau }$ are similar to those of the matrix $U^{\sigma }$ (see Figure 2). The SEAL-Windows implementation takes on average 0.913 times compared to PALISADE-Linux. The best gain is 0.824 times for a 10 × 10 matrix and the worst case is 1.008 times with an 11 × 11 matrix.

The SEAL-Windows implementation also leads in encoding the $V^k$ matrix, see Figure 3, requiring 0.92 times the time of PALISADE-Linux for the same operation. In this case, the best result is also observed for a matrix size of 10 × 10 with 0.82 times. In the worst case, the improvement is about 1.05 times for a 4 × 4 matrix.

As in the previous cases, encoding the $W^k$ matrix with SEAL-Windows is more efficient (see Figure 4). It takes on average only 87% of the time that PALISADE-Linux needs for the same operation. In the best case, the time drops to 0.73 for a matrix size of 19 × 19. The worst case occurs for an 11 × 11 matrix, where the implementation takes 1.09 times more time than PALISADE-Linux.

Figure 5 shows the encoding time of the input matrix and confirms that SEAL-Windows provides the best implementation for encoding. On average, SEAL-Windows takes 0.835 of the time that PALISADE-Linux takes; the best reduction is only 63.1% of the time for an 18 × 18 matrix. The worst case occurs with a 4 × 4 matrix and an increase of 8% of the time. Furthermore, the PALISADE Windows implementation outperforms the PALISADE Linux implementation.

5.2. Encryption Time

For matrix encryption, SEAL-Linux is the most advantageous implementation, followed by PALISADE-Windows (see Figure 6). On average, SEAL-Linux requires only 93.4% of the time required by PALISADE-Linux. The best result is 0.58 times for the 17 × 17 matrix and the worst result is 1.13 times for the 14 × 14 matrix.

Figure 7 shows that the PALISADE-Linux implementation has the best performance in encrypting the input matrix. PALISADE-Windows is the second most efficient implementation with an average time gain of 20% for this operation. The minimum difference between PALISADE-Linux and SEAL-Windows is 1.002 times for the 4 × 4 matrix and the maximum difference is 16.37 times for the 17 × 17 matrix.

5.3. Matrix Multiplication Time

The PALISADE Linux implementation is also the most efficient on average at performing matrix multiplication (see Figure 8). Other implementations take at least 1.3-times more to complete this operation. However, the PALISADE-Windows implementation is more advantageous for small matrices (up to 8 × 8), and its best result is observed for a matrix size of 5 × 5. SEAL-Windows shows the worst performance for a 2 × 2 matrix, taking 6.5 times longer than the PALISADE-Linux implementation.

5.4. Decryption Time

Figure 9 shows the decryption time of the resulting matrix. The SEAL-Windows implementation is the most efficient with an average decryption time of only 0.3 times compared to PALISADE-Linux. The best performance is observed for SEAL-Windows with the 5 × 5 matrix, requiring 0.15 of the time spent by PALISADE-Linux. SEAL-Linux performs worst with the 16 × 16 matrix, taking 2.06 times as much time as the PALISADE-Linux implementation.

5.5. Decoding Time

The PALISADE Linux implementation is the most efficient in decoding, see Figure 10. Other implementations take at least 69.6 times longer to complete this process. This result is observed for the implementation with a matrix size of 15 × 15. The worst performance is provided by SEAL-Linux with a matrix size of 14 × 14 and takes more time than PALISADE-Linux.

5.6. Execution Time

Figure 11 shows the execution time of the entire algorithm. All implementations require on average at least 1.28 times more time to execute the entire algorithm. In general, PALISADE-Linux outperforms PALISADE-Windows, SEAL-Linux, and SEAL-Windows by 1.28, 1.59, and 1.67 times, respectively, on average.

However, for matrices from 5 × 5 to 8 × 8, the PALISADE-Windows implementation is more efficient than PALISADE-Linux, taking 0.91 times on average. The SEAL-Windows implementation takes five times longer than the PALISADE-Linux implementation for the 2 × 2 matrix.

The difference in efficiency between the libraries and systems considered is that the PALISADE library is optimized for HE implementation under the Linux operating system, while Microsoft SEAL is optimized for HE implementation under the Windows operating system.

6. Extrapolation

To obtain benchmarks for approximate values outside the tested range of values, we further extrapolate the obtained curves by computing the approximate polynomials and reliability values. Deriving the approximate values outside the tested range allows us to evaluate the computational resources for implementing matrix multiplication in a cloud environment.

We estimate the efficiency of the HE matrix multiplication algorithm by inferring unknown values from trends in the known data.

The least squares method allows fine-tuning of the numerical parameters of a model function to fit a data set as well as possible.

The dataset of resulting measurements consists of n = 18 observations $(d_i,T_i)$, $i={\displaystyle \overline{1,n}}$, where $d_i\in \{\mathrm{2,3},\dots ,19\}$ is the order of a square matrix, and $T_i$ defines the execution time of the entire algorithm. Estimates of the execution time outside the original observation can be found based on the relationship between the observed time and the order of the square matrix.

We use a polynomial extrapolation of the form:

$$f\left(d\right)=\sum _{i=0}^m{a}_i{d}^i$$

(15)

where $m$ is the degree of a polynomial $f\left(d\right)$ and $a_i$ is a coefficient of the polynomial $f\left(d\right)$.

The residual is a deviation measure used to evaluate the fit of a model to a data point. It is defined as the difference between the actual value of the dependent variable and the value predicted by the model:

$$r\left(d_i,T_i\right)=f\left(d_i\right)-T_i$$

(16)

The least-squares method finds the best parameter values by minimizing the sum $S_{res}$ of squared residuals:

$$S_{res}=\sum _{i=1}^n{r}^2(d_i,T_i)\to min$$

(17)

The coefficients $a_i$ are calculated by solving the matrix equation $B\times A=C$ where $b_{i,j}={\sum }_{k=1}^n{d}_k^{i+j-2}$, $c_i={\sum }_{k=1}^n{T_{k}d}_k^{i-1}$, and $i,j\in \overline{1,m+1}$, see, Equation (15).

We use the coefficient of determination $R^2$ to estimate the polynomial extrapolation; it shows the degree of agreement of the mathematical model with the original data. This value can be between 0 and 1. The closer the value is to 1, the more accurately the model describes the available data. The value of $R^2$ is calculated according to the following formula:

$$R^2=1-\frac{S_{res}}{S_{tot}}$$

(18)

where $S_{tot}={\sum }_{i=1}^n{\left(T_i-\overline{T}\right)}^2$ and $\overline{T}=\frac{1}{n}{\sum }_i^n{T}_i$.

The value of $R^2$ for each implementation depends on the degree of polynomial extrapolation. Therefore, determining a degree of extrapolation with sufficient accuracy is essential.

Figure 12 shows the coefficient $R^2$ for the four implementations (library OS) and six polynomial extrapolations. It can be seen that $R^2$ is greater than 0.99 for all implementations only when the degree of polynomial extrapolation m is equal to or greater than 3.

We compute a third-degree extrapolation polynomial for each implementation as they can provide approximations with sufficient accuracy ($R^2\ge$ 0.99 values):

• SEAL-Windows

$$f_{S-W}\left(d\right)=0.14d^3-0.209d^2+2.315d+2$$

(19)

• SEAL-Linux:

$$f_{S-L}\left(d\right)=0.403d^3-4.440d^2+23.516d-26.728$$

(20)

• PALISADE-Windows:

$$f_{P-W}\left(d\right)=0.807d^3-14.28d^2+82.607d-110.15$$

(21)

• PALISADE-Linux:

$$f_{P-L}\left(d\right)=0.147d^3-0.991d^2+7.588d-9.826$$

(22)

Asymptotically, the degradations of the other implementations compared to PALISADE-Linux (times) for $d\to \mathrm{\infty }$ are given by:

$$\begin{gathered} \underset{d\to \mathrm{\infty }}{\lim } \frac{f_{S-W}\left(d\right)}{f_{P-L}\left(d\right)}=\frac{0.14}{0.147}\approx 0.952; \\ \underset{d\to \mathrm{\infty }}{\lim } \frac{f_{S-L}\left(d\right)}{f_{P-L}\left(d\right)}=\frac{0.403}{0.147}\approx 2.741; \\ \underset{d\to \mathrm{\infty }}{\lim } \frac{f_{P-W}\left(d\right)}{f_{P-L}\left(d\right)}=\frac{0.807}{0.147}\approx 5.490. \end{gathered}$$

After extrapolation analysis, PALISADE-Linux proves to be the best implementation for multiplying square matrices of order less than or equal to 104, since $\frac{f_{S-W}\left(104\right)}{f_{P-L}\left(104\right)}\approx 1.0003$. Similarly, SEAL-Windows would be a recommended option when matrices of order greater than or equal to 105 are used, since $\frac{f_{S-W}\left(105\right)}{f_{P-L}\left(105\right)}\approx 0.9999$.

7. Discussion

HE has the potential to be used in the development of privacy-friendly matrix operations that protect confidential data while enabling efficient processing and analysis. Matrix operations are fundamental to many machine-learning algorithms, including PPNN, and often require the processing of large amounts of data. HE can help protect these data while enabling efficient computation.

Using HE to create privacy-friendly matrix operations offers several benefits:

Improved privacy: HE provides a high level of data protection by encrypting the data before processing. This ensures that data remain secure throughout the computational process.

Efficient processing: HE allows computations to be performed on encrypted data without decryption, significantly reducing the computational cost of privacy-preserving matrix operations.

Flexibility: HE is a universal technique that can be used for various matrix operations, including those used in machine learning.

Compliance with regulations: HE can help organizations comply with regulations by providing a privacy-compliant solution for processing sensitive data.

Despite these benefits, there are also challenges associated with using HE to build matrix operations. These challenges include the high computational cost of homomorphic encryption, which can make it impractical for large-scale matrix operations, and the difficulty of implementing homomorphic encryption in existing matrix operation algorithms.

Overall, HE has significant potential for building privacy-friendly matrix operations, and as the technology advances, it is likely to become an increasingly important tool for protecting sensitive data while enabling efficient processing and analysis.

8. Conclusions

In this paper, we focus on the comparative analysis of two state-of-the-art HE matrix multiplication algorithms with the best time and space complexities for secure outsourcing. We analyze the Cheon–Kim–Kim–Song (CKKS) fixed-point homomorphic encryption scheme based on the Microsoft SEAL and PALISADE libraries on Windows and Linux. We show that the Windows operating system is preferred for the SEAL library and the Linux operating system is the best option for the PALISADE library.

PALISADE-Linux outperforms PALISADE-Windows, SEAL-Linux, and SEAL-Windows by an average of 1.28, 1.59, and 1.67 times, respectively, in most cases.

SEAL-Windows is more efficient at encrypting matrices and decrypting the resulting matrix. SEAL-Linux provides the best implementation for input matrix encryption. PALISADE-Linux shows the best performance in encrypting the input matrix into a single vector, in matrix multiplication, and in decrypting the resulting matrix.

To provide effective guidance in selecting a good implementation, we provide high-precision extrapolation formulas to asymptotically estimate the computation time of HE multiplication of larger matrices.

We found that polynomial extrapolation of at least degree m = 3 has a coefficient of determination $R^2\ge$ 0.99.

In future work, we will investigate the performance of homomorphic matrix multiplication under other factors such as CPU architecture, cache size, bus, memory parameters, libraries, and other scenarios. We will also develop an optimization of matrix multiplication for the SEAL and PALISADE libraries, considering their characteristics.