Fyber: A Flexible-Dimension Post-Quantum Key Encapsulation Mechanism for IoT

Abstract

Following the completion of the NIST post-quantum cryptography standardization, Kyber has been adopted as a key encapsulation mechanism (KEM) for quantum-resistant communication. Although lattice-based KEMs provide strong security and efficiency, most existing designs restrict the cyclotomic ring dimension to powers of two, which limits parameter flexibility for heterogeneous and resource-constrained Internet of Things (IoT) devices. In this paper, we propose Fyber, a post-quantum KEM based on the Module Learning With Errors (M-LWE) problem over a module ring defined by the cyclotomic polynomial $f\left(x\right)=x^n-x^{n/2}+1$, where n is a product of powers of 2 and 3. This construction enables mixed-radix parameter selection and allows finer-grained trade-offs between security and efficiency. To further improve performance on constrained platforms, we introduce an efficient non-Gaussian sampling method. The proposed KEM supports flexible security-level stratification for IoT applications, achieving reduced public key and ciphertext sizes for selected parameter sets at the cost of moderately increased computational overhead compared to Kyber, and fills intermediate security gaps between existing standardized parameter sets.

1. Introduction

With the rapid development of Internet of Things (IoT) systems [1], IoT applications have expanded across diverse domains, including smart homes, cities, transportation, and healthcare [2]. These devices can collect, process, and share data with each other and with external systems over the Internet. To ensure data confidentiality and integrity, key encapsulation mechanisms (KEMs) are required to securely distribute symmetric keys between devices, which are then used for encrypting transmitted data [3]. With the development of quantum computers [4], conventional cryptographic schemes are expected to become vulnerable, which motivates the consideration and deployment of post-quantum cryptography (PQC) in IoT devices.

KEMs based on the Learning With Errors (LWE) problem [5,6] and its variants [7,8] play a central role in establishing secure channels in PQC-enabled systems. In 2015, Langlois et al. [8] proposed the LWE problem on modular lattices (M-LWE). Compared to LWE, cryptosystems based on M-LWE offer greater flexibility in parameter selection and have become a primary focus of mainstream research in recent years. Especially after the National Institute of Standards and Technology (NIST) designated CRYSTALS-Kyber [9] as a standard post-quantum KEM, constructions based on the hardness of lattice problems have been considered to be among the primary contenders for replacing current KEMs that rely on the computational hardness of traditional number-theoretic problems such as integer factorization and discrete logarithm.

Despite its practical success, Kyber (and most existing M-LWE-based KEMs) relies on cyclotomic polynomial rings whose degrees n are restricted to powers of two [9,10,11]. In this case, the n-th cyclotomic polynomial takes the form $x^{n/2}+1$, which is maximally sparse. Moreover, this design choice simplifies the implementation of number-theoretic transforms (NTTs) [12], enabling polynomial multiplication with complexity $O(nlogn)$ rather than $O\left(n^2\right)$. As a result, high performance can be achieved on general-purpose platforms. However, restricting the ring dimension to powers of two significantly reduces the flexibility of parameter selection, making it difficult to achieve well-balanced trade-offs among security, computational efficiency, and memory footprint in IoT environments. In practice, IoT deployments encompass a wide spectrum of devices, ranging from ultra-low-power microcontrollers [13] to more capable gateways [14]. Restricting ring dimensions to powers of two only permits coarse-grained parameter scaling, forcing designers to choose between insufficient security margins with small dimensions or unnecessary computational and memory overhead with large dimensions. To illustrate the flexibility enabled by ring dimensions of the form $n=2^k·3^{\mathcal{l}}$, Table 1 presents representative parameter choices for different classes of IoT devices. These configurations are illustrative examples following standard M-LWE parameterization principles and are intended to demonstrate how non-power-of-two dimensions enable finer-grained trade-offs between security and resource consumption, rather than to define standardized parameter sets. Specifically, ultra-low-power MCUs typically target NIST level-1 security [15] due to strict memory and energy constraints, while mid-range IoT devices can support moderate parameters at level-3 security. Gateways and edge devices, being less resource-constrained, can adopt higher security levels (e.g., level-5) with larger ring dimensions. For instance, on microcontrollers with only tens of kilobytes of SRAM, a power-of-two dimension such as $n=256$ may exceed the memory budget, whereas $n=128$ may not meet security requirements. In contrast, intermediate dimensions such as $n=192$ achieve security levels that are difficult to realize using power-of-two dimensions alone, while significantly reducing key size and memory footprint.

Table 1. Example parameter configurations enabled by non-power-of-two ring dimensions for IoT devices [13,14]. (n is the degree of the cyclotomic polynomial.)

Target Device	n	Security Level	Key Size	Comment
Ultra-low MCU	$192=2^6·3$	128	Smaller	Fits 32 KB RAM
Mid-range IoT	$216=2^3·3^3$	192	Moderate	Balanced trade-off
Gateway	$288=2^5·3^2$	256	Larger	High security

Non-power-of-two cyclotomic rings expand the parameter space by enabling a wider range of ring dimensions, thereby allowing finer-grained security-level tuning and potentially reducing bandwidth and ciphertext expansion for intermediate security requirements. Moreover, many existing cryptanalytic techniques have been primarily studied and optimized for power-of-two cyclotomic rings [16,17,18]. While this does not imply a general security advantage, adopting more diverse cyclotomic structures may reduce exposure to structure-specific attack optimizations and broaden the design space for mitigating known attack strategies. Despite these potential benefits, designing KEMs over general cyclotomic rings is non-trivial and introduces challenges at multiple levels. First, the equality $n{R}^{\vee }=R$ no longer holds, which requires sampling secret keys from the dual ring ${\left(R^{\vee }\right)}^d$ and incurs higher computational complexity. Second, naive coefficient scaling distorts the noise distribution in the embedded space and violates the spherical Gaussian assumption, thereby opening the door to several known attacks [19,20,21]. Finally, NTT cannot be directly applied to accelerate polynomial multiplication in non-power-of-two rings, which further complicates efficient implementations.

Contributions. In this work, we address these challenges by proposing Fyber, a post-quantum CCA-secure KEM based on the Module Learning With Errors (M-LWE) assumption over cyclotomic rings defined by the polynomial $f\left(x\right)=x^n-x^{n/2}+1$, where $n=2^k·3^{\mathcal{l}}$. This mixed-radix ring structure enables flexible parameter selection and supports mixed-radix NTT implementations, allowing fine-grained trade-offs between security, computational efficiency, and memory usage that are well-suited to IoT devices.

To enhance practicality, we develop an efficient non-Gaussian sampling method and a fast polynomial multiplication algorithm tailored to the proposed ring structure. Compared with Kyber, Fyber achieves smaller public key and ciphertext sizes for selected parameter sets while maintaining post-quantum security, and it naturally supports intermediate security levels that are not covered by existing standardized parameter sets. Overall, Fyber demonstrates improved parameter adaptability and security-level stratification for IoT-oriented post-quantum communication.

The remainder of this paper is organized as follows. Section 2 surveys the related work, and Section 3 presents the relevant background definitions. Section 4 describes the proposed public key encryption (PKE) scheme, which is subsequently converted into a key encapsulation mechanism (KEM) using the Fujisaki–Okamoto (FO) transform. Section 5 offers a comprehensive evaluation of the performance of our construction. Finally, Section 6 summarizes our work.

2. Related Work

2.1. Lattice-Based KEMs

In addition to the post-quantum standardized KEM Kyber discussed above, another major class of lattice-based KEMs is built upon the NTRU encryption paradigm [22]. NTRU-based constructions are typically instantiated over cyclotomic rings and, in principle, allow greater flexibility in the choice of ring parameters and algebraic structures. Representative schemes include the NTRU KEM [23] and NTRU Prime [24] submissions to the NIST post-quantum standardization process, the latter of which adopts an alternative ring selection strategy to mitigate certain forms of structural attacks. Although these schemes advanced to the final rounds of the NIST process, they were not ultimately selected for standardization. Compared to Kyber, NTRU-type KEMs follow a different design line and exhibit distinct performance and bandwidth profiles depending on parameter choices and platforms; however, they were not ultimately selected for NIST standardization, whereas Kyber/ML-KEM has been standardized and widely optimized for deployment. Moreover, several works have shown that NTRU implementations can be susceptible to implementation-level attacks, including power analysis and related side-channel attacks [25,26]. In this paper, we focus on constructions based on the M-LWE assumption, which provides a well-understood security foundation, standardized parameterization, and favorable efficiency–security trade-offs compared to alternative lattice-based approaches.

Despite the rich literature on lattice-based KEMs, existing standardized M-LWE-based designs largely restrict ring dimensions to powers of two, which limits parameter flexibility and results in coarse-grained security-level choices. Prior work on alternative lattice structures either follows different design paradigms (e.g., NTRU-type schemes) or does not target fine-grained parameter tuning within the M-LWE framework. In contrast, our work fills this gap by enabling non-power-of-two ring dimensions within an M-LWE-based KEM, allowing finer-grained security-efficiency trade-offs while preserving a standardized security foundation.

2.2. PQC KEMs on Constrained and IoT Platforms

A growing body of work investigates the practicality of post-quantum KEMs on resource-constrained platforms and in IoT deployments, where memory footprint, code size, and energy consumption are often the dominant constraints. Benchmarking frameworks such as pqm4 enable systematic evaluation of NIST PQC candidates on ARM Cortex-M4 microcontrollers and have become a common baseline for comparing embedded performance [27]. Beyond micro-benchmarks, several studies integrate PQC KEMs into real-world communication stacks and evaluate end-to-end performance. For example, post-quantum TLS handshakes on embedded devices have been studied with respect to runtime and memory overheads [28], and protocol-level optimizations such as KEMTLS have been evaluated in IoT-relevant network settings [29]. Recent surveys further summarize the challenges and optimization directions for deploying PQC in IoT systems [30,31].

In addition to protocol and system level evaluations, several works explore algorithmic and implementation-level optimizations tailored to IoT environments. In 2021, Khalid et al. proposed a KEM construction and corresponding co-processor design for lightweight IoT applications based on a new hardness assumption, namely the multilevel approximate R-LWE problem [32]. More recently, a number of studies have focused on improving the efficiency of Kyber-KEM on resource-constrained platforms through implementation-level optimizations. In particular, Kundu et al. [33] and Shehzadi et al. [34] demonstrated that replacing the original hash function components in Kyber with lightweight alternatives can reduce computational overhead and memory usage on embedded devices. However, existing work largely focuses on implementation- and system-level optimizations, and does not explicitly address the need for fine-grained security-level stratification when deploying KEMs in heterogeneous IoT environments.

In summary, prior studies primarily improve the deployability of existing KEMs on constrained platforms through implementation and protocol-level optimizations, but largely assume fixed, coarse-grained parameter sets defined by standardized schemes. The need for fine-grained security-level stratification to accommodate heterogeneous IoT devices remains underexplored. Our work addresses this gap by redesigning the parameterization space of M-LWE-based KEMs, enabling flexible security–performance trade-offs tailored to diverse IoT deployment scenarios.

3. Preliminaries

3.1. Notation

We denote $\mathbb{N},\mathbb{Z},\mathbb{Q},\mathbb{R}$, and $\mathbb{C}$ as the set of all natural numbers, integers, rational numbers, real numbers, and complex numbers, respectively. For a fixed integer m, we denote ${\mathbb{Z}}_m^{*}$ as $\left\{a\right|gcd(a,m)=1,a\in {\mathbb{Z}}_m\}$, where $gcd(*,*)$ means the greatest common divisor function. It follows that the number of the group $|{\mathbb{Z}}_m^{*}|=\phi \left(m\right)$. By default, $\phi$ is the Euler totient function. For a real $a\in \mathbb{R}$, we denote $\lfloor a\rceil$ as the integer closest to a, denote $\lceil a\rceil =\lfloor a+{\displaystyle \frac{1}{2}}\rceil$, and denote $\lfloor a\rfloor =\lfloor a-{\displaystyle \frac{1}{2}}\rceil$. For a real coefficients polynomial $f\in \mathbb{R}\left[x\right]$, we let $\lfloor f\rceil \in \mathbb{Z}\left[x\right]$ indicate that each coefficient is rounded individually.

Fixing a prime q, for any $v\in {\mathbb{Z}}_q$, we denote ${\parallel v\parallel }_{\infty }=|vmod{}^{\pm }q|$. For any $v\in R$, we denote ${\parallel v\parallel }_{\infty }={max}_i{\parallel v_i\parallel }_{\infty }$, where $v_i$ means the i-th coefficient of v. Similarly, for any $\mathbf{v}=(v_1,v_2,\cdots ,v_d)\in R^d$, we let ${\parallel \mathbf{v}\parallel }_{\infty }={max}_i{\parallel v_i\parallel }_{\infty }$ denote the $l_{\infty }$ norm.

Throughout this paper, for a fixed integer $m\ge 2$, we always denote the integer

$$m^{\prime }=\left\{\begin{array}{cc}\hfill m& \mathrm{if}\mathrm{m}\mathrm{is}\mathrm{odd},\hfill \\ \hfill m/2& \mathrm{otherwise}.\hfill \end{array}\right.$$

(1)

And we define the polynomial

$${\Theta }_m\left[x\right]=\left\{\begin{array}{cc}\hfill x^m-1& \mathrm{if}\mathrm{m}\mathrm{is}\mathrm{odd},\hfill \\ \hfill x^{m/2}+1& \mathrm{otherwise}.\hfill \end{array}\right.$$

(2)

3.2. Lattice and Algebraic Background

Cyclotomic ring. When fixing an integer $m\ge 2$, for m-th primitive root of unity ${\zeta }_m$, cyclotomic polynomial is defined by

$$\begin{array}{c}\hfill {\Phi }_m\left(x\right)=\prod _{\begin{array}{c}s=1\\ gcd(s,m)=1\end{array}}^m(x-{\zeta }_m^s),\end{array}$$

which is a monic irreducible polynomial with integer coefficients. The degree of ${\Phi }_m\left(x\right)$ is $\phi \left(m\right)$. The number field $K=\mathbb{Q}\left({\zeta }_m\right)\cong \mathbb{Q}\left[x\right]/\left({\Phi }_m\left[x\right]\right)$ is an extension of $\mathbb{Q}$ by adjoining ${\zeta }_m$ and $[\mathbb{Q}\left({\zeta }_m\right):\mathbb{Q}]=\phi \left(m\right)$. When considering $\mathbb{Q}\left({\zeta }_m\right)$ as an $\phi \left(m\right)$-dimensional vector space over $\mathbb{Q}$, the power basis is defined as ${\left\{{\zeta }_m^i\right\}}_{i\in {\mathbb{Z}}_m^{*}}$.

The set of all algebraic integers in a number field K, denoted as R, forms an integral domain and is referred to as the ring of algebraic integers. For $K=\mathbb{Q}\left({\zeta }_m\right)$, the ring of algebraic integers is $R=\mathbb{Z}\left[{\zeta }_m\right]\cong \mathbb{Z}\left[x\right]/\left({\Phi }_m\left[x\right]\right)$.

In addition to this, by the definition of ${\Theta }_m\left(x\right)$ (2), we can obtain ${\Phi }_m\left(x\right)\mid {\Theta }_m\left(x\right)$. Thus, the ring $\mathbb{Z}\left[x\right]/\left({\Theta }_m\left[x\right]\right)$ is an expansion of the cyclotomic ring $\mathbb{Z}\left[x\right]/\left({\Phi }_m\left[x\right]\right)$. Furthermore, the reduction modulo ${\Phi }_m\left[x\right]$, denoted as $\beta$, is a ring morphism. The power basis of $\mathbb{Z}\left[x\right]/\left({\Theta }_m\left[x\right]\right)$ consists of $\{1,{\zeta }_m,\cdots ,{\zeta }_m^{m-1}\}$ for odd m and $\{1,{\zeta }_m,\cdots ,{\zeta }_m^{{\displaystyle \frac{m}{2}}-1}\}$ for even m.

Module learning with error. Let ${\mathcal{GD}}_{\sigma }$ be a fixed distribution with standard deviation $\sigma$ and $\mathbf{s}\in {\left(R_q^{\vee }\right)}^d$ be a secret vector. The decisional M-LWE problem asks any distinguisher $\mathcal{A}$ to distinguish between $(\mathbf{A},\mathbf{b}=\mathbf{As}+\mathbf{e})$ and $(\mathbf{A},{\mathbf{b}}^{\prime })$ with a negligible advantage, where $\mathbf{A}\leftarrow R_q^{k\times d}$, $\mathbf{e}\leftarrow {\mathcal{GD}}_{\sigma }^k$ and ${\mathbf{b}}^{\prime }\leftarrow R_q^k$.

3.3. PKE and KEM

A public key encryption (PKE) scheme is a tuple $(\mathbf{KeyGen},\mathbf{Enc},\mathbf{Dec})$ of polynomial-time algorithms over a message space $\mathcal{M}$, where $\mathbf{KeyGen}\left(\lambda \right)$ outputs $(pk,sk)$, $\mathbf{Enc}(pk,m)$ outputs a ciphertext c for $m\in \mathcal{M}$, and $\mathbf{Dec}(sk,c)$ outputs m or ⊥. Correctness requires that for all $m\in \mathcal{M}$,

$$Pr\left[\mathbf{Dec}\right(sk,\mathbf{Enc}(pk,m))=m]\ge 1-\mathsf{negl}\left(\lambda \right).$$

A PKE scheme is indistinguishability under chosen-plaintext attacks (IND-CPA) secure [35] if for any polynomial-time adversary $\mathcal{A}$, its advantage in the standard IND-CPA experiment is negligible, i.e.,

$${\mathsf{Adv}}_{\mathrm{PKE},\mathcal{A}}^{\mathrm{IND}\text{-}\mathrm{CPA}}\left(\lambda \right)=\left|Pr\left[b=b^{\prime }:\begin{array}{c}(pk,sk)\leftarrow \mathbf{KeyGen}\left(\lambda \right);\hfill \\ (m_0,m_1,s)\leftarrow \mathcal{A}\left(pk\right);\hfill \\ b\stackrel{\$}{\leftarrow }\{0,1\};\hfill \\ c^{*}\leftarrow \mathbf{Enc}(pk,m_b);\hfill \\ b^{\prime }\leftarrow \mathcal{A}(s,c^{*})\hfill \end{array}\right]-{\displaystyle \frac{1}{2}}\right|.$$

A key encapsulation mechanism (KEM) is a tuple $(\mathbf{KeyGen},\mathbf{Encaps},\mathbf{Decaps})$ of polynomial-time algorithms. The key generation algorithm $\mathbf{KeyGen}\left(\lambda \right)$ outputs a public/secret key pair $(pk,sk)$. The encapsulation algorithm $\mathbf{Encaps}\left(pk\right)$ takes a public key $pk$ as input and outputs a shared secret K together with a ciphertext c. The decapsulation algorithm $\mathbf{Decaps}(sk,c)$ takes the secret key $sk$ and a ciphertext c as input and outputs either the shared secret K or ⊥. Correctness requires that

$$Pr\left[\mathbf{Decaps}\right(sk,c)=K:(K,c)\leftarrow \mathbf{Encaps}(pk\left)\right]\ge 1-\mathsf{negl}\left(\lambda \right),$$

where $(pk,sk)\leftarrow \mathbf{KeyGen}\left(\lambda \right)$. The advantage of an adversary $\mathcal{A}$ against the indistinguishability under chosen-ciphertext attacks (IND-CCA) security of a KEM is defined as

$${\mathsf{Adv}}_{\mathrm{KEM},\mathcal{A}}^{\mathrm{IND}\text{-}\mathrm{CCA}}\left(\lambda \right)=\left|Pr\left[b=b^{\prime }:\begin{array}{c}(pk,sk)\leftarrow \mathbf{KeyGen}\left(\lambda \right);\hfill \\ b\stackrel{\$}{\leftarrow }\{0,1\};\hfill \\ (c^{*},K_0^{*})\leftarrow \mathbf{Encaps}\left(pk\right);\hfill \\ K_1^{*}\leftarrow \mathcal{K};\hfill \\ b^{\prime }\leftarrow {\mathcal{A}}^{\mathbf{Decaps}(·)}(pk,c^{*},K_b^{*})\hfill \end{array}\right]-{\displaystyle \frac{1}{2}}\right|.$$

3.4. Compression and Decompression Functions

We utilize Kyber’s compression and decompression functions to decrease the ciphertext size without impacting decryption. Let q be a fixed prime, and let $x\in {\mathbb{Z}}_q$ and an integer t such that $t<\lfloor logq\rfloor$. The compression and decompression functions are denoted as

$$\begin{array}{c}\hfill {\mathrm{Compress}}_q(x,t)=⌊{\displaystyle \frac{2^t}{q}}·x⌉ mod 2^t,{\mathrm{Decompress}}_q(x,t)=⌊{\displaystyle \frac{q}{2^t}}·x⌉.\end{array}$$

For $x\in R_q$ and $\mathbf{x}\in R_q^d$, the functions are applied component-wise to each entry. The following lemma is essential for calculating the decryption failure rate.

Lemma 1.

For fixed $x\in {\mathbb{Z}}_q$, we have $x^{\prime }=Decompres{s}_q(Compres{s}_q(x,t),t)$, an element close to x; more precisely:

$$|x^{\prime }-xmod{}^{\pm }q|\le B_q:=⌊{\displaystyle \frac{q}{2^{t+1}}}⌉.$$

Definition 1

(Mod switching errors distribution, MSED). For fixed $t,d\in {\mathbb{Z}}^{+}$, we define the distribution ${\psi }_t^d$ by uniformly sampling $\mathbf{x}\leftarrow R^d$ and then returning

$$(\mathbf{x}-Decompres{s}_q(Compres{s}_q(\mathbf{x},t),t))mod{}^{\pm }q.$$

4. M-LWE-Based IND-CPA Security PKE

In this section, we construct an IND-CPA security PKE based on the hardness of M-LWE problem. Our construction is inspired by the work of Ducas and Durmus [36], which proposed a PKE scheme instantiated over general cyclotomic rings under the R-LWE assumption. While their work established an important theoretical foundation, it primarily focused on the R-LWE setting and did not allow for flexible selection of the module rank, nor include an efficient practical instantiation. In this work, we extend their approach to the more general M-LWE setting and provide a complete security analysis, which is given in Appendix A. This generalization enables finer-grained parameter selection and allows the use of non-Gaussian sampling techniques. As a result, our PKE construction offers improved parameter flexibility and practical efficiency compared to the original R-LWE-based scheme.

4.1. Fyber’s PKE Scheme

For clarity, we consistently use n to denote the ring dimension (i.e., the degree of the cyclotomic polynomial), and m to denote the modulus parameter in the M-LWE instance throughout the paper.

For the security parameter $\lambda$, the system parameter params is generated as follows: an integer m is chosen such that $m=2^k{3}^{\mathcal{l}}$, where $k,l\in \mathbb{N}$. Thus $n=\phi \left(m\right)=2^k{3}^{\mathcal{l}-1}$. And modulus q is a prime congruent to 1 modulo n. Note that $R_q={\mathbb{Z}}_q\left[x\right]/\left({\Phi }_m\left(x\right)\right)$, where ${\Phi }_m\left(x\right)=x^n-x^{n/2}+1$. Here, polynomial ${\Theta }_m\left(x\right)=x^{m/2}+1$ and integer $m^{\prime }=m/2$, which are respectively defined by (2) and (1) (since m is an even). They also have small positive integers $t_u$ and $t_v$ that are smaller than $\lfloor {log}_{2}q\rfloor$. Then, for given positive integers d and $\eta$, params is given by $(n,q,d,\eta ,t_u,t_v,R_q,{\Theta }_m\left[x\right])$.

We use Kyber’s encoding/decoding algorithms to convert a n bit-string message $\mu =({\mu }_0,\dots ,{\mu }_{n-1})\in \mathcal{M}={\{0,1\}}^n$ to its polynomial form ${\mu }^{\prime }=\mathbf{Encode}\left(\mu \right)={\sum }_{i=0}^{n-1}{\mu }_i{X}^i\in R$ and vice versa via $\mathbf{Decode}\left({\mu }^{\prime }\right)=\mu$.

The centered binomial distribution (CBD) ${\Delta }_{\eta }$ is defined as follows: randomly choosing samples ${(x_i,y_i)}_{i=1}^{\eta }$ from ${\{0,1\}}^{\eta }$, and outputting ${\sum }_{i=1}^{\eta }(x_i-y_i)$. The convolution of the centered binomial and uniform distribution is defined as follows: randomly choosing samples $X_1$ from ${\Delta }_{\eta }$, with $X_2$ from uniform distibution $\mathcal{S}$ over $\left[-0.5,0.5\right)\cap \mathbb{Q}$, outputting random variety $Y=X_1+X_2$. The probability of $Y_i$ is equal to ${\sum }_{Y_i=X_1+X_2}P\left(X_1\right)\times P\left(X_2\right)$. Subsequently, we abbreviate it as BUD, denoted by ${\Psi }_{\eta }$. We will discuss the Rényi divergence between this distribution and the Gaussian distribution in detail in Section 4.2. Our PKE construction is as follows:

• KeyGen($1^n$).

  1.

  Generate seed $\rho \in {\{0,1\}}^n$.

  2.

  Generate matrix $\mathbf{A}\in R_q^{d\times d}$ uniformly by seed $\rho$.

  3.

  Sample $\mathbf{s},{\mathbf{e}}_1\in {(\mathbb{Q}\left[X\right]/\left({\Theta }_m\right))}^d$ from the distribution ${\Psi }_{{\eta }_1}^{m^{\prime }\times d}$, respectively.

  4.

  Calculate $\widehat{\mathbf{s}}=⌊\mathbf{s}mod{\Phi }_m⌉\in R_q^d$; ${\widehat{\mathbf{e}}}_1=⌊{\mathbf{e}}_{1}mod{\Phi }_m⌉\in R_q^d$.

  5.

  $pk=(\mathbf{A},\mathbf{t}=\mathbf{A}\widehat{\mathbf{s}}+{\widehat{\mathbf{e}}}_{1}mod{R}_q^d);sk=\widehat{\mathbf{s}}$.

• Enc($pk,\mu$).

  1.

  ${\mu }^{\prime }\leftarrow {\mathbf{Encode}}_1(\mu ,m)$;

  2.

  Sample $\mathbf{r}$ from ${\Psi }_{{\eta }_1}^{m^{\prime }\times d}$, ${\mathbf{e}}_2$ from ${\Psi }_{{\eta }_2}^{m^{\prime }\times d}$, and $e_3$ from ${\Psi }_{{\eta }_2}^{m^{\prime }}$.

  3.

  Calculate $\widehat{\mathbf{r}}=⌊\mathbf{r}mod{\Phi }_m⌉\in R_q^d$; ${\widehat{\mathbf{e}}}_2=⌊{\mathbf{e}}_{2}mod{\Phi }_m⌉\in R_q^d$; ${\widehat{e}}_3=⌊e_{3}mod{\Phi }_m⌉\in R_q$.

  4.

  Compute $\mathbf{u}={\mathbf{A}}^T\widehat{\mathbf{r}}+{\widehat{\mathbf{e}}}_2\in R_q^d,v={\mathbf{t}}^T\widehat{\mathbf{r}}+{\widehat{e}}_3+{\mathrm{Decompress}}_q({\mu }^{\prime },1)\in R_q$.

  5.

  Output $c=({\mathbf{c}}_1\parallel c_2)=({\mathrm{Compress}}_q(\mathbf{u},t_u),{\mathrm{Compress}}_q(v,t_v))\in R_{t_u}^d\times R_{t_v}$

• Dec($sk,c$).

  1.

  Compute ${\mathbf{u}}^{\prime }={\mathrm{Decompress}}_q({\mathbf{c}}_1,t_u)$ and $v^{\prime }={\mathrm{Decompress}}_q(c_2,t_v)$.

  2.

  Compute ${\mu }^{\prime \prime }={\mathrm{Compress}}_q(v^{\prime }-{\widehat{\mathbf{s}}}^T{\mathbf{u}}^{\prime },1)$.

  3.

  Output $\mu \leftarrow {\mathbf{Decode}}_1({\mu }^{\prime \prime },m)$.

4.2. On Non-Gaussian Errors

The hardness of the M-LWE problem is typically analyzed via worst-case to average-case reductions formulated for continuous Gaussian error distributions over the rationals [8]. In contrast, practical schemes such as Kyber and Newhope [10] employ efficient non-Gaussian sampling, which comprises notably centered binomial distributions. While such noise is implementation-friendly, its tail bounds and overall statistical profile can be less robust in the M-LWE setting with non-power-of-two dimensions [36]. To achieve a more stable security–efficiency trade-off without incurring the cost of high-precision Gaussian sampling, we adopt a convolution of a centered binomial and a uniform distribution (BUD). This choice improves the statistical behavior and tunability of the noise distribution while introducing negligible implementation overhead.

Noise Distribution

In our PKE scheme, the secret and error vectors are sampled from a rational-valued distribution ${\Psi }_{\eta }$ obtained by combining a centered binomial distribution with a uniform distribution. Figure 1a illustrates the resulting distribution for $\eta =1$. Specifically, any rational number $\alpha \in \mathbb{Q}$ can be uniquely written as $\alpha =z+\omega$, where $z\in \mathbb{Z}$ and $\omega \in [-1/2,1/2)\cap \mathbb{Q}$. We sample z from a centered binomial distribution and $\omega$ uniformly from $[-1/2,1/2)\cap \mathbb{Q}$; the resulting value $\alpha$ then follows the distribution ${\Psi }_{\eta }$.

Figure 1. Distributions of secret coefficients after reduction and rounding.

We fix $m=2^k{3}^{\mathcal{l}}$ and let ${\Theta }_m\left(x\right)$ be a polynomial of degree $m^{\prime }=2^{k-1}{3}^{\mathcal{l}}$. The cyclotomic polynomial ${\Phi }_m\left(x\right)={\Phi }_{3n}\left(x\right)=x^n-x^{n/2}+1$ has degree $n=2^k{3}^{\mathcal{l}-1}$. After sampling $s\leftarrow {\Psi }_{\eta }$, the derivation of the secret vector $\widehat{s}$ proceeds in two steps.

In the first step, since $m^{\prime }=3n/2$, for any $s={\sum }_{i=0}^{m^{\prime }-1}{s}_i{x}^i\in \mathbb{Q}\left[x\right]/\left({\Theta }_m\left(x\right)\right)$ satisfying $|s_i|\le \eta +1/2$, reducing s modulo ${\Phi }_m\left(x\right)$ yields coefficients

$$s_i^{\prime }=\left\{\begin{array}{cc}{s}_i-s_{n+i},\hfill & 0\le i\le {\displaystyle \frac{n}{2}}-1,\hfill \\ s_i+s_{n+i},\hfill & {\displaystyle \frac{n}{2}}\le i\le n-1.\hfill \end{array}\right.$$

Consequently, for all $0\le i<n$, we have $|s_i^{\prime }|\le 2\eta +1$. Figure 1b illustrates the resulting distribution, corresponding to the additive convolution of two BUD samples. In the second step, the coefficients are rounded after the modulo operation, resulting in the distribution shown in Figure 1c, which we refer to as MR-BUD.

In works such as [9,10], using a Rényi divergence-based analysis, the authors show that replacing the n-dimensional discrete Gaussian distribution over $\mathbb{Z}$ with standard deviation $\sqrt{\eta /2}$ by a centered binomial distribution ${\Delta }_{\eta }$ only marginally affects security. This observation relies on the fact that, for LWE-based encryption schemes, the security is robust under small statistical deviations from the ideal Gaussian noise, rather than being tied to an exact noise distribution [10]. The following theorem shows that, compared to Gaussian noise, using MR-BUD as the error distribution in our PKE scheme does not incur a significant loss of security.

Theorem 1.

Let $\sigma >0$ be a real integer, and η be a small integer such that $\sigma =\sqrt{\eta /2}$. Denoted ${\mathcal{GD}}_{\sigma }$ is the rounded Gaussian distribution. Let $\mathcal{N}$ be the M-LWE sample under a standard definition; the noise sampling is taken from ${\mathcal{GD}}_{\sigma }$. If there exists an algorithm that succeeds in recovering the secret error when given an M-LWE instance of our PKE as input with probability $\mathcal{P}$, then it would also succeed against $\mathcal{N}$ with probability $\mathcal{Q}>{\mathcal{P}}^6/4$.

Proof. 

For two probability distributions $P,Q$, the Rényi divergence of $\alpha >1$ is defined as [37,38]:

$$\begin{array}{c}\hfill R_{\alpha }(P\parallel Q)={\left(\sum _{x\in Supp\left(P\right)}{\displaystyle \frac{P{\left(x\right)}^{\alpha }}{Q{\left(x\right)}^{\alpha -1}}}\right)}^{{\displaystyle \frac{1}{\alpha -1}}}.\end{array}$$

It bounds $Q\left(E\right)$ via:

$$Q\left(E\right)\ge P{\left(E\right)}^{\alpha /(\alpha -1)}/R_{\alpha }(P\parallel Q)$$

(3)

for the same event E. In our PKE, under the random oracle model, any attacker can successfully and accurately recover the plaintext message $\mu$ by recovering the secret vector $\widehat{\mathbf{s}}$ in the M-LWE samples. By setting the value of $\eta$, we ensure that the secret vector $\widehat{\mathbf{s}}$ and noise vector $\widehat{\mathbf{e}}$ in our PKE follow the MR-BUD, i.e., the distribution after rounding by the BUD module operation. We define the event E be the attacker successfully recover the secret vector. We also denoted ${\mathcal{GD}}_{\sigma }$ as the rounded Gaussian distribution of standard deviation $\sigma =\sqrt{\eta /2}$; that is, the distribution of $\lfloor \sqrt{\eta /2}·x\rceil$ where x follows the standard normal distribution. We give the parameter suggestions in Table 2, where the largest of $m=n·d=216\times 4=864$ and $\eta =2$; we can obtain that for fixed real $\alpha =1.2$ we have

$$\begin{array}{c}\hfill R_{\alpha }({\Psi }_{\eta }\parallel {\mathcal{GD}}_{\sigma })\approx 1.00028\end{array}$$

by script Renyi.py. Since $5m=4320$ samples are used per instance of our PKE, we used $R_{\alpha }(P^n\parallel Q^n)=R_{\alpha }{(P\parallel Q)}^n$ have $R_{\alpha }{({\Psi }_{\eta }\parallel {\mathcal{GD}}_{\sigma })}^{4320}<4$. Thus, by (3), we have

$$\begin{array}{c}\hfill \mathcal{Q}>{\mathcal{P}}^{\alpha /(\alpha -1)}/R_{\alpha }{({\Psi }_{\eta }\parallel {\mathcal{GD}}_{\sigma })}^{5m}>{\mathcal{P}}^6/4.\end{array}$$

   □

4.3. Correctness and Security

Below, we explain that, to correctly recover all message bits, the $l_{\infty }$-norm of error polynomial $e={\widehat{\mathbf{e}}}_1^T\widehat{\mathbf{r}}-{\widehat{\mathbf{s}}}^T{\widehat{\mathbf{e}}}_2+{\widehat{e}}_3-{\widehat{\mathbf{s}}}^T{\mathbf{c}}_u+c_v\in R_q$ must be smaller than $q/4$. We can make the decryption error negligible by controlling the parameters, as discussed in Section 5. We have the following theorem.

Theorem 2

(Correctness). Let

$$\begin{array}{c}\hfill \epsilon =Pr\{\parallel {\widehat{\mathit{e}}}_1^T\widehat{\mathit{r}}-{\widehat{\mathit{s}}}^T{\widehat{\mathit{e}}}_2+{\widehat{e}}_3-{\widehat{\mathit{s}}}^T{\mathit{c}}_u+c_v{\parallel }_{\infty }\ge \lfloor q/4\rceil \},\end{array}$$

then our PKE scheme is $(1-\epsilon )$-correct.

Proof. 

Based on the hardness of M-LWE, we have ${\mathbf{u}}^{\prime },v^{\prime }$ are pseudo-random. By the Enc algorithm, we have

$$\begin{array}{cc}\hfill {\mathbf{u}}^{\prime }& =Decompres{s}_q(Compres{s}_q(\mathbf{u},t_u),t_u)=\mathbf{u}+{\mathbf{c}}_u,\hfill \\ \hfill v^{\prime }& =Decompres{s}_q(Compres{s}_q(v,t_u),t_u)=v+c_v,\hfill \end{array}$$

where ${\mathbf{c}}_u\leftarrow {\psi }_{t_u}^d,c_v\leftarrow {\psi }_{t_v}$ (see Definition 1). Thus, the value of $v^{\prime }-{\widehat{\mathbf{s}}}^T{\mathbf{u}}^{\prime }$ in step 2 of Dec is $v-{\widehat{\mathbf{s}}}^T\mathbf{u}-{\widehat{\mathbf{s}}}^T{\mathbf{c}}_u+c_v$, where $v,\mathbf{u}$ are from line 4 of Enc. For ${\mu }^{\prime }={\sum }_{i=0}^{m-1}{\mu }_i{X}^i$, by definition of ${\mathrm{Decompress}}_q$ we can write ${\mathrm{Decompress}}_q({\mu }^{\prime },1)=⌊q/2⌉·{\mu }^{\prime }$. It is clear that

$$\begin{array}{c}\hfill v^{\prime }-{\widehat{\mathbf{s}}}^T{\mathbf{u}}^{\prime }={\mathbf{t}}^T\widehat{\mathbf{r}}+{\widehat{e}}_3+⌊q/2⌉·{\mu }^{\prime }-{\widehat{\mathbf{s}}}^T({\mathbf{A}}^T\widehat{\mathbf{r}}+{\widehat{\mathbf{e}}}_2)-{\widehat{\mathbf{s}}}^T{\mathbf{c}}_u+c_v\end{array}$$

Then we have

$$\begin{array}{c}\hfill v^{\prime }-{\widehat{\mathbf{s}}}^T{\mathbf{u}}^{\prime }={\widehat{\mathbf{e}}}_1^T\widehat{\mathbf{r}}-{\widehat{\mathbf{s}}}^T{\widehat{\mathbf{e}}}_2+{\widehat{e}}_3-{\widehat{\mathbf{s}}}^T{\mathbf{c}}_u+c_v+⌊q/2⌉·{\mu }^{\prime }=e+⌊q/2⌉·{\mu }^{\prime }.\end{array}$$

The line of 2 of Dec means that decrypt the i-th coefficient of ${\mu }^{\prime \prime }$ is equal to 1 if the $v^{\prime }-{\widehat{\mathbf{s}}}^T{\mathbf{u}}^{\prime }$ is closer to ${\displaystyle \frac{q}{2}}$ and as 0 otherwise. It is clear that as long as ${\parallel e\parallel }_{\infty }<⌊q/4⌉,$ the decryption succeeds, which concludes the proof of the correctness.    □

This part we would give the security proof of our PKE using the hybrid argument. Our scheme is IND-CPA secure if the M-LWE assumption holds. Now for any adversary $\mathcal{A}$, we define

$$\begin{array}{cc}\hfill Ad{v}_{m,d,q,s}^{\mathrm{M}\text{-}\mathrm{LWE}}\left(\mathcal{A}\right)=|& Pr[b^{\prime }=1:\mathbf{A}\leftarrow R_q^{d\times d};(\mathbf{s},\mathbf{e})\leftarrow {\Psi }_{\eta }^d\times {\Psi }_{\eta }^d;\mathbf{b}=\mathbf{As}+\mathbf{e};b^{\prime }\leftarrow \mathcal{A}\left(\mathbf{A},\mathbf{b}\right)]\hfill \\ \hfill & -Pr[b^{\prime }=1:\mathbf{A}\leftarrow R_q^{d\times d};\mathbf{b}\in R_q^d;b^{\prime }\leftarrow \mathcal{A}\left(\mathbf{A},\mathbf{b}\right)]|.\hfill \end{array}$$

Theorem 3

(IND-CPA security). Our PKE is IND-CPA secure, provided that the M-LWE problem is hard. More precisely, if $\mathcal{B},\mathcal{C}$ are two distinguishers for M-LWE, then

$$\begin{array}{c}\hfill Ad{v}_{\mathit{PKE}}^{\mathit{IND}\text{-}\mathit{CPA}}\left(\mathcal{A}\right)\le Ad{v}_{m,d,q,s}^{M\text{-}\mathit{LWE}}\left(\mathcal{B}\right)+2Ad{v}_{m,d,q,s}^{M\text{-}\mathit{LWE}}\left(\mathcal{C}\right)\end{array}$$

where $\mathcal{A}$ is an adversary for the PKE.

Proof. 

Now, we prove the theorem under the M-LWE assumption. To argue that an adversary cannot distinguish between the two cases, we rely on the hardness of the decision M-LWE problem and adopt a standard game-based proof framework. Specifically, we consider a sequence of security games, where each game represents an interaction-based security experiment between a challenger and an adversary $\mathcal{A}$. For $i=0,1,2,3$, let $S_i$ denote the event that $\mathcal{A}$ correctly guesses the challenge bit in Game i, and let ${\mathcal{D}}_i$ denote the distribution induced by Game i.

Game 0. Game 0 is the real game in which the public key is properly generated and the ciphertext is an encryption of ${\mu }_0$. It follows that

$${\mathcal{D}}_0=\{pk=\left(\mathbf{A},\mathbf{t}=\mathbf{A}\widehat{\mathbf{s}}+{\widehat{\mathbf{e}}}_1\right),c=\left({\mathbf{A}}^T\widehat{\mathbf{r}}+{\widehat{\mathbf{e}}}_2,{\mathbf{t}}^T\widehat{\mathbf{r}}+{\widehat{e}}_3+{\mu }_0⌊q/2⌉\right)\}.$$

Game 1. In this game, we replace the public key $\mathbf{t}$ by a uniformly random element in $R_q^d$. It is given that

$${\mathcal{D}}_1=\{pk=\left(\mathbf{A},\mathbf{t}\stackrel{\$}{\leftarrow }{R}_q^d\right),c=\left({\mathbf{A}}^T\widehat{\mathbf{r}}+{\widehat{\mathbf{e}}}_2,{\mathbf{t}}^T\widehat{\mathbf{r}}+{\widehat{e}}_3+{\mu }_0⌊q/2⌉\right)\}.$$

Thus, there exists a distinguisher for M-LWE such that $|Pr\left[S_0\right]-Pr\left[S_1\right]|\le Ad{v}_{m,d,q,s}^{\mathrm{M}-\mathrm{LWE}}\left(\mathcal{B}\right).$

Game 2. In this game, ${\mathbf{A}}^T\widehat{\mathbf{r}}+{\widehat{\mathbf{e}}}_2$ is replaced with uniformly random polynomials in $R_q^d$, ${\mathbf{t}}^T\widehat{\mathbf{r}}+{\widehat{e}}_3$ is replaced with uniformly random polynomial in $R_q$. ${\mathcal{D}}_2$ is given as follows:

1.

Let $\mathbf{u}\stackrel{\$}{\leftarrow }{R}_q^d,v\stackrel{\$}{\leftarrow }{R}_q$.

2.

${\mathcal{D}}_2=\left\{pk=\left(\mathbf{A},\mathbf{t}\stackrel{\$}{\leftarrow }{R}_q^d\right),c=\left(\mathbf{u}\stackrel{\$}{\leftarrow }{R}_q^d,v\stackrel{\$}{\leftarrow }{R}_q,v+{\mu }_0⌊q/2⌉\right)\right\}.$

It follows that $|Pr\left[S_1\right]-Pr\left[S_2\right]|\le 2Ad{v}_{m,d,q,s}^{\mathrm{M}-\mathrm{LWE}}\left(\mathcal{C}\right)$, where $\mathcal{C}$ is a distinguisher for M-LWE.

Game 3. In this game, message ${\mu }_0$ is replaced with ${\mu }_1\in \mathcal{M}$. We have

$$\begin{array}{cc}\hfill {\mathcal{D}}_3=\{& pk=\left(\mathbf{A},\mathbf{t}\stackrel{\$}{\leftarrow }{R}_q^d\right),c=\left(\mathbf{u}\stackrel{\$}{\leftarrow }{R}_q^d,v\stackrel{\$}{\leftarrow }{R}_q,v+{\mu }_1⌊q/2⌉\right)\}.\hfill \end{array}$$

Obviously, $Pr\left[S_2\right]=Pr\left[S_3\right].$

In conclusion, collecting the probabilities yields the required bound.    □

4.4. The CCA-Secure KEM

In this subsection, we construct a CCA-secure KEM Fyber from the proposed PKE scheme using the Fujisaki–Okamoto (FO) transformation [39]. Let $\mathbf{Fyber}.\mathbf{PKE}=$$(\mathbf{KeyGen},\mathbf{Enc},\mathbf{Dec})$ be the IND-CPA secure PKE scheme defined in the previous section. The resulting KEM is denoted by $\mathbf{Fyber}.\mathbf{KEM}=$$(\mathbf{KeyGen},\mathbf{Encaps},\mathbf{Decaps})$.

We describe the KEM algorithms in detail below. Let $\mathcal{K}$ denote the key space and let ${\mathcal{H}}_1:{\{0,1\}}^{*}\to {\{0,1\}}^{2\times n}$, ${\mathcal{H}}_2:{\{0,1\}}^{*}\to {\{0,1\}}^n$ be hash functions modeled as random oracles. The key generation algorithm $\mathbf{KeyGen}\left(1^n\right)$ runs the $\mathbf{KeyGen}$ of PKE to obtain $(pk,sk)$ and outputs the same key pair.

• $\mathbf{Encaps}\left(pk\right)$

  1.

  Sample a random message $m\stackrel{\$}{\leftarrow }\mathcal{M}$.

  2.

  Compute $(\widehat{K},r)={\mathcal{H}}_1(m,{\mathcal{H}}_2\left(pk\right))$.

  3.

  Compute the ciphertext $c\leftarrow \mathbf{Fyber}.\mathbf{Enc}(pk,m;r)$.

  4.

  Compute the shared key $K={\mathcal{H}}_2(\widehat{K},{\mathcal{H}}_2\left(c\right))$.

  5.

  Output $(c,K)$.

• $\mathbf{Decaps}(sk,c)$

  1.

  Recover $m^{\prime }\leftarrow \mathbf{Fyber}.\mathbf{Dec}(sk,c)$.

  2.

  If $m^{\prime }=\perp$, output ⊥.

  3.

  Recompute $({\widehat{K}}^{\prime },r^{\prime })={\mathcal{H}}_1(m^{\prime },{\mathcal{H}}_2\left(pk\right))$ and $c^{\prime }\leftarrow \mathbf{Fyber}.\mathbf{Enc}(pk,m^{\prime };r^{\prime })$.

  4.

  If $c\ne c^{\prime }$, output ⊥.

  5.

  Otherwise, output $K^{\prime }={\mathcal{H}}_2({\widehat{K}}^{\prime },{\mathcal{H}}_2\left(c\right))$.

The correctness of the KEM follows directly from the correctness of the underlying PKE scheme. For honestly generated keys and ciphertexts, the decrypted message $m^{\prime }$ equals the encapsulated message m, and the recomputed ciphertext $c^{\prime }$ matches the original ciphertext c. Consequently, both parties derive the same shared key K except with negligible probability.

The IND-CCA security analysis of the proposed KEM Fyber follows the standard FO transformation and is identical to that of [9], given that the underlying PKE is IND-CPA secure. We present the following theorem and omit the proof.

Theorem 4

(CCA-secure KEM). Assuming that the underlying PKE scheme is IND-CPA secure and that the hash functions ${\mathcal{H}}_1$ and ${\mathcal{H}}_2$ are modeled as random oracles, Flexkyber.KEM is IND-CCA secure.

5. Implementation

In this section, we conduct an analysis of the challenges encountered during the implementation process, provide corresponding solutions, and present a summary of the experimental data in Table 2. Our scheme is implemented in C, with security level and failure rate estimation performed using Python 3.11.9. The computational environment consists of an AMD Ryzen 7 4800H CPU @2.9 GHz and 16 GB RAM, running Ubuntu 22.04 LTS.

Table 2. Parameter selection and comparison with Kyber.

	Kyber			Fyber
$\mathbf{\lambda }$	Kyber-512	Kyber-768	Kyber-1024	Fyber-576	Fyber-768	Fyber-864
${\Phi }_m\left[x\right]$	$x^n+1$			$x^n-x^{n/2}+1$
$n=\phi \left(m\right)$	256	256	256	192	192	216
d	2	3	4	3	4	4
q	3329	3329	3329	3457	3457	3457
${\eta }_1$ *	3	3	2	$1\left(3\right)$	$1\left(3\right)$	$1\left(3\right)$
${\eta }_2$ *	3	2	2	$1\left(3\right)$	$1\left(3\right)$	$1\left(3\right)$
$t_u$	10	10	11	10	11	11
$t_v$	4	4	5	4	5	6
$\left|pk\right|$ (bytes)	800	1184	1568	870	1152	1297
$\left|c\right|$ (bytes)	768	1088	1568	816	1176	1350
KeyGen (cycles)	74,442	119,479	189,659	239,655	401,098	390,252
Enc (cycles)	95,438	146,652	228,258	246,557	400,837	395,733
Dec (cycles)	33,726	43,673	56,085	53,214	69,135	67,453
Decryption failure rate $\epsilon$	$2^{-139}$	$2^{-165}$	$2^{-175}$	$2^{-154}$	$2^{-163}$	$2^{-154}$
Security level (Quantum)	$118\left(107\right)$	$181\left(164\right)$	$253\left(230\right)$	$136\left(123\right)$	$192\left(174\right)$	$220\left(200\right)$

* The values of ${\eta }_1$ and ${\eta }_2$ are represented as $1\left(3\right)$, which means when the value is 1, the actual coefficient distribution is $\{-3,\dots ,3\}$. This notation arises from our hybrid sampling that combines the centered binomial distribution with the uniform distribution. Discussion in Section 4.2.

Table 2. Parameter selection and comparison with Kyber.

	Kyber			Fyber
$\mathbf{\lambda }$	Kyber-512	Kyber-768	Kyber-1024	Fyber-576	Fyber-768	Fyber-864
${\Phi }_m\left[x\right]$	$x^n+1$			$x^n-x^{n/2}+1$
$n=\phi \left(m\right)$	256	256	256	192	192	216
d	2	3	4	3	4	4
q	3329	3329	3329	3457	3457	3457
${\eta }_1$ *	3	3	2	$1\left(3\right)$	$1\left(3\right)$	$1\left(3\right)$
${\eta }_2$ *	3	2	2	$1\left(3\right)$	$1\left(3\right)$	$1\left(3\right)$
$t_u$	10	10	11	10	11	11
$t_v$	4	4	5	4	5	6
$\left|pk\right|$ (bytes)	800	1184	1568	870	1152	1297
$\left|c\right|$ (bytes)	768	1088	1568	816	1176	1350
KeyGen (cycles)	74,442	119,479	189,659	239,655	401,098	390,252
Enc (cycles)	95,438	146,652	228,258	246,557	400,837	395,733
Dec (cycles)	33,726	43,673	56,085	53,214	69,135	67,453
Decryption failure rate $\epsilon$	$2^{-139}$	$2^{-165}$	$2^{-175}$	$2^{-154}$	$2^{-163}$	$2^{-154}$
Security level (Quantum)	$118\left(107\right)$	$181\left(164\right)$	$253\left(230\right)$	$136\left(123\right)$	$192\left(174\right)$	$220\left(200\right)$

* The values of ${\eta }_1$ and ${\eta }_2$ are represented as $1\left(3\right)$, which means when the value is 1, the actual coefficient distribution is $\{-3,\dots ,3\}$. This notation arises from our hybrid sampling that combines the centered binomial distribution with the uniform distribution. Discussion in Section 4.2.

5.1. Polynomial Multiplication

In lattice-based encryption schemes, polynomial multiplication constitutes the dominant component of the computational cost. While our construction employs non-power-of-two ring dimensions to enable flexible parameter selection, such choices typically preclude the direct use of standard NTT-friendly polynomial rings and may introduce additional computational overhead.

To mitigate this effect and improve practical performance, especially in resource-constrained IoT environments, we adopt an efficient polynomial multiplication strategy inspired by existing implementation-oriented optimizations [40]. The core idea is to embed polynomial multiplication over $R_q={\mathbb{Z}}_q\left[x\right]/(x^n-x^{n/2}+1)$ into a larger convolution-friendly ring ${\mathbb{Z}}_{q^{\prime }}\left[x\right]/(x^N-1)$, where $N\ge 2n$ is chosen as a power of two to enable the use of the Number-Theoretic Transform (NTT).

Specifically, given two polynomials $a,b\in R_q$ with coefficient bounds $|a_i|\le q/2$ and $|b_i|\le 2\eta +1$, we select a modulus $q^{\prime }$ large enough to avoid modular overflow during intermediate computations, i.e., $q^{\prime }>(2\eta +1)nq/2$, and that satisfies $q^{\prime }\equiv 1(modN)$ to support NTT operations. The polynomials are zero-padded to length N, multiplied using NTT over ${\mathbb{Z}}_{q^{\prime }}$, and subsequently reduced modulo ${\Phi }_m\left(x\right)$ and q to obtain the final product in $R_q$. Algorithm 1 summarizes this procedure.

5.2. Public Key and Ciphertext Sizes

For fixed parameters $(n,d,q)$, the public key produced by the PKE scheme is of the form $(\mathbf{A},\mathbf{t})$. Here, $\mathbf{A}={\left(a_{ij}\right)}_{i,j\in \left[d\right]}\in R_q^{d\times d}$ is a public matrix whose entries are polynomials over ${\mathbb{Z}}_q$. Each polynomial $a_{ij}$ is deterministically generated from an n-bit seed $\rho$. The vector $\mathbf{t}\in R_q^d$ consists of d polynomials, each having n coefficients represented using $\lceil {log}_{2}q\rceil$ bits. Consequently, the size of the public key is

$${\displaystyle \frac{n+d·n·\lceil {log}_{2}q\rceil }{8}}\mathrm{bytes}.$$

The ciphertext is of the form

$$c=({\mathbf{c}}_1\parallel c_2):=\left({\mathsf{Compress}}_q(\mathbf{u},t_u),{\mathsf{Compress}}_q(v,t_v)\right)\in R_{t_u}^d\times R_{t_v},$$

where $\mathbf{u}={\mathbf{A}}^T\widehat{\mathbf{r}}+{\widehat{\mathbf{e}}}_2\in R_q^d$ contains d polynomials with n coefficients each, and $v={\mathbf{t}}^T\widehat{\mathbf{r}}+{\widehat{e}}_3+\mu ·\lfloor q/2\rceil \in R_q$ is a single polynomial with n coefficients. Each coefficient of $\mathbf{u}$ and v is represented using $\lceil {log}_{2}q\rceil$ bits prior to compression. After applying the compression function, the total size of the ciphertext is

$${\displaystyle \frac{d·n·t_u+n·t_v}{8}}\mathrm{bytes}.$$

Algorithm 1: Polynomial multiplication

5.3. Decryption Failure Rate of Fyber

We outline the procedure used to evaluate the decryption failure rate. The coefficients of $\widehat{\mathbf{s}},\widehat{\mathbf{r}},{\widehat{\mathbf{e}}}_1,{\widehat{\mathbf{e}}}_2$, and ${\widehat{e}}_3$ follow the MR-BUD distribution, with each coefficient supported on the interval $[-2\eta -1,2\eta +1]$, as illustrated in Figure 1. In addition, by Lemma 1, each coefficient of ${\mathbf{c}}_u$ and $c_v$ lies in the interval $[-B_q,B_q]$. By choosing $2^{t+1}$ close to q, the value of $B_q$ can be made sufficiently small. For example, when $t=10$ and $q=3457$, we obtain $B_q=1$.

Given the resulting distributions and the concrete parameter choices for $\eta$, t, and q, the decryption failure rate is evaluated using the script PKE-failure.py, and the corresponding results are reported in Table 2. Furthermore, following the analysis in Theorem 2, the probability distribution of each error-term coefficient can be explicitly derived, enabling an estimation of the concrete failure probability of the scheme. Since these coefficients are discrete and bounded by small values, their distributions can be computed efficiently. Overall, the resulting decryption failure rate, denoted by $\epsilon$, is negligible for all parameter sets considered.

5.4. Comparison to Kyber

In this part, we focus on a comparison with the Kyber. The outcomes are displayed in Table 2. The Fyber-576/768/864 parameter sets are chosen to approximately match NIST security categories 1/3/5 under BKZ-based concrete security estimation. Specifically, the parameters $(n,d,q)$ are selected such that the estimated cost of the best-known lattice attacks achieves security levels comparable to Kyber-512/768/1024, while enabling finer-grained control of n through non-power-of-two dimensions. We calculate the time complexity of the BKZ algorithm for breaking the scheme to ensure concrete security. The estimation model used for the BKZ algorithm is based on Kyber [9]. When considering the complexity of the SVP oracle (which is called by the BKZ algorithm), we use $2^{0.292b(1+o(1\left)\right)}$ for classic complexity and $2^{0.265b(1+o(1\left)\right)}$ for quantum complexity [9], where b represents the block size in the BKZ algorithm. This estimator assumes the standard geometric series assumption (GSA) and the best-known sieving-based SVP algorithms in both classical and quantum models. Although this model does not necessarily reflect the latest improvements in lattice reduction heuristics, it is widely adopted in the literature and provides a consistent baseline for comparing concrete security levels with Kyber.

The choice of module polynomials in Kyber is ${\Phi }_m\left(x\right)=x^n+1$, where n is a power of 2. However, our PKE selects ${\Phi }_m\left(x\right)=x^n-x^{n/2}+1$, where $n=2^k{3}^{\mathcal{l}}$ and k and l are positive integers. This decision is based on the fact that the density of chosen values for n in our scheme is higher than in Kyber’s scheme. Additionally, m also takes the form of a power of 2 multiplied by a power of 3. The sparsity of the ${\Phi }_m$ form is moderate compared to other more general m. Secondly, we provide three sets of parameter selection suggestions: Fyber-576, 768, and 864, corresponding to Kyber’s three parameter sets: Kyber-512, 768, and 1024, respectively.

Figure 2a,b illustrate the public key and ciphertext sizes of Fyber and Kyber under three representative parameter sets, respectively. From the horizontal axis, it can be observed that the coarse granularity between adjacent security levels in Kyber is effectively bridged by our construction. Furthermore, our KEM is well-suited for IoT environments, as it enables more flexible choices of the degree n to accommodate diverse application scenarios, thereby allowing a finer-grained stratification of security levels.

Figure 2. Comparison of communication overhead.

When comparing Fyber-768 with Kyber-768 under the same product $n·d$, our scheme achieves a $2.7\%$ reduction in public key size. Although the ciphertext size increases slightly by approximately $2\%$; this overhead is incurred while providing a higher concrete security level. This improvement is mainly attributed to the flexibility of our scheme in selecting a smaller n. Furthermore, we compare Kyber-1024 with Fyber-864, as Kyber-1024 targets the 192-bit quantum security level. At this security level, our scheme achieves a $17.3\%$ reduction in public key size and a $13.9\%$ reduction in ciphertext size. This advantage stems from the ability of our parameterization to approach the 192-bit security level more flexibly through a reduced value of n. Overall, our results demonstrate that Fyber effectively fills the security-level gap around the 192-bit setting in Kyber.

5.5. Computational Overhead

Figure 3 reports the median and average cycle counts measured over 1000 executions of key generation, encryption, and decryption for Fyber and Kyber. Compared to Kyber, Fyber incurs a moderate increase in computational cost: encryption requires approximately 73–$173\%$ more cycles, while decryption requires 20–$58\%$ more cycles. This overhead mainly stems from supporting non-power-of-two ring dimensions, which enable finer-grained parameter tuning. Compared with Kyber, Fyber reduces communication overhead at the cost of moderately increased computation, while requiring a comparable implementation memory footprint, as both schemes use similar arithmetic building blocks. All comparisons with Kyber are conducted under comparable NIST security levels.

Figure 3. Comparison of the running time of KeyGen, Enc and Dec.

Despite this increase, the resulting performance remains practical for IoT-oriented deployments. In IoT and low-power communication settings, bandwidth is often a more critical constraint than computation. Traditional KEMs based on classical number-theoretic assumptions typically exhibit high computational latency but small communication overhead, whereas lattice-based KEMs such as Kyber prioritize high arithmetic efficiency at the cost of larger ciphertexts. Fyber deliberately moves toward the communication efficiency of traditional KEMs by trading a modest amount of computational efficiency for reduced ciphertext size and improved adaptability.

In contrast to Kyber’s coarse-grained parameterization, Fyber supports fine-grained security-level selection through flexible ring dimensions, allowing ciphertext sizes to be tailored to specific security requirements. Overall, Fyber strikes a balanced trade-off between computation and bandwidth, making it well-suited for resource-constrained IoT KEM deployments.

6. Conclusions

In this paper, we proposed a post-quantum key-encapsulation mechanism (KEM) based on the M-LWE assumption over a more flexible class of cyclotomic rings. In contrast to the NIST-standardized KEM Kyber, our construction is instantiated over cyclotomic trinomials of the form $x^n-x^{n/2}+1$, where $n=2^k·3^{\mathcal{l}}$ for positive integers k and ℓ. While Kyber adopts polynomial rings of the form ${\mathbb{Z}}_q\left[x\right]/(x^n+1)$, which enable highly efficient implementations, this choice restricts parameter selection to power-of-two dimensions and results in coarse-grained security levels. As a consequence, achieving intermediate security requirements (e.g., around the 192-bit level) may incur unnecessary bandwidth overhead.

By leveraging more flexible ring dimensions, our scheme enables finer-grained security-level stratification and achieves improved trade-offs between security and bandwidth consumption. In particular, the proposed parameter sets fill the gap between existing standardized security levels while maintaining a low decryption failure rate. To address the technical challenges arising from general cyclotomic rings, we further design a fast polynomial multiplication algorithm and an efficient noise sampling method tailored to the proposed ring structure. Experimental results indicate that the resulting performance remains competitive with Kyber, while offering enhanced parameter adaptability for resource-constrained IoT deployments.

Overall, our work demonstrates that extending M-LWE-based KEMs beyond power-of-two cyclotomic rings is a viable and practical approach for achieving flexible security levels and efficient post-quantum key establishment in heterogeneous IoT environments.