An Improved Attack on the RSA Variant Based on Cubic Pell Equation

Abstract

In this paper, we present a novel method to solve trivariate polynomial modular equations of the form $x(y^2+Ay+B)+z\equiv 0 (mod e).$ Our approach integrates Coppersmith’s method with lattice basis reduction to efficiently solve the former equation. Several variants of RSA are based on the cubic Pell equation $x^3+f{y}^3+f^2{z}^3-3fxyz\equiv 1 (mod N)$, where f is a cubic nonresidue modulus $N=pq$. In these variants, the public exponent e and the private exponent d satisfy $ed\equiv 1 (mod \psi (N\left)\right)$ with $\psi \left(N\right)=\left(p^2+p+1\right)\left(q^2+q+1\right)$. Moreover, d can be written in the form $d\equiv {\displaystyle \frac{v_0}{z_0}} (mod \psi \left(N\right))$ with any $z_0$ satisfying $gcd(z_0,\psi \left(N\right))=1$. In this paper, we apply our method to attack the variants when $d\equiv {\displaystyle \frac{v_0}{z_0}} (mod \psi \left(N\right))$ and when $|z_0|$ and $|v_0|$ are suitably small. We also show that our method significantly improves the bounds of the private exponents d of the previous attacks on the variants, particularly in the scenario of small private exponents and in the scenarios where partial information about the primes is available.

1. Introduction

In 1978, Rivest, Shamir, and Adleman [1] proposed a number theoretic system, called RSA, as the first public key cryptosystem. It is highly significant in terms of its practical applications. Its security is based on hard mathematical problems such as the integer factorization problem. In the RSA cryptosystem, two large prime numbers p and q of equal bit size are generated, and the RSA modulus $N=pq$ is then computed. Next, a public exponent e is selected to satisfy $gcd\left(e,\phi \left(N\right)\right)=1$, where $\phi \left(N\right)=(p-1)(q-1)$ is the Euler’s totient function. This enables computation of the private exponent d, which is the inverse of e modulo $\phi \left(N\right)$, that is, d satisfies the modular equation $ed\equiv 1 (mod \phi (N\left)\right)$. The equation $ed-k\phi \left(N\right)=1$ is well defined and called the key equation. The public key consists of the pair $(N,e)$, whereas $(N,d)$ is the private key. For a plaintext m such that $m<N$, the encryption is done by computing $c\equiv m^e (mod N)$. The decryption of the ciphertext c is performed by calculating $m\equiv c^d (mod N)$. It is well known that the encryption time is $O\left({(log\left(N\right))}^{2}log\left(e\right)\right)$, while the decryption time is $O\left({(log\left(N\right))}^{2}log\left(d\right)\right)$.

To enhance the performance of RSA, small private exponents are tempting to use. However, in 1990, Wiener [2] showed that one can break RSA for private exponents $d<{\displaystyle \frac{1}{3}}{N}^{{\displaystyle \frac{1}{4}}}$. This bound was later improved by Boneh and Durfee [3] up to $N^{0.292}$.

In 1996, Coppersmith [4] proposed a polynomial time algorithm to find small roots of modular polynomial equations of the form $f\left(x\right)\equiv 0 (mod M)$, where $f\left(x\right)\in \mathbb{Z}\left[x\right]$, and M is an integer of unknown factorization. As a consequence, Coppersmith showed that one can break RSA if the public exponent is 3 under some constraints. Since then, Coppersmith’s method has been generalized to polynomials with more variables.

In 1998, Boneh and Durfee [3] presented an attack on RSA by transforming the key equation $ed-k\phi \left(N\right)=1$ into a specific equation $x(-N+y)\equiv 1 (mod e)$. They used Coppersmith’s method to extract the root $(x,y)=(k,p+q-1)$ and then factored the modulus N in polynomial time. As a significant result, they improved Wiener’s bound up to $N^{0.292}$.

To enhance both security and efficiency in RSA, alternative schemes have been proposed. Some of these variants retain the same modulus $N=pq$ but modify the Euler totient function $\phi \left(N\right)$, replacing it with $\psi \left(N\right)=(p^2+p+1)(q^2+q+1)$m as in [5], or with ${\psi }^{\prime }\left(N\right)=(p^2-1)(q^2-1)$, as in [6,7,8,9].

Recently, Feng et al. [10] proposed a new method to solve the modular polynomial equation $f(x,y)\equiv 0 (mod e)$, where $f(x,y)=x(y^2+Ay+B)+1$. Their method is based on Coppersmith’s method and lattice-based reduction. They showed that $e=N^{\alpha }$, $\left|x\right|\le N^{\beta }$, $\left|y\right|\le N^{\gamma }$, and

$$\beta <\left\{\begin{array}{cccc}& \alpha -\sqrt{2\gamma \alpha },\hfill & & \mathrm{if}\beta +2\gamma <\alpha <\beta +3\gamma ,\hfill \\ & min\left({\displaystyle \frac{2}{3}}\alpha -{\displaystyle \frac{3}{2}}\gamma ,\alpha -3\gamma \right),\hfill & & \mathrm{if}\alpha \ge \beta +3\gamma .\hfill \end{array}\right.$$

Thus, one can find x and y in polynomial time. They used their method to break some RSA variants by performing a partial prime exposure attack. They proved that if $N=pq$ is an RSA modulus, $p_0$ is an approximation of p such that $|p-p_0|=N^{\gamma }$, and $e=N^{\alpha }$ satisfies the key equation $ed-k\psi \left(N\right)=1$ with $d<N^{\delta }$, then the factorization of N can be found under the conditions

$$\delta <\left\{\begin{array}{cccc}& 2-\sqrt{2\alpha \gamma },\hfill & & \mathrm{if}2\gamma <\alpha <{\displaystyle \frac{9}{2}}\gamma ,\hfill \\ & 2-{\displaystyle \frac{1}{3}}\alpha -{\displaystyle \frac{3}{2}}\gamma ,\hfill & & \mathrm{if}{\displaystyle \frac{9}{2}}\gamma \le \alpha <6-{\displaystyle \frac{9}{2}}\gamma .\hfill \end{array}\right.$$

Observe that a private exponent d can be written in the form $d\equiv {\displaystyle \frac{v_0}{z_0}} (mod \psi \left(N\right))$ with any $z_0$, satisfying $gcd(z_0,\psi \left(N\right))=1$ and $v_0\equiv d{z}_0 (mod \psi \left(N\right))$. From the modular equation $ed\equiv 1mod \psi \left(N\right)$, one gets $e{\displaystyle \frac{v_0}{z_0}}\equiv 1 (mod \psi \left(N\right))$, and consequently, $e{v}_0\equiv z_0 (mod \psi \left(N\right))$.

In this paper, we transform the modular equation $e{v}_0\equiv z_0 (mod \psi \left(N\right))$ into an equation of the form $x(y^2+Ay+B)+z\equiv 0 (mod e)$ and present a method to find its small solutions $(x,y,z)$. We then apply our method to attack the variants when $d\equiv {\displaystyle \frac{v_0}{z_0}} (mod \psi \left(N\right))$, and $|z_0|$ and $|v_0|$ are suitably small.

As a byproduct, we revisit the work of Feng et al. [10] by presenting a different approach for solving the generalized equation $x(y^2+Ay+B)+z\equiv 0 (mod e)$. We prove that if $e=N^{\alpha }$, $\left|x\right|\le N^{\beta }$, $\left|y\right|\le N^{\gamma }$, $z\le N^{\mu }$, $\alpha \ge 2\gamma$, and $\beta +{\displaystyle \frac{1}{2}}\mu <\alpha -\sqrt{2\alpha \gamma }$, one can find x, y, and z in polynomial time. This improves the bound of Feng et al. [10] for $\alpha \ge \beta +3\gamma$. Notice that their method becomes a particular case of our method if $\mu =0$ and $\beta +2\gamma <\alpha <\beta +3\gamma$.

The organization of the paper is as follows. In Section 2, we present some useful preliminaries. In Section 3, we introduce the new method to find the small roots of the equation $x(y^2+Ay+B)+z\equiv 0 (mod e)$. In Section 4, we compare our method with the method of Feng et al. [10] In Section 5, we apply the method to break the RSA variant. In Section 6, we compare our attack with the existing attacks. In Section 7, we conduct numerical experiments to confirm the effectiveness of our new method. Finally, we conclude the paper in Section 8.

2. Preliminaries

In this section, we introduce essential results and key concepts that are relevant to our approach.

2.1. The Cubic Pell Curve

Let N be a positive integer and f be a noncubic residue in $\mathbb{Z}/N\mathbb{Z}$. The cubic Pell curve is defined by the equation

$$x^3+f{y}^3+f^2{z}^3-3fxyz\equiv 1(mod N).$$

(1)

The set of the integers $(x,y,z)\in \mathbb{Z}/N{\mathbb{Z}}^3$ satisfying (1) form a group where the addition of two points $P_1=(x_1,y_1,z_1)$ and $P_2=(x_2,y_2,z_2)$ is defined by

$$P_1\oplus P_2=\left(x_1{x}_2+f(y_2{z}_1+y_1{z}_2),x_2{y}_1+x_1{y}_2+f{z}_1{z}_2,y_1{y}_2+x_2{z}_1+x_1{z}_2\right).$$

The neutral element is $\mathcal{O}=(1,0,0)$, and the inverse of $P=(x,y,z)$ is given by $P^{-1}=(x^2-fyz,f{z}^2-xy,y^2-xz)$.

The use of the cubic Pell curve in cryptography was first proposed by Murru and Saettone in [5]. The proposed scheme is a variant of the RSA scheme, and the key equation is of the form $ed\equiv 1 (mod \psi (N\left)\right)$.

2.2. Useful Lemmas

Let $N=pq$ denote an RSA modulus with $q<p<2q$. The result that follows specifies the bounds for p and q using N.

Lemma 1

([11]). Let p and q be two prime numbers such that $N=pq$ and $q<p<2q$. Then,

$${\displaystyle \frac{\sqrt{2}}{2}}\sqrt{N}<q<\sqrt{N}<p<\sqrt{2}\sqrt{N},$$

and

$$2\sqrt{N}<p+q<3\sqrt{N}.$$

The result below demonstrates that, given an approximation of p for a modulus $N=pq$, it is possible to approximate both q and $p+q$.

Lemma 2

([10]). Let $N=pq$ represent an RSA modulus, where $q<p<2q$. Suppose $p_0$ is an approximation of p such that $|p-p_0|=N^{\gamma }$. Then, the quantity $q_0=⌊{\displaystyle \frac{N}{p_0}}⌋$ is an approximation of q, and the following inequalities hold

$$|q-q_0| <N^{\gamma },|p+q-p_0-q_0|<2N^{\gamma }.$$

2.3. RSA Variants with the Key Equation $ed-k\psi \left(N\right)=1$

In 2003, Said and Loxton [9] introduced ${\mathrm{LUC}}_3$, an enhanced version of the LUC cryptosystem that leverages the third-order linear recurrence of the Lucas function for both encryption and decryption. The system operates with a modulus $N=pq$, where p and q are distinct primes. It employs a public exponent e and a private exponent d, which satisfy the congruence relation $ed\equiv 1 (mod \mathrm{\Phi }(N\left)\right)$. Here, $\mathrm{\Phi }$ is an extension of Euler’s totient function, which can take forms such as $\mathrm{\Phi }\left(N\right)=(p^2-1)(q^2-1)$ or $\mathrm{\Phi }\left(N\right)=\psi \left(N\right)$.

In 2018, Murru and Saettone [5] developed a variant of RSA that incorporates a novel arithmetic operation ⊙ based on the cubic Pell equation $x^3+f{y}^3+f^2{z}^3-3fxyz\equiv 1 (mod N)$. In this context, $N=pq$, and f is defined as a cube nonresidue modulo of both p and q. The public exponent e is selected such that $gcd(e,\psi (N\left)\right)=1$. The private exponent d fulfills the condition $ed\equiv 1 (mod \psi (N\left)\right)$.

Encryption within this framework transforms a plaintext message $(m_1,m_2)$ into the ciphertext $(c_1,c_2)={(m_1,m_2)}^{\odot e} (mod N)$, while decryption retrieves the original plaintext via $(m_1,m_2)={(c_1,c_2)}^{\odot d} (mod N)$.

2.4. Lattice Basis Reduction

In the geometry of numbers, lattices serve as fundamental mathematical structures, representing discrete subgroups within finite-dimensional vector spaces.

Specifically, when the vector space in question is ${\mathbb{R}}^d$, a lattice can be formally defined as follows.

Definition 1.

Let ω and d be positive integers such that $\omega \le d$. Consider ${\mathbf{v}}_1,{\mathbf{v}}_2,\dots ,{\mathbf{v}}_{\omega }$ as ω linearly independent vectors in ${\mathbb{R}}^d$. The lattice $\mathcal{L}\subset {\mathbb{R}}^d$ spanned by $\left\{{\mathbf{v}}_1,{\mathbf{v}}_2,\dots ,{\mathbf{v}}_{\omega }\right\}$ is the set of all integer linear combinations of these vectors, that is,

$$\mathcal{L}=\mathbb{Z}{\mathbf{v}}_1+\mathbb{Z}{\mathbf{v}}_2+\dots +\mathbb{Z}{\mathbf{v}}_{\omega }.$$

The set $\{{\mathbf{v}}_1,{\mathbf{v}}_2,\dots ,{\mathbf{v}}_{\omega }\}$ is called a basis of the lattice $\mathcal{L}$. The positive integers d and ω are referred to as the dimension and rank of the lattice $\mathcal{L}$, respectively. If $\omega =d$, the lattice $\mathcal{L}$ is said to have full rank.

A lattice $\mathcal{L}$ can be represented by a matrix $\mathcal{B}$, where the rows of $\mathcal{B}$ correspond to the basis vectors ${\mathbf{v}}_1,{\mathbf{v}}_2,\dots ,{\mathbf{v}}_{\omega }$. The determinant of the lattice $\mathcal{L}$ is defined as

$$det\left(\mathcal{L}\right)=\sqrt{det\left({\mathcal{B}}^t\mathcal{B}\right)},$$

where ${\mathcal{B}}^t$ denotes the transpose of $\mathcal{B}$. For a full-rank lattice, the determinant simplifies to $det\left(\mathcal{L}\right)=|det(\mathcal{B}\left)\right|.$

It is well known that a lattice $\mathcal{L}$ of rank $\omega \ge 2$ admits infinitely many bases, which are all composed of $\omega$ vectors and sharing the same determinant (see [12]). However, finding a basis consisting of short vectors is a computationally difficult problem, especially as the lattice’s dimension increases.

In 1982, Lenstra, Lenstra, and Lovász [12] introduced the LLL algorithm, which provides an efficient polynomial-time method for finding a reduced basis with relatively short vectors. The following theorem, adapted from [13], quantifies the properties of a reduced basis produced by the LLL algorithm.

Theorem 1

([13]). Let $\mathcal{L}$ be a lattice spanned by a basis $\{{\mathbf{v}}_1,{\mathbf{v}}_2,\dots ,{\mathbf{v}}_{\omega }\}$. The LLL algorithm computes a reduced basis $\{{\mathbf{w}}_1,{\mathbf{w}}_2,\dots ,{\mathbf{w}}_{\omega }\}$, satisfying

$$\parallel {\mathbf{w}}_1\parallel \le \parallel {\mathbf{w}}_2\parallel \le \dots \le \parallel {\mathbf{w}}_j\parallel \le 2^{{\displaystyle \frac{\omega (\omega -1)}{4(\omega +1-j)}}}det{\left(\mathcal{L}\right)}^{{\displaystyle \frac{1}{\omega +1-j}}},$$

for each $j=1,\dots ,\omega$.

2.5. Coppersmith’s Method

Cryptanalysis frequently involves solving polynomial equations. In 1996, Coppersmith [4] introduced a powerful method for efficiently finding small roots of modular polynomial equations of the form $F\left(y\right)\equiv 0 (mod N)$, even when the factorization of the modulus N is unavailable. Over time, this approach has been extended to multivariate polynomials expressed as

$$F(y_1,y_2,\dots ,y_n)=\sum _{i_1,i_2,\dots ,i_n}{\gamma }_{i_1,i_2,\dots ,i_n}{y}_1^{i_1}{y}_2^{i_2}\dots y_n^{i_n},$$

where the coefficients ${\gamma }_{i_1,i_2,\dots ,i_n}$ are integers. The Euclidean norm of such a polynomial is defined as

$$\parallel F(y_1,y_2,\dots ,y_n)\parallel =\sqrt{\sum _{i_1,i_2,\dots ,i_n}{\gamma }_{i_1,i_2,\dots ,i_n}^2}.$$

In 1997, Howgrave-Graham [14] enhanced Coppersmith’s method, presenting a new approach for identifying small roots. This breakthrough led to a fundamental result that became a cornerstone in the field.

Theorem 2

([14]). Let $F(y_1,y_2,\dots ,y_n)\in \mathbb{Z}[y_1,y_2,\dots ,y_n]$ be a multivariate polynomial containing at most ω monomials. For positive integers e and m, assume that the following conditions hold:

1.

$F\left(y_1^{\left(0\right)},y_2^{\left(0\right)},\dots ,y_n^{\left(0\right)}\right)\equiv 0 (mod e^m)$;

2.

$\parallel F(y_1{Y}_1,y_2{Y}_2,\dots ,y_n{Y}_n)\parallel <{\displaystyle \frac{e^m}{\sqrt{\omega }}}$;

3.

For each $i=1,\dots ,n$, $|y_i^{\left(0\right)}| <Y_i$.

Then, $F\left(y_1^{\left(0\right)},y_2^{\left(0\right)},\dots ,y_n^{\left(0\right)}\right)=0$ holds over the integers.

When working with systems involving more than two variables, extensions of Coppersmith’s method often rely on heuristic techniques. For this work, we make the following assumption [3,15,16,17].

Assumption 1.

The polynomials $f_1,f_2,\dots ,f_{\omega }$ derived using the LLL algorithm are algebraically independent.

Under this assumption, the unique solution $\left(y_1^{\left(0\right)},y_2^{\left(0\right)},\dots ,y_n^{\left(0\right)}\right)$ satisfying the system of polynomial equations $f_i(y_1,y_2,\dots ,y_n)=0$ for $i=1,\dots ,n$ can be determined using methods such as Gröbner basis computations or resultants.

3. Solving the Trivariate Polynomial Equation $\mathit{x}({\mathit{y}}^{\mathbf{2}}+\mathit{Ay}+\mathit{B})+\mathit{z}\equiv \mathbf{0} (mod \mathit{e})$

In this section, we present our method to solve the equation $x(y^2+Ay+B)+z\equiv 0 (mod e)$ and present a numerical example to show the details.

3.1. Solving the Equation

Theorem 3.

Let N and e be two positive integers. Let $R\left(y\right)\in \mathbb{Z}\left[y\right]$ be a monic polynomial with degree 2. Suppose there exist three integers $x_0$, $y_0$, and $z_0$ such that $x_{0}R\left(y_0\right)+z_0\equiv 0 (mod e)$, with $e=N^{\alpha }$, $|x_0| \le N^{\beta }$, $|y_0| \le N^{\gamma }$, and $|z_0| \le N^{\mu }$. Then, one can find $x_0$, $y_0$, and $z_0$ in polynomial time if $\alpha \ge 2\gamma$ and

$$\beta +{\displaystyle \frac{1}{2}}\mu <\alpha -\sqrt{2\alpha \gamma }.$$

Proof. 

Write $R\left(y\right)=y^2+Ay+B\in \mathbb{Z}\left[y\right]$ and put $h(x,y,z)=xR\left(y\right)+z$. We use Coppersmith’s method [4] to solve the equation $h(x,y,z)\equiv 0 (mod e)$. Let $m\ge 1$ be an integer and $t>0$ be a parameter to be optimized. Let $w=x{y}^2+z$. Then, $h(x,y,z)=H(x,y,w)$, where

$$H(x,y,w)=w+x(Ay+B).$$

Next, consider the polynomial

$$\begin{array}{cc}\hfill H_{k,e_x,e_y,e_z}& (x,y,z,w)=x^{e_x}{y}^{e_y}{z}^{e_z}H{(x,y,w)}^k{e}^{m-k},\hfill \end{array}$$

where $(k,e_x,e_y,e_z)$ in one of the two disjoint sets

$$\begin{array}{cc}\hfill \mathcal{A}& =\{(k,e_x,e_y,e_z)\mid e_y=0,1,k=0,\dots ,m,e_x=1,\dots ,m-k,e_z=m-e_x-k\},\hfill \\ \hfill \mathcal{B}& =\{(k,e_x,e_y,e_z)\mid e_y=0,\dots ,\lfloor mt\rfloor ,k=⌊{\displaystyle \frac{1}{t}}⌋e_y,\dots ,m,e_x=0,e_z=m-k\}.\hfill \end{array}$$

Setting $w_0=x_0{y}_0^2+z_0$, then the former polynomials satisfy

$$H_{k,e_x,e_y,e_z}(x_0,y_0,z_0,w_0)\equiv 0(mod e^m).$$

To apply Coppersmith’s ideas, the starting point is to consider the lattice $\mathcal{L}$ spanned by the matrix whose rows are the coefficient vectors of the polynomials $H_{k,e_x,e_y,e_z}(xX,yY,zZ,wW)$, where X, Y, Z, and W are positive integers to be determined later. For $(k,e_x,e_y,e_z)\in \mathcal{A}\cup \mathcal{B}$, we can structure the lattice basis matrix in a left triangular form by arranging the rows according to the tuples $(k,e_x,e_y)$ in lexicographical order. Specifically, we say that $(k,e_x,e_y){\prec }_{\mathrm{lex}}(k^{\prime },e_x^{\prime },e_y^{\prime })$ if $k<k^{\prime }$, or if $k=k^{\prime }$ and $e_x<e_x^{\prime }$, or if $k=k^{\prime }$, $e_x=e_x^{\prime }$ and $e_y<e_y^{\prime }$. Similarly, we order the monomials $x^{e_x}{y}^{e_y}{z}^{e_z}{w}^k$ that correspond to the columns of the matrix. Table 1 presents a typical example with $m=2$ and $t=1$. The star entries are nonzero terms which do not contribute to the determinant.

Table 1. The lattice basis matrix for $(m,t)=(2,1)$, where a ★ represents an nonzero entry.

${\mathit{H}}_{\mathit{k},{\mathit{e}}_{\mathit{x}},{\mathit{e}}_{\mathit{y}},{\mathit{e}}_{\mathit{z}}}$	${\mathit{z}}^2$	$\mathit{xz}$	$\mathit{xyz}$	${\mathit{x}}^2$	${\mathit{x}}^2\mathit{y}$	$\mathit{zw}$	$\mathit{yzw}$	$\mathit{xw}$	$\mathit{xyw}$	${\mathit{w}}^2$	${\mathit{yw}}^2$	${\mathit{y}}^2{\mathit{w}}^2$
$H_{0,0,0,2}$	$e^2{Z}^2$	0	0	0	0	0	0	0	0	0	0	0
$H_{0,1,0,1}$	0	$e^{2}XZ$	0	0	0	0	0	0	0	0	0	0
$H_{0,1,1,1}$	0	0	$e^{2}XYZ$	0	0	0	0	0	0	0	0	0
$H_{0,2,0,0}$	0	0	0	$e^2{X}^2$	0	0	0	0	0	0	0	0
$H_{0,2,1,0}$	0	0	0	0	$e^2{X}^{2}Y$	0	0	0	0	0	0	0
$H_{1,0,0,1}$	0	★	★	0	0	$eZW$	0	0	0	0	0	0
$H_{1,0,1,1}$	★	0	★	0	0	★	$eYZW$	0	0	0	0	0
$H_{1,1,0,0}$	0	0	0	★	★	0	0	$eXW$	0	0	0	0
$H_{1,1,1,0}$	0	★	0	0	★	0	0	★	$eXYW$	0	0	0
$H_{2,0,0,0}$	0	★	0	★	★	0	0	★	★	$W^2$	0	0
$H_{2,0,1,0}$	0	★	★	0	★	★	0	★	★	★	$Y{W}^2$	0
$H_{2,0,2,0}$	★	★	★	0	0	★	★	★	★	★	★	$Y^2{W}^2$

Since the matrix of the lattice is triangular, then its determinant is of the form

$$\begin{array}{c}\hfill det\left(\mathcal{L}\right)=X^{{\xi }_X}{Y}^{{\xi }_Y}{Z}^{{\xi }_Z}{W}^{{\xi }_W}{e}^{{\xi }_e}.\end{array}$$

(2)

To compute the former exponents, we need to define the function $s\mapsto \sigma \left(s\right)$ defined by

$$\sigma \left(s\right)=\sum _{e_y=0}^1\sum _{k=0}^m\sum _{e_x=1}^{m-k}s+\sum _{e_y=0}^{\lfloor mt\rfloor }\sum _{k=⌊{\displaystyle \frac{1}{t}}⌋e_y}^m\sum _{e_x=0}^{0}s.$$

To simplify the computations, we approximate $\lfloor mt\rfloor \approx mt$ and $⌊{\displaystyle \frac{1}{t}}⌋\approx {\displaystyle \frac{1}{t}}$. Then, the dominant parts of the former parameters, as well as of the dimension $\omega$ of the lattice, satisfy

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill {\xi }_X& =\sigma \left(e_x\right)={\displaystyle \frac{1}{3}}{m}^3+o\left(m^3\right)\hfill \\ \hfill {\xi }_Y& =\sigma \left(e_y\right)={\displaystyle \frac{1}{6}}{t}^2{m}^3+o\left(m^3\right)\hfill \\ \hfill {\xi }_Z& =\sigma (m-e_x-k)={\displaystyle \frac{1}{6}}(t+2)m^3+o\left(m^3\right)\hfill \\ \hfill {\xi }_W& =\sigma \left(k\right)={\displaystyle \frac{1}{3}}(t+1)m^3+o\left(m^3\right)\hfill \\ \hfill {\xi }_e& =\sigma (m-k)={\displaystyle \frac{1}{6}}(t+4)m^3+o\left(m^3\right)\hfill \\ \hfill \omega & =\sigma \left(1\right)={\displaystyle \frac{1}{2}}(t+2)m^2+o\left(m^2\right).\hfill \end{array}\end{array}$$

(3)

Let $e=N^{\alpha }$, $|x_0| \le N^{\beta }$, $|y_0| \le N^{\gamma }$, and $|z_0| \le N^{\mu }$. To obtain a bound of $w_0$, we suppose $|z_0| < |x_0|y_0^2$ so that $|w_0| = |x_0{y}_0^2+z_0| < 2|x_0|y_0^2$. Hence, the bounds X, Y, Z, and W can be defined in terms of N as follows:

$$\begin{array}{c}\hfill |x_0| \le X=N^{\beta },|y_0| \le Y=N^{\gamma },|z_0| \le Z=N^{\mu },\left|w_0\right|<W=2N^{\beta +2\gamma }.\end{array}$$

(4)

Next, we combine Theorem 2 and Theorem 1 with $i=4$ by setting

$$det\left(\mathcal{L}\right)<2^{-{\displaystyle \frac{\omega (\omega -1)}{4}}}{\omega }^{-{\displaystyle \frac{\omega -3}{2}}}{e}^{m(\omega -3)}.$$

In addition to (2), this gives

$$\begin{array}{c}\hfill e^{{\xi }_e-m(\omega -3)}{X}^{{\xi }_X}{Y}^{{\xi }_Y}{Z}^{{\xi }_Z}{W}^{{\xi }_W}<2^{-{\displaystyle \frac{\omega (\omega -1)}{4}}}{\omega }^{-{\displaystyle \frac{\omega -3}{2}}}.\end{array}$$

(5)

Substituting $X=N^{\beta }$, $Y=N^{\gamma }$, $Z=N^{\mu }$, $W=2N^{\beta +r\gamma }$, and $e=N^{\alpha }$, the former inequality can be simplified as

$$\begin{array}{c}\hfill \gamma t^2+(4\gamma -2\alpha +2\beta +\mu )t+4\gamma -2\alpha +4\beta +2\mu <-\epsilon ,\end{array}$$

(6)

where $\epsilon$ is a small positive constant related to m and N. The optimal value for t is $t_0={\displaystyle \frac{2\alpha -2\beta -4\gamma -\mu }{2\gamma }}$. To satisfy $t_0\ge 0$, the following parameters should hold:

$$\begin{array}{c}\hfill \alpha \ge \beta +2\gamma +{\displaystyle \frac{1}{2}}\mu .\end{array}$$

(7)

The former inequality implies that $\alpha \ge 2\gamma$. Plugging $t_0$ in (6), we get

$$-2{\beta }^2+2(2\alpha -\mu )\beta +4\alpha \gamma -2{\alpha }^2+2\alpha \mu -{\displaystyle \frac{1}{2}}{\mu }^2<-{\epsilon }_2,$$

for a small ${\epsilon }_2>0$. Solving the inequality for $\beta$, we get

$$\begin{array}{c}\hfill \beta +{\displaystyle \frac{1}{2}}\mu <\alpha -\sqrt{2\alpha \gamma }.\end{array}$$

Combining with (7), we get for $\alpha \ge 2\gamma$ that

$$\beta +{\displaystyle \frac{1}{2}}\mu <min\left(\alpha -\sqrt{2\alpha \gamma },\alpha -2\gamma \right)=\alpha -\sqrt{2\alpha \gamma }.$$

By utilizing four reduced, algebraically independent polynomials $f_1(x,y,z,w)$, $f_2(x,y,z,w)$, $f_3(x,y,z,w)$, and $f_4(x,y,z,w)$, one can determine the values of $(x,y,z,w)$ by solving the system of equations $f_i(x,y,z,w)=0$ over the integers. This can be achieved through the application of Gröbner basis methods or resultant techniques, thereby completing the proof. □

3.2. A Numerical Example

Consider the following parameters:

$$\begin{array}{cc}\hfill N=& 2635819154519362642659647391448070624651311688183462920698769,\hfill \\ \hfill A=& 2884138664459193125400607495498882831925138520643501991055090,\hfill \\ \hfill B=& 8318255835828458190632643460128245929395808370538368180671214\setminus \hfill \\ \hfill & 794571788522608753534458460815328697439209903199865441742833,\hfill \\ \hfill e=& 7229977954052138524685255760410197558319269011060224222175946\setminus \hfill \\ \hfill & 238982678044782565876339409047548555673030291683414635524257.\hfill \end{array}$$

imply that $e=N^{\alpha }$, with $\alpha \approx 2.$

The goal here is to find a small solution $(x_0,y_0,z_0)$ of the equation $x(y^2+Ay+B)+z\equiv 0 (mod e)$, with $z_0=1$. To apply Theorem 3, we set $\beta =0.4$$\gamma =0.51$, and $\mu =0$. Hence, the conditions of Theorem 3 are satisfied, namely, $\alpha \ge 2\gamma$ and $\beta +{\displaystyle \frac{1}{2}}\mu <\alpha -\sqrt{2\alpha \gamma }\approx 0.571$. Together with the bound $Z=1$, we take

$$\begin{array}{cc}\hfill & X\approx ⌊N^{\beta }⌋=1527588761761190743375872,\hfill \\ \hfill & Y\approx ⌊N^{\gamma }⌋=6832949794180177482773906325504,\hfill \\ \hfill & W\approx 2X{Y}^2=50785084508005144657742888380047856602690371809450716988\setminus \hfill \\ \hfill & 0416349290534591020103683276800.\hfill \end{array}$$

Let $(m,t)=(3,2)$. The lattice $\mathcal{L}$ is built by the coefficients of the polynomials

$$\begin{array}{cc}\hfill H_{k,e_x,e_y,e_z}& (x,y,z,w)=x^{e_x}{y}^{e_y}{z}^{e_z}H{(x,y,w)}^k{e}^{m-k},\hfill \end{array}$$

where $(k,e_x,e_y,e_z)$ in one of the two disjoint sets

$$\begin{array}{cc}\hfill \mathcal{A}& =\{(k,e_x,e_y,e_z)\mid e_y=0,1,k=0,\dots ,m,e_x=1,\dots ,m-k,e_z=m-e_x-k\},\hfill \\ \hfill \mathcal{B}& =\{(k,e_x,e_y,e_z)\mid e_y=0,\dots ,\lfloor mt\rfloor ,k=⌊{\displaystyle \frac{1}{t}}⌋e_y,\dots ,m,e_x=0,e_z=m-k\},\hfill \end{array}$$

and each term $x{y}^2$ is replaced by $w-z$. The lattice has dimension $\omega =40$. Applying the LLL algorithm to the lattice $\mathcal{L}$ yields a reduced matrix, from which we derive 40 polynomials. A computation of the Gröbner basis then provides the solution

$$\begin{array}{cc}\hfill x_0=& 775498829232538581123082,\hfill \\ \hfill y_0=& 3487384624245570654520525920690,\hfill \\ \hfill w_0=& 943150161306261052140984582339690768969934167219917544953518167111\setminus \hfill \\ \hfill & 1521807051406540201.\hfill \end{array}$$

Both the LLL algorithm and Gröbner basis method took less than one second. Notice that $\alpha \ge \beta +3\gamma =1.93$, and then the bound of Feng et al. [10] is $min\left({\displaystyle \frac{2}{3}}\alpha -{\displaystyle \frac{3}{2}}\gamma ,\alpha -3\gamma \right)\approx 0.469$, which is lower than our bound $\alpha -\sqrt{2\alpha \gamma }\approx 0.571$.

4. Comparison with the Method of Feng et al. [10]

In 2024, Feng et al. [10] solved the modular equation $x(y^2+Ay+B)+1\equiv 0 (mod e)$. They showed that if $\left|x\right|\le N^{\beta }$ and $\left|y\right|\le N^{\gamma }$, then one can find x and y if

$$\beta <\left\{\begin{array}{cccc}& \alpha -\sqrt{2\gamma \alpha },\hfill & & \mathrm{if}\beta +2\gamma <\alpha <\beta +3\gamma ,\hfill \\ & min\left({\displaystyle \frac{2}{3}}\alpha -{\displaystyle \frac{3}{2}}\gamma ,\alpha -3\gamma \right),\hfill & & \mathrm{if}\alpha \ge \beta +3\gamma .\hfill \end{array}\right.$$

In our method, we can solve the equation $x(y^2+Ay+B)+1\equiv 0 (mod e)$ whenever $\alpha \ge 2\gamma$ and $\beta <\alpha -\sqrt{2\gamma \alpha }$. This shows that the range of $\alpha$ is larger their range. Let us compare the bounds of $\beta$. Denote by $\Delta$ the difference between our bound and their bound.

If $\beta +2\gamma <\alpha <\beta +3\gamma$, we can retrieve their bound by setting $\mu =0$ in Theorem 3. This shows that their method is a special case of ours.

If $\alpha \ge \beta +3\gamma$, their bound is

$$min\left({\displaystyle \frac{2}{3}}\alpha -{\displaystyle \frac{3}{2}}\gamma ,\alpha -3\gamma \right)=\left\{\begin{array}{cccc}& {\displaystyle \frac{2}{3}}\alpha -{\displaystyle \frac{3}{2}}\gamma ,\hfill & & \mathrm{if}\alpha \ge {\displaystyle \frac{9}{2}}\gamma ,\hfill \\ & \alpha -3\gamma ,\hfill & & \mathrm{if}\alpha <{\displaystyle \frac{9}{2}}\gamma .\hfill \end{array}\right.$$

For $\alpha \ge {\displaystyle \frac{9}{2}}\gamma$, a simple calculation yields

$$\begin{array}{cc}\hfill \Delta =& \left(\alpha -\sqrt{2\gamma \alpha }\right)-\left({\displaystyle \frac{2}{3}}\alpha -{\displaystyle \frac{3}{2}}\gamma \right)\hfill \\ \hfill =& {\displaystyle \frac{{(2\alpha -9\gamma )}^2}{2\alpha +9\gamma +6\sqrt{2\gamma \alpha }}}\ge 0,\hfill \end{array}$$

which shows that our bound is better that their bound.

For $\alpha <{\displaystyle \frac{9}{2}}\gamma$, a straightforward computation gives

$$\begin{array}{cc}\hfill \Delta =& \left(\alpha -\sqrt{2\gamma \alpha }\right)-\left(\alpha -3\gamma \right)\hfill \\ \hfill =& {\displaystyle \frac{\gamma (9\gamma -2\alpha )}{3\gamma +\sqrt{2\gamma \alpha }}}>0,\hfill \end{array}$$

which shows again that our bound is better than theirs.

5. Application of the New Method

In this section, we apply Theorem 3 to break RSA variants with a key equation of the form $ev-x\psi \left(N\right)=z$.

Theorem 4.

Let $(N,e)$ be an RSA public key with $N=pq$, $q<p<2q$, and $e=N^{\alpha }$. Suppose there are three integers $v_0$, $x_0$, and $z_0$ such that $|z_0| \le N^{\mu }$, $|v_0| \le N^{\zeta }$ and $e{v}_0-x_0\psi \left(N\right)=z_0$. If $p_0$ is a known approximation of p such that $|p-p_0| =N^{\gamma }$, with $0\le \gamma \le {\displaystyle \frac{1}{2}}$, $2\gamma \le \alpha \le {\displaystyle \frac{2}{\gamma }}$, and $\zeta +{\displaystyle \frac{1}{2}}\mu <2-\sqrt{2\alpha \gamma }$, then the modulus N can be factored in polynomial time.

Proof. 

Let $p_0$ be an approximation of p such that $|p-p_0| =N^{\gamma }$. According to Lemma 2, the integer $q_0=⌊{\displaystyle \frac{N}{p_0}}⌋$ is an approximation of q such that

$$|q-q_0| <N^{\gamma },|p+q-p_0-q_0|<2N^{\gamma }.$$

If $N=pq$, then the equation $e{v}_0-x_0\psi \left(N\right)=z_0$ can be rewritten as a modular one, namely, $x_0(y_0^2+A{y}_0+B)+z_0\equiv 0 (mod e)$, where $y_0=p+q-p_0-q_0$, $A=N+2(p_0+q_0)+1$, and

$$B=N^2+(p_0+q_0-1)N+{(p_0+q_0)}^2+p_0+q_0+1.$$

On the other hand, using $\psi \left(N\right)>N^2$, and assuming that $|z_0| <e{v}_0$, we get

$$x_0={\displaystyle \frac{e{v}_0-z_0}{\psi \left(N\right)}}<{\displaystyle \frac{2e{v}_0}{N^2}}\le 2N^{\alpha +\zeta -2}=X.$$

Also, we have $|y_0| <2N^{\gamma }=Y$. To apply Theorem 3, we need a bound for $w_0=x_0{y}_0^2+z_0$. We assume $|z_0| <x_0{y}_0^2$ so that $\left|w\right|<2X{Y}^2=W$. Then, by applying Theorem 3 for $\beta =\alpha +\zeta -2$ and $\gamma$, the inequality $\beta +{\displaystyle \frac{1}{2}}\mu <\alpha -\sqrt{2\alpha \gamma }$ leads to $\zeta +{\displaystyle \frac{1}{2}}\mu <2-\sqrt{2\alpha \gamma }$, and the conditions $\alpha \ge 2\gamma$, $0\le \zeta <2-\sqrt{2\alpha \gamma }$ turn to $2\gamma \le \alpha \le {\displaystyle \frac{2}{\gamma }}$. After finding the root $(x_0,y_0,z_0,w_0)$ and using the values of $N=pq$ and $y_0+p_0+q_0=p+q$, we get p and q. This concludes the proof. □

6. Comparison with the Former Attacks

In this section, we compare our attack with the former attacks in terms of the bounds.

6.1. Comparison with the Attack of Zheng et al. [17]

In [17], Zheng et al. proposed an attack targeting the RSA variant introduced by Murru and Saettone [5], which is based on the key equation $ed-k\psi \left(N\right)=1.$ They showed that the system is vulnerable when $e=N^{\alpha }$, $d=N^{\delta }$, and $\delta <2-\sqrt{\alpha }$. This represents the optimal bound for small private exponent attacks on this scheme.

Let $\Delta$ denote the difference between the bound established in Theorem 4, and then we have $2-\sqrt{\alpha }$. Then,

$$\Delta =2-\sqrt{2\alpha \gamma }-\left(2-\sqrt{\alpha }\right)=\sqrt{\alpha }(1-\sqrt{2\gamma }).$$

By fixing $\gamma <0.5$, it follows that $\Delta >0$, confirming that our bound is always better than theirs.

6.2. Comparison with the Attack of Feng et al. [10]

In [10], Feng et al. proposed an attack on the cryptographic scheme introduced by Murru and Saettone [5], refining the earlier results of Zheng et al. [17]. They demonstrated that for $N=pq$, $e=N^{\alpha }$, $d<N^{\delta }$, and $|p-p_0| =N^{\gamma }$, where $p_0$ is an approximation of p, and the scheme becomes vulnerable if $\delta <2-\sqrt{2\alpha \gamma }$. Notably, this bound can be obtained by setting $\mu =0$ in Theorem 4, indicating that Feng et al.’s method is a specific instance of the more general framework presented in our approach.

On the one hand, the number of exponents e of size $N^{\alpha }$ that are vulnerable to this attack can be approximated by

$$\#\left\{d\mid d<N^{2-\sqrt{2\gamma \alpha }},gcd(d,\psi \left(N\right))=1\right\}=\mathcal{O}\left(N^{2-\sqrt{2\gamma \alpha }-\epsilon }\right),$$

where $\epsilon$ is a small positive value that reflects the exponents that are not coprime with $\psi \left(N\right)$.

On the other hand, the number of weak exponents e of size $N^{\alpha }$ relative to our attack is given by

$$\mathcal{W}=\#\left\{(v_0,z_0)\left|\right|v_0| \le N^{\zeta },\left|z_0\right|\le N^{\mu },\zeta +{\displaystyle \frac{1}{2}}\mu <2-\sqrt{2\gamma \alpha },gcd(v_0,\psi \left(N\right))=1\right\}.$$

This implies that

$$\mathcal{W}\le \#\left\{(v_0,z_0)\left|\right|v_0| \le N^{\zeta },|z_0| \le N^{\mu },\left|v_0\right|\sqrt{|z_0|}<N^{2-\sqrt{2\gamma \alpha }},gcd(v_0,\psi \left(N\right))=1\right\}.$$

Let $b=2-\sqrt{2\gamma \alpha }$. Then, $1\le |z_0| <{\displaystyle \frac{N^{2b}}{|v_0{|}^2}}$, with $1\le |v_0| <N^b$. The upper bound for this count is obtained by summing the number of possible values of $|z_0|$ in the interval $\left[1,{\displaystyle \frac{N^{2b}}{|v_0{|}^2}}\right]$ for each $v_0$ such that $1\le |v_0| <N^b$. This gives

$$\mathcal{W}\le 4\sum _{v_0=1}^{N^b}\sum _{z_0=1}^{{\displaystyle \frac{N^{2b}}{|v_0{|}^2}}}1=4\sum _{v_0=1}^{N^b}{\displaystyle \frac{N^{2b}}{|v_0{|}^2}}=4N^{2b}\sum _{v_0=1}^{N^b}{\displaystyle \frac{1}{|v_0{|}^2}}\le {\displaystyle \frac{2{\pi }^2}{3}}{N}^{2b},$$

where we used the known result ${\sum }_{s=1}^{+\infty }{\displaystyle \frac{1}{s^2}}={\displaystyle \frac{{\pi }^2}{6}}$. Consequently,

$$\mathcal{W}=\mathcal{O}\left(N^{2\left(2-\sqrt{2\gamma \alpha }\right)-\epsilon }\right),$$

where $\epsilon$ is a small positive constant accounting for the integers $v_0$ that are not coprime with $\psi \left(N\right)$. This bound is significantly larger than the number of exponents e of size $N^{\alpha }$ that are vulnerable to the attack proposed by Feng et al. [10].

7. Experimental Results

In this section, we verify the validity of our proposed attacks. The experiments were carried out employing the HPC-MARWAN cluster [18] using SageMath software (version 10.0). HPC-MARWAN is a cluster dedicated to scientific research, providing computational power and storage capacity, deployed by the National Center for Scientific and Technical Research (CNRST) in Morocco.

We showcase the effectiveness of our method in breaking the RSA variant. We also highlight that our attack retains its efficacy even when both the public exponent e and the private exponent d are approximately of size $N^2$, which is a scenario in which existing attacks fail to be effective.

7.1. A Detailed Example for Theorem 4 with the Equation $ev-x(y^2+Ay+B)=z$

Here are the following public values to consider: $N\approx 2^{512}$ and $e\approx 2^{1022}$. Furthermore, we have

$$\begin{array}{cc}\hfill N=& 92090123703993789630510505576230736645889095247500414117819966585\setminus \hfill \\ \hfill & 56971851097330338536685771902273569360378931731016906043849221198\setminus \hfill \\ \hfill & 198036935744123832820589,\hfill \\ \hfill e=& 38934215691607950469120012361006013254362521225220257539170479082\setminus \hfill \\ \hfill & 12083936845833181184061443322817097190960300942509047121150184684\setminus \hfill \\ \hfill & 05032721615209801105248073935917468358425919934056305719304476900\setminus \hfill \\ \hfill & 13326171658830108432349387168622831806240456963423109366659468641\setminus \hfill \\ \hfill & 767246402078467795775932869025626858492401033951,\hfill \\ \hfill p_0=& 11579208923731619542357098500868790785326998466564056403945758400\setminus \hfill \\ \hfill & 7913129619533.\hfill \end{array}$$

This yields $e=N^{\alpha }$, with $\alpha \approx 1.9978$. Additionally, $q_0=⌊{\displaystyle \frac{N}{p_0}}⌋$ provides an approximation of q, where

$$\begin{array}{cc}\hfill q_0=& 795305830567188711157306392157135729480301169187366271932474635603\setminus \hfill \\ \hfill & 00498156651.\hfill \end{array}$$

The objective is to find a small solution $(x_0,y_0,z_0)$ to the equation

$$x(y^2+Ay+B)+z\equiv 0(mod \psi \left(N\right)),$$

where the coefficients A and B are given by

$$\begin{array}{cc}\hfill A& =N+2(p_0+q_0)+1,\hfill \\ \hfill B& =N^2+(p_0+q_0-1)N+{(p_0+q_0)}^2+p_0+q_0+1.\hfill \end{array}$$

Specifically, we have

$$\begin{array}{cc}\hfill A=& 920901237039937896305105055762307366458890952475004141178199\setminus \hfill \\ \hfill & 665855697185109772098388127384203535217260882773469261950624\setminus \hfill \\ \hfill & 7017975580663447030880551088372958,\hfill \\ \hfill B=& 848059088381687885222692986094202987557717535259737888174631\setminus \hfill \\ \hfill & 414964151825948608250261552605339973955837573041297651395427\setminus \hfill \\ \hfill & 901782819579679100678550541876230524170482254615158647078480\setminus \hfill \\ \hfill & 961854145040641977050073093968898196703731531353503540263696\setminus \hfill \\ \hfill & 969306755617743680388039859455084301800191045010714420214587\setminus \hfill \\ \hfill & 73916749.\hfill \end{array}$$

To apply the method outlined in Theorem 4, we assume the following conditions $|v_0| \le N^{\zeta }$, $|z_0| \le N^{\mu }$, and $|p-p_0| \approx N^{\gamma }$, where $\zeta =0.4$, $\mu =0.3899$, and $\gamma =0.49179$. It is important to note that these conditions satisfy the requirements of Theorem 4, specifically

$$\begin{array}{c}\left\{\begin{array}{c}2\gamma \approx 0.983<\alpha <{\displaystyle \frac{2}{\gamma }}\approx 4.066,\hfill \\ \zeta +{\displaystyle \frac{1}{2}}\mu \approx 0.594<2-\sqrt{2\alpha \gamma }\approx 0.598.\hfill \end{array}\right.\end{array}$$

This establishes the bounds

$$\begin{array}{cc}\hfill X& \approx \lfloor 2N^{\alpha +\zeta -2}\rfloor =35317824933741584086703883528303714266057154312907285\setminus \hfill \\ \hfill & 271150592,\hfill \\ \hfill Y& \approx \lfloor 2N^{\gamma }\rfloor =104493754698738524355519630920748673797643265656422206730\setminus \hfill \\ \hfill & 78934100159252922368,\hfill \\ \hfill Z& \approx \lfloor N^{\mu }\rfloor =1073116001210012283097441801606665179596152191451837623173\setminus \hfill \\ \hfill & 120,\hfill \\ \hfill W& =2X{Y}^2\approx 7712667597695774572677877060455265161976005042632523391603\setminus \hfill \\ \hfill & 081758469953781216418569262422889415953648852041303652656380831526898\setminus \hfill \\ \hfill & 381818448408398069785728319647442138981910138399151181599383434359979\setminus \hfill \\ \hfill & 090634271952470016.\hfill \end{array}$$

Setting $m=4$ and $t=2$, we construct a lattice $\mathcal{L}$ of dimension $\omega =65$ using the previous methodology. By applying the LLL algorithm, we obtain a reduced matrix from which we derive 65 polynomials. Utilizing the Gröbner basis method, we then select the first four polynomials and solve them over the integers. This process yields the following solution

$$\begin{array}{cc}\hfill x_0=& 476922631141216258824709909224364166021406637821749589143304,\hfill \\ \hfill y_0=& -14656198973036963042431218136491400396244274845042329427279636\setminus \hfill \\ \hfill & 15709918784714,\hfill \\ \hfill z_0=& 996433282843842715907804162488388224532595860353406044735637,\hfill \\ \hfill w_0=& 1024449691435018776340137001955759656771048969519650439490082783\setminus \hfill \\ \hfill & 2638023125347661454319655031429861706474386683637844031231494225\setminus \hfill \\ \hfill & 2383581431101959647815237784662331111283958734677149072166350844\setminus \hfill \\ \hfill & 5598874271464349621.\hfill \end{array}$$

Consequently, $v_0$ can be computed using

$$\begin{array}{cc}\hfill v_0& ={\displaystyle \frac{x_0(y_0^2+A{y}_0+B)+z_0}{e}},\hfill \\ \hfill & =1038825528162352588568000862000004781199865632503791922107619.\hfill \end{array}$$

Note that the conditions $|z_0| <e{v}_0$ and $|z_0| <x_0{y}_0^2$ are satisfied, as stipulated in Theorem 4. The LLL algorithm and the Gröbner basis method were executed in less than nine seconds. Using $N=pq$ and $y_0+p_0+q_0=p+q$, we obtain

$$\begin{array}{cc}\hfill p=& 110571356571418192257697953498072866373701107402166325725750051578\setminus \hfill \\ \hfill & 456079806541,\hfill \\ \hfill q=& 832856958253131779773605489126794743879745666977066325642270323740\setminus \hfill \\ \hfill & 47629184929.\hfill \end{array}$$

Observe that e is of the form

$$e\equiv {\displaystyle \frac{z_0}{v_0}} \left(mod \psi \left(N\right)\right).$$

Then, we can compute $d\equiv {\displaystyle \frac{1}{e}} \left(mod \psi \left(N\right)\right)$, and obtain

$$\begin{array}{cc}\hfill d=& 1655976650302073678969564407000807245248647342975144716132433455575\setminus \hfill \\ \hfill & 6230549465308393591474500785245363456869535980499891838145913408081\setminus \hfill \\ \hfill & 2794424287762914612538408390381867384695524863158843802553880691450\setminus \hfill \\ \hfill & 9112178358072157830766955945445112918958250946247537278292973203123\setminus \hfill \\ \hfill & 5295228023136388116354527659154529687250.\hfill \end{array}$$

Hence $d=N^{\delta }$ with $\delta \approx 1.995$.

Notice that the condition established by Feng [10] for breaking the system is given by $\delta <2-\sqrt{2\gamma \alpha }$, which represents the optimal bound for partial prime exposure attacks. In this numerical example, we find that $2-\sqrt{2\gamma \alpha }\approx 0.598<\delta$. This shows that their method cannot be broken the system in this particular instance.

Similarly, the optimal bound for small private exponent attacks is given by $\delta <2-\sqrt{\alpha }$, as presented by Zheng et al. [17]. In our example, we find that $2-\sqrt{\alpha }\approx 0.586<\delta$. This indicates that their method is insufficient to break the system in this instance.

7.2. Experiments for Theorem 4 for Large Public Keys

We implement the method outlined in Theorem 4 using large values for the public key $\left(N,e\right)$. Through a series of experiments, we solve the equation $ev-xR\left(y\right)=z$, where

$$R\left(y\right)=y^2+\left(N+2(p_0+q_0)+1\right)y+N^2+(p_0+q_0-1)N+{(p_0+q_0)}^2+p_0+q_0+1.$$

This approach allows us to factor the modulus $N=pq$ efficiently when the most significant bits of p are known. The experiments are summarized in Table 2 where the parameters in each column are defined as follows.

Table 2. Experimental results by exposing the most significant bits of p.

nb (N)	nb (e)	$\mathit{\zeta }$	$\mathit{\delta }$	$\mathit{\alpha }$	$\mathit{\mu }$	$\mathit{\gamma }$	nbk (p)	m	t	$\mathit{\omega }$	Time (s)
700	1399	0.35657	2.000	1.9993	0.390	0.49676	50	4	2	65	13.48
800	1598	0.36285	1.994	1.9996	0.390	0.49710	88	4	2	65	16.59
899	1798	0.37716	1.999	1.9991	0.390	0.49734	138	4	2	65	18.47
1000	1999	0.36942	1.998	1.9988	0.390	0.49558	250	4	2	65	22.04
1299	2598	0.34597	2.000	2.0000	0.390	0.49879	177	4	2	65	35.99
1499	2996	0.37337	1.999	1.9992	0.390	0.49986	250	4	2	65	42.20
1999	3997	0.34999	1.999	1.9993	0.390	0.49904	270	3	3	52	09.06
2499	4997	0.36788	2.000	1.9997	0.390	0.49948	282	3	3	52	12.59
3000	5999	0.36650	2.000	1.9998	0.390	0.49895	532	3	3	52	17.37

• $\mathrm{nb}\left(x\right)$ stands for the number of bits of x.
• $\zeta$ is a parameter satisfies $\left|v\right|\le N^{\zeta }$.
• $\delta$ is the parameter for which $d=N^{\delta }$.
• $\alpha$ is defined by $e=N^{\alpha }$.
• $\mu$ is a parameter such that $\left|z\right|\le N^{\mu }$.
• $\gamma$ is the parameter defined by $\gamma ={\displaystyle \frac{log\left(|p-p_0|\right)}{log\left(N\right)}}$.
• $\mathrm{nbk}\left(p\right)$ stands for the number of known bits of p.
• m and t are parameters for constructing the lattice $\mathcal{L}$ with dimension $\omega$.
• Time is specified for the time in seconds required to perform both the LLL algorithm and the Gröbner basis method.

Table 2 presents our experimental results when the most significant bits of the prime factor p of $N=pq$ are known. These results are related to the method of Theorem 4.

8. Conclusions

In this paper, we introduced a novel approach for solving the generalized equation $ev-x(y^2+Ay+B)=z$, where x, y, and z are small unknown integers, and $A,B\in \mathbb{Z}$. Our method builds upon Coppersmith’s technique and leverages lattice basis reduction. Furthermore, we applied this approach to the cryptanalysis of the RSA variants based on the cubic Pell equation, represented as $eu-x(p^2+p+1)(q^2+q+1)=z$. Notably, our technique enabled the factorization of the RSA modulus $N=pq$ in polynomial time. This research demonstrates that our attack improves upon previous methods such the method of Feng et al. [10] and the method of Zheng et al. [17], targeting small private exponent attacks and partial prime exposure attacks.