Next Article in Journal
Multiparty Homomorphic Encryption for IoV Based on Span Program and Conjugate Search Problem
Previous Article in Journal
Key Derivation: A Dynamic PBKDF2 Model for Modern Cryptographic Systems
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

An Improved Attack on the RSA Variant Based on Cubic Pell Equation

1
ACSA Laboratory, Department of Mathematics and Computer Science, Sciences Faculty, Mohammed First University, Oujda 60000, Morocco
2
LMNO, CNRS, UNICAEN, Caen Normandie University, 14000 Caen, France
3
Faculty of Sciences and Technology Al Hoceima, Abdelmalek Essaadi University, BP 34. Ajdir, Al Hoceima 32003, Morocco
*
Author to whom correspondence should be addressed.
Cryptography 2025, 9(2), 40; https://doi.org/10.3390/cryptography9020040
Submission received: 25 April 2025 / Revised: 2 June 2025 / Accepted: 3 June 2025 / Published: 6 June 2025

Abstract

In this paper, we present a novel method to solve trivariate polynomial modular equations of the form x ( y 2 + A y + B ) + z 0   ( mod   e ) . Our approach integrates Coppersmith’s method with lattice basis reduction to efficiently solve the former equation. Several variants of RSA are based on the cubic Pell equation x 3 + f y 3 + f 2 z 3 3 f x y z 1   ( mod   N ) , where f is a cubic nonresidue modulus N = p q . In these variants, the public exponent e and the private exponent d satisfy e d 1   ( mod   ψ ( N ) ) with ψ ( N ) = p 2 + p + 1 q 2 + q + 1 . Moreover, d can be written in the form d v 0 z 0   ( mod   ψ ( N ) ) with any z 0 satisfying gcd ( z 0 , ψ ( N ) ) = 1 . In this paper, we apply our method to attack the variants when d v 0 z 0   ( mod   ψ ( N ) ) and when | z 0 | and | v 0 | are suitably small. We also show that our method significantly improves the bounds of the private exponents d of the previous attacks on the variants, particularly in the scenario of small private exponents and in the scenarios where partial information about the primes is available.

1. Introduction

In 1978, Rivest, Shamir, and Adleman [1] proposed a number theoretic system, called RSA, as the first public key cryptosystem. It is highly significant in terms of its practical applications. Its security is based on hard mathematical problems such as the integer factorization problem. In the RSA cryptosystem, two large prime numbers p and q of equal bit size are generated, and the RSA modulus N = p q is then computed. Next, a public exponent e is selected to satisfy gcd e , φ ( N ) = 1 , where φ ( N ) = ( p 1 ) ( q 1 ) is the Euler’s totient function. This enables computation of the private exponent d, which is the inverse of e modulo φ ( N ) , that is, d satisfies the modular equation e d 1   ( mod   φ ( N ) ) . The equation e d k φ ( N ) = 1 is well defined and called the key equation. The public key consists of the pair ( N , e ) , whereas ( N , d ) is the private key. For a plaintext m such that m < N , the encryption is done by computing c m e   ( mod   N ) . The decryption of the ciphertext c is performed by calculating m c d   ( mod   N ) . It is well known that the encryption time is O ( log ( N ) ) 2 log ( e ) , while the decryption time is O ( log ( N ) ) 2 log ( d ) .
To enhance the performance of RSA, small private exponents are tempting to use. However, in 1990, Wiener [2] showed that one can break RSA for private exponents d < 1 3 N 1 4 . This bound was later improved by Boneh and Durfee [3] up to N 0.292 .
In 1996, Coppersmith [4] proposed a polynomial time algorithm to find small roots of modular polynomial equations of the form f ( x ) 0   ( mod   M ) , where f ( x ) Z [ x ] , and M is an integer of unknown factorization. As a consequence, Coppersmith showed that one can break RSA if the public exponent is 3 under some constraints. Since then, Coppersmith’s method has been generalized to polynomials with more variables.
In 1998, Boneh and Durfee [3] presented an attack on RSA by transforming the key equation e d k φ ( N ) = 1 into a specific equation x ( N + y ) 1   ( mod   e ) . They used Coppersmith’s method to extract the root ( x , y ) = ( k , p + q 1 ) and then factored the modulus N in polynomial time. As a significant result, they improved Wiener’s bound up to N 0.292 .
To enhance both security and efficiency in RSA, alternative schemes have been proposed. Some of these variants retain the same modulus N = p q but modify the Euler totient function φ ( N ) , replacing it with ψ ( N ) = ( p 2 + p + 1 ) ( q 2 + q + 1 ) m as in [5], or with ψ ( N ) = ( p 2 1 ) ( q 2 1 ) , as in [6,7,8,9].
Recently, Feng et al. [10] proposed a new method to solve the modular polynomial equation f ( x , y ) 0   ( mod   e ) , where f ( x , y ) = x ( y 2 + A y + B ) + 1 . Their method is based on Coppersmith’s method and lattice-based reduction. They showed that e = N α , | x | N β , | y | N γ , and
β < α 2 γ α , if β + 2 γ < α < β + 3 γ , min 2 3 α 3 2 γ , α 3 γ , if α β + 3 γ .
Thus, one can find x and y in polynomial time. They used their method to break some RSA variants by performing a partial prime exposure attack. They proved that if N = p q is an RSA modulus, p 0 is an approximation of p such that | p p 0 | = N γ , and e = N α satisfies the key equation e d k ψ ( N ) = 1 with d < N δ , then the factorization of N can be found under the conditions
δ < 2 2 α γ , if 2 γ < α < 9 2 γ , 2 1 3 α 3 2 γ , if 9 2 γ α < 6 9 2 γ .
Observe that a private exponent d can be written in the form d v 0 z 0   ( mod   ψ ( N ) ) with any z 0 , satisfying gcd ( z 0 , ψ ( N ) ) = 1 and v 0 d z 0   ( mod   ψ ( N ) ) . From the modular equation e d 1 mod   ψ ( N ) , one gets e v 0 z 0 1   ( mod   ψ ( N ) ) , and consequently, e v 0 z 0   ( mod   ψ ( N ) ) .
In this paper, we transform the modular equation e v 0 z 0   ( mod   ψ ( N ) ) into an equation of the form x ( y 2 + A y + B ) + z 0   ( mod   e ) and present a method to find its small solutions ( x , y , z ) . We then apply our method to attack the variants when d v 0 z 0   ( mod   ψ ( N ) ) , and | z 0 | and | v 0 | are suitably small.
As a byproduct, we revisit the work of Feng et al. [10] by presenting a different approach for solving the generalized equation x ( y 2 + A y + B ) + z 0   ( mod   e ) . We prove that if e = N α , | x | N β , | y | N γ , z N μ , α 2 γ , and β + 1 2 μ < α 2 α γ , one can find x, y, and z in polynomial time. This improves the bound of Feng et al. [10] for α β + 3 γ . Notice that their method becomes a particular case of our method if μ = 0 and β + 2 γ < α < β + 3 γ .
The organization of the paper is as follows. In Section 2, we present some useful preliminaries. In Section 3, we introduce the new method to find the small roots of the equation x ( y 2 + A y + B ) + z 0   ( mod   e ) . In Section 4, we compare our method with the method of Feng et al. [10] In Section 5, we apply the method to break the RSA variant. In Section 6, we compare our attack with the existing attacks. In Section 7, we conduct numerical experiments to confirm the effectiveness of our new method. Finally, we conclude the paper in Section 8.

2. Preliminaries

In this section, we introduce essential results and key concepts that are relevant to our approach.

2.1. The Cubic Pell Curve

Let N be a positive integer and f be a noncubic residue in Z / N Z . The cubic Pell curve is defined by the equation
x 3 + f y 3 + f 2 z 3 3 f x y z 1 ( mod   N ) .
The set of the integers ( x , y , z ) Z / N Z 3 satisfying (1) form a group where the addition of two points P 1 = ( x 1 , y 1 , z 1 ) and P 2 = ( x 2 , y 2 , z 2 ) is defined by
P 1 P 2 = x 1 x 2 + f ( y 2 z 1 + y 1 z 2 ) , x 2 y 1 + x 1 y 2 + f z 1 z 2 , y 1 y 2 + x 2 z 1 + x 1 z 2 .
The neutral element is O = ( 1 , 0 , 0 ) , and the inverse of P = ( x , y , z ) is given by P 1 = ( x 2 f y z , f z 2 x y , y 2 x z ) .
The use of the cubic Pell curve in cryptography was first proposed by Murru and Saettone in [5]. The proposed scheme is a variant of the RSA scheme, and the key equation is of the form e d 1   ( mod   ψ ( N ) ) .

2.2. Useful Lemmas

Let N = p q denote an RSA modulus with q < p < 2 q . The result that follows specifies the bounds for p and q using N.
Lemma 1
([11]). Let p and q be two prime numbers such that N = p q and q < p < 2 q . Then,
2 2 N < q < N < p < 2 N ,
and
2 N < p + q < 3 N .
The result below demonstrates that, given an approximation of p for a modulus N = p q , it is possible to approximate both q and p + q .
Lemma 2
([10]). Let N = p q represent an RSA modulus, where q < p < 2 q . Suppose p 0 is an approximation of p such that | p p 0 | = N γ . Then, the quantity q 0 = N p 0 is an approximation of q, and the following inequalities hold
| q q 0 |   < N γ , | p + q p 0 q 0 | < 2 N γ .

2.3. RSA Variants with the Key Equation e d k ψ ( N ) = 1

In 2003, Said and Loxton [9] introduced LUC 3 , an enhanced version of the LUC cryptosystem that leverages the third-order linear recurrence of the Lucas function for both encryption and decryption. The system operates with a modulus N = p q , where p and q are distinct primes. It employs a public exponent e and a private exponent d, which satisfy the congruence relation e d 1   ( mod   Φ ( N ) ) . Here, Φ is an extension of Euler’s totient function, which can take forms such as Φ ( N ) = ( p 2 1 ) ( q 2 1 ) or Φ ( N ) = ψ ( N ) .
In 2018, Murru and Saettone [5] developed a variant of RSA that incorporates a novel arithmetic operation ⊙ based on the cubic Pell equation x 3 + f y 3 + f 2 z 3 3 f x y z 1   ( mod   N ) . In this context, N = p q , and f is defined as a cube nonresidue modulo of both p and q. The public exponent e is selected such that gcd ( e , ψ ( N ) ) = 1 . The private exponent d fulfills the condition e d 1   ( mod   ψ ( N ) ) .
Encryption within this framework transforms a plaintext message ( m 1 , m 2 ) into the ciphertext ( c 1 , c 2 ) = ( m 1 , m 2 ) e   ( mod   N ) , while decryption retrieves the original plaintext via ( m 1 , m 2 ) = ( c 1 , c 2 ) d   ( mod   N ) .

2.4. Lattice Basis Reduction

In the geometry of numbers, lattices serve as fundamental mathematical structures, representing discrete subgroups within finite-dimensional vector spaces.
Specifically, when the vector space in question is R d , a lattice can be formally defined as follows.
Definition 1.
Let ω and d be positive integers such that ω d . Consider v 1 , v 2 , , v ω as ω linearly independent vectors in R d . The lattice L R d spanned by v 1 , v 2 , , v ω is the set of all integer linear combinations of these vectors, that is,
L = Z v 1 + Z v 2 + + Z v ω .
The set { v 1 , v 2 , , v ω } is called a basis of the lattice L . The positive integers d and ω are referred to as the dimension and rank of the lattice L , respectively. If ω = d , the lattice L is said to have full rank.
A lattice L can be represented by a matrix B , where the rows of B correspond to the basis vectors v 1 , v 2 , , v ω . The determinant of the lattice L is defined as
det ( L ) = det ( B t B ) ,
where B t denotes the transpose of B . For a full-rank lattice, the determinant simplifies to det ( L ) = | det ( B ) | .
It is well known that a lattice L of rank ω 2 admits infinitely many bases, which are all composed of ω vectors and sharing the same determinant (see [12]). However, finding a basis consisting of short vectors is a computationally difficult problem, especially as the lattice’s dimension increases.
In 1982, Lenstra, Lenstra, and Lovász [12] introduced the LLL algorithm, which provides an efficient polynomial-time method for finding a reduced basis with relatively short vectors. The following theorem, adapted from [13], quantifies the properties of a reduced basis produced by the LLL algorithm.
Theorem 1
([13]). Let L be a lattice spanned by a basis { v 1 , v 2 , , v ω } . The LLL algorithm computes a reduced basis { w 1 , w 2 , , w ω } , satisfying
w 1     w 2     w j   2 ω ( ω 1 ) 4 ( ω + 1 j ) det ( L ) 1 ω + 1 j ,
for each j = 1 , , ω .

2.5. Coppersmith’s Method

Cryptanalysis frequently involves solving polynomial equations. In 1996, Coppersmith [4] introduced a powerful method for efficiently finding small roots of modular polynomial equations of the form F ( y ) 0   ( mod   N ) , even when the factorization of the modulus N is unavailable. Over time, this approach has been extended to multivariate polynomials expressed as
F ( y 1 , y 2 , , y n ) = i 1 , i 2 , , i n γ i 1 , i 2 , , i n y 1 i 1 y 2 i 2 y n i n ,
where the coefficients γ i 1 , i 2 , , i n are integers. The Euclidean norm of such a polynomial is defined as
F ( y 1 , y 2 , , y n )   = i 1 , i 2 , , i n γ i 1 , i 2 , , i n 2 .
In 1997, Howgrave-Graham [14] enhanced Coppersmith’s method, presenting a new approach for identifying small roots. This breakthrough led to a fundamental result that became a cornerstone in the field.
Theorem 2
([14]). Let F ( y 1 , y 2 , , y n ) Z [ y 1 , y 2 , , y n ] be a multivariate polynomial containing at most ω monomials. For positive integers e and m, assume that the following conditions hold:
1.
F y 1 ( 0 ) , y 2 ( 0 ) , , y n ( 0 ) 0   ( mod   e m ) ;
2.
F ( y 1 Y 1 , y 2 Y 2 , , y n Y n )   < e m ω ;
3.
For each i = 1 , , n , | y i ( 0 ) |   < Y i .
Then,  F y 1 ( 0 ) , y 2 ( 0 ) , , y n ( 0 ) = 0 holds over the integers.
When working with systems involving more than two variables, extensions of Coppersmith’s method often rely on heuristic techniques. For this work, we make the following assumption [3,15,16,17].
Assumption 1.
The polynomials f 1 , f 2 , , f ω derived using the LLL algorithm are algebraically independent.
Under this assumption, the unique solution y 1 ( 0 ) , y 2 ( 0 ) , , y n ( 0 ) satisfying the system of polynomial equations f i ( y 1 , y 2 , , y n ) = 0 for i = 1 , , n can be determined using methods such as Gröbner basis computations or resultants.

3. Solving the Trivariate Polynomial Equation x ( y 2 + Ay + B ) + z 0   ( mod   e )

In this section, we present our method to solve the equation x ( y 2 + A y + B ) + z 0   ( mod   e ) and present a numerical example to show the details.

3.1. Solving the Equation

Theorem 3.
Let N and e be two positive integers. Let R ( y ) Z [ y ] be a monic polynomial with degree 2. Suppose there exist three integers x 0 , y 0 , and z 0 such that x 0 R ( y 0 ) + z 0 0   ( mod   e ) , with e = N α , | x 0 |   N β , | y 0 |   N γ , and | z 0 |   N μ . Then, one can find x 0 , y 0 , and z 0 in polynomial time if α 2 γ and
β + 1 2 μ < α 2 α γ .
Proof. 
Write R ( y ) = y 2 + A y + B Z [ y ] and put h ( x , y , z ) = x R ( y ) + z . We use Coppersmith’s method [4] to solve the equation h ( x , y , z ) 0   ( mod   e ) . Let m 1 be an integer and t > 0 be a parameter to be optimized. Let w = x y 2 + z . Then, h ( x , y , z ) = H ( x , y , w ) , where
H ( x , y , w ) = w + x ( A y + B ) .
Next, consider the polynomial
H k , e x , e y , e z ( x , y , z , w ) = x e x y e y z e z H ( x , y , w ) k e m k ,
where ( k , e x , e y , e z ) in one of the two disjoint sets
A = { ( k , e x , e y , e z ) e y = 0 , 1 , k = 0 , , m , e x = 1 , , m k , e z = m e x k } , B = { ( k , e x , e y , e z ) e y = 0 , , m t , k = 1 t e y , , m , e x = 0 , e z = m k } .
Setting w 0 = x 0 y 0 2 + z 0 , then the former polynomials satisfy
H k , e x , e y , e z ( x 0 , y 0 , z 0 , w 0 ) 0 ( mod   e m ) .
To apply Coppersmith’s ideas, the starting point is to consider the lattice L spanned by the matrix whose rows are the coefficient vectors of the polynomials H k , e x , e y , e z ( x X , y Y , z Z , w W ) , where X, Y, Z, and W are positive integers to be determined later. For ( k , e x , e y , e z ) A B , we can structure the lattice basis matrix in a left triangular form by arranging the rows according to the tuples ( k , e x , e y ) in lexicographical order. Specifically, we say that ( k , e x , e y ) lex ( k , e x , e y ) if k < k , or if k = k and e x < e x , or if k = k , e x = e x and e y < e y . Similarly, we order the monomials x e x y e y z e z w k that correspond to the columns of the matrix. Table 1 presents a typical example with m = 2 and t = 1 . The star entries are nonzero terms which do not contribute to the determinant.
Since the matrix of the lattice is triangular, then its determinant is of the form
det ( L ) = X ξ X Y ξ Y Z ξ Z W ξ W e ξ e .
To compute the former exponents, we need to define the function s σ ( s ) defined by
σ ( s ) = e y = 0 1 k = 0 m e x = 1 m k s + e y = 0 m t k = 1 t e y m e x = 0 0 s .
To simplify the computations, we approximate m t m t and 1 t 1 t . Then, the dominant parts of the former parameters, as well as of the dimension ω of the lattice, satisfy
ξ X = σ ( e x ) = 1 3 m 3 + o ( m 3 ) ξ Y = σ ( e y ) = 1 6 t 2 m 3 + o ( m 3 ) ξ Z = σ ( m e x k ) = 1 6 ( t + 2 ) m 3 + o ( m 3 ) ξ W = σ ( k ) = 1 3 ( t + 1 ) m 3 + o ( m 3 ) ξ e = σ ( m k ) = 1 6 ( t + 4 ) m 3 + o ( m 3 ) ω = σ ( 1 ) = 1 2 ( t + 2 ) m 2 + o ( m 2 ) .
Let e = N α , | x 0 |   N β , | y 0 |   N γ , and | z 0 |   N μ . To obtain a bound of w 0 , we suppose | z 0 |   <   | x 0 | y 0 2 so that | w 0 |   =   | x 0 y 0 2 + z 0 |   <   2 | x 0 | y 0 2 . Hence, the bounds X, Y, Z, and W can be defined in terms of N as follows:
| x 0 |   X = N β , | y 0 |   Y = N γ , | z 0 |   Z = N μ , | w 0 | < W = 2 N β + 2 γ .
Next, we combine Theorem 2 and Theorem 1 with i = 4 by setting
det ( L ) < 2 ω ( ω 1 ) 4 ω ω 3 2 e m ( ω 3 ) .
In addition to (2), this gives
e ξ e m ( ω 3 ) X ξ X Y ξ Y Z ξ Z W ξ W < 2 ω ( ω 1 ) 4 ω ω 3 2 .
Substituting X = N β , Y = N γ , Z = N μ , W = 2 N β + r γ , and e = N α , the former inequality can be simplified as
γ t 2 + ( 4 γ 2 α + 2 β + μ ) t + 4 γ 2 α + 4 β + 2 μ < ε ,
where ε is a small positive constant related to m and N. The optimal value for t is t 0 = 2 α 2 β 4 γ μ 2 γ . To satisfy t 0 0 , the following parameters should hold:
α β + 2 γ + 1 2 μ .
The former inequality implies that α 2 γ . Plugging t 0 in (6), we get
2 β 2 + 2 ( 2 α μ ) β + 4 α γ 2 α 2 + 2 α μ 1 2 μ 2 < ε 2 ,
for a small ε 2 > 0 . Solving the inequality for β , we get
β + 1 2 μ < α 2 α γ .
Combining with (7), we get for α 2 γ that
β + 1 2 μ < min α 2 α γ , α 2 γ = α 2 α γ .
By utilizing four reduced, algebraically independent polynomials f 1 ( x , y , z , w ) , f 2 ( x , y , z , w ) , f 3 ( x , y , z , w ) , and f 4 ( x , y , z , w ) , one can determine the values of ( x , y , z , w ) by solving the system of equations f i ( x , y , z , w ) = 0 over the integers. This can be achieved through the application of Gröbner basis methods or resultant techniques, thereby completing the proof. □

3.2. A Numerical Example

Consider the following parameters:
N = 2635819154519362642659647391448070624651311688183462920698769 , A = 2884138664459193125400607495498882831925138520643501991055090 , B = 8318255835828458190632643460128245929395808370538368180671214 794571788522608753534458460815328697439209903199865441742833 , e = 7229977954052138524685255760410197558319269011060224222175946 238982678044782565876339409047548555673030291683414635524257 .
imply that e = N α , with α 2 .
The goal here is to find a small solution ( x 0 , y 0 , z 0 ) of the equation x ( y 2 + A y + B ) + z 0   ( mod   e ) , with z 0 = 1 . To apply Theorem 3, we set β = 0.4 γ = 0.51 , and μ = 0 . Hence, the conditions of Theorem 3 are satisfied, namely, α 2 γ and β + 1 2 μ < α 2 α γ 0.571 . Together with the bound Z = 1 , we take
X N β = 1527588761761190743375872 , Y N γ = 6832949794180177482773906325504 , W 2 X Y 2 = 50785084508005144657742888380047856602690371809450716988 0416349290534591020103683276800 .
Let ( m , t ) = ( 3 , 2 ) . The lattice L is built by the coefficients of the polynomials
H k , e x , e y , e z ( x , y , z , w ) = x e x y e y z e z H ( x , y , w ) k e m k ,
where ( k , e x , e y , e z ) in one of the two disjoint sets
A = { ( k , e x , e y , e z ) e y = 0 , 1 , k = 0 , , m , e x = 1 , , m k , e z = m e x k } , B = { ( k , e x , e y , e z ) e y = 0 , , m t , k = 1 t e y , , m , e x = 0 , e z = m k } ,
and each term x y 2 is replaced by w z . The lattice has dimension ω = 40 . Applying the LLL algorithm to the lattice L yields a reduced matrix, from which we derive 40 polynomials. A computation of the Gröbner basis then provides the solution
x 0 = 775498829232538581123082 , y 0 = 3487384624245570654520525920690 , w 0 = 943150161306261052140984582339690768969934167219917544953518167111 1521807051406540201 .
Both the LLL algorithm and Gröbner basis method took less than one second. Notice that α β + 3 γ = 1.93 , and then the bound of Feng et al. [10] is min 2 3 α 3 2 γ , α 3 γ 0.469 , which is lower than our bound α 2 α γ 0.571 .

4. Comparison with the Method of Feng et al. [10]

In 2024, Feng et al. [10] solved the modular equation x ( y 2 + A y + B ) + 1 0   ( mod   e ) . They showed that if | x | N β and | y | N γ , then one can find x and y if
β < α 2 γ α , if β + 2 γ < α < β + 3 γ , min 2 3 α 3 2 γ , α 3 γ , if α β + 3 γ .
In our method, we can solve the equation x ( y 2 + A y + B ) + 1 0   ( mod   e ) whenever α 2 γ and β < α 2 γ α . This shows that the range of α is larger their range. Let us compare the bounds of β . Denote by Δ the difference between our bound and their bound.
If β + 2 γ < α < β + 3 γ , we can retrieve their bound by setting μ = 0 in Theorem 3. This shows that their method is a special case of ours.
If α β + 3 γ , their bound is
min 2 3 α 3 2 γ , α 3 γ = 2 3 α 3 2 γ , if α 9 2 γ , α 3 γ , if α < 9 2 γ .
For α 9 2 γ , a simple calculation yields
Δ = α 2 γ α 2 3 α 3 2 γ = ( 2 α 9 γ ) 2 2 α + 9 γ + 6 2 γ α 0 ,
which shows that our bound is better that their bound.
For α < 9 2 γ , a straightforward computation gives
Δ = α 2 γ α α 3 γ = γ ( 9 γ 2 α ) 3 γ + 2 γ α > 0 ,
which shows again that our bound is better than theirs.

5. Application of the New Method

In this section, we apply Theorem 3 to break RSA variants with a key equation of the form e v x ψ ( N ) = z .
Theorem 4.
Let ( N , e ) be an RSA public key with N = p q , q < p < 2 q , and e = N α . Suppose there are three integers v 0 , x 0 , and z 0 such that | z 0 |   N μ , | v 0 |   N ζ and e v 0 x 0 ψ ( N ) = z 0 . If p 0 is a known approximation of p such that | p p 0 |   = N γ , with 0 γ 1 2 , 2 γ α 2 γ , and ζ + 1 2 μ < 2 2 α γ , then the modulus N can be factored in polynomial time.
Proof. 
Let p 0 be an approximation of p such that | p p 0 |   = N γ . According to Lemma 2, the integer q 0 = N p 0 is an approximation of q such that
| q q 0 |   < N γ , | p + q p 0 q 0 | < 2 N γ .
If N = p q , then the equation e v 0 x 0 ψ ( N ) = z 0 can be rewritten as a modular one, namely, x 0 ( y 0 2 + A y 0 + B ) + z 0 0   ( mod   e ) , where y 0 = p + q p 0 q 0 , A = N + 2 ( p 0 + q 0 ) + 1 , and
B = N 2 + ( p 0 + q 0 1 ) N + ( p 0 + q 0 ) 2 + p 0 + q 0 + 1 .
On the other hand, using ψ ( N ) > N 2 , and assuming that | z 0 |   < e v 0 , we get
x 0 = e v 0 z 0 ψ ( N ) < 2 e v 0 N 2 2 N α + ζ 2 = X .
Also, we have | y 0 |   < 2 N γ = Y . To apply Theorem 3, we need a bound for w 0 = x 0 y 0 2 + z 0 . We assume | z 0 |   < x 0 y 0 2 so that | w | < 2 X Y 2 = W . Then, by applying Theorem 3 for β = α + ζ 2 and γ , the inequality β + 1 2 μ < α 2 α γ leads to ζ + 1 2 μ < 2 2 α γ , and the conditions α 2 γ , 0 ζ < 2 2 α γ turn to 2 γ α 2 γ . After finding the root ( x 0 , y 0 , z 0 , w 0 ) and using the values of N = p q and y 0 + p 0 + q 0 = p + q , we get p and q. This concludes the proof. □

6. Comparison with the Former Attacks

In this section, we compare our attack with the former attacks in terms of the bounds.

6.1. Comparison with the Attack of Zheng et al. [17]

In [17], Zheng et al. proposed an attack targeting the RSA variant introduced by Murru and Saettone [5], which is based on the key equation e d k ψ ( N ) = 1 . They showed that the system is vulnerable when e = N α , d = N δ , and δ < 2 α . This represents the optimal bound for small private exponent attacks on this scheme.
Let Δ denote the difference between the bound established in Theorem 4, and then we have 2 α . Then,
Δ = 2 2 α γ 2 α = α ( 1 2 γ ) .
By fixing γ < 0.5 , it follows that Δ > 0 , confirming that our bound is always better than theirs.

6.2. Comparison with the Attack of Feng et al. [10]

In [10], Feng et al. proposed an attack on the cryptographic scheme introduced by Murru and Saettone [5], refining the earlier results of Zheng et al. [17]. They demonstrated that for N = p q , e = N α , d < N δ , and | p p 0 |   = N γ , where p 0 is an approximation of p, and the scheme becomes vulnerable if δ < 2 2 α γ . Notably, this bound can be obtained by setting μ = 0 in Theorem 4, indicating that Feng et al.’s method is a specific instance of the more general framework presented in our approach.
On the one hand, the number of exponents e of size N α that are vulnerable to this attack can be approximated by
# d d < N 2 2 γ α , gcd ( d , ψ ( N ) ) = 1 = O N 2 2 γ α ε ,
where ε is a small positive value that reflects the exponents that are not coprime with ψ ( N ) .
On the other hand, the number of weak exponents e of size N α relative to our attack is given by
W = # ( v 0 , z 0 ) | | v 0 |   N ζ , | z 0 | N μ , ζ + 1 2 μ < 2 2 γ α , gcd ( v 0 , ψ ( N ) ) = 1 .
This implies that
W # ( v 0 , z 0 ) | | v 0 |   N ζ , | z 0 |   N μ , | v 0 | | z 0 | < N 2 2 γ α , gcd ( v 0 , ψ ( N ) ) = 1 .
Let b = 2 2 γ α . Then, 1   | z 0 |   < N 2 b | v 0 | 2 , with 1   | v 0 |   < N b . The upper bound for this count is obtained by summing the number of possible values of | z 0 | in the interval 1 , N 2 b | v 0 | 2 for each v 0 such that 1   | v 0 |   < N b . This gives
W 4 v 0 = 1 N b z 0 = 1 N 2 b | v 0 | 2 1 = 4 v 0 = 1 N b N 2 b | v 0 | 2 = 4 N 2 b v 0 = 1 N b 1 | v 0 | 2 2 π 2 3 N 2 b ,
where we used the known result s = 1 + 1 s 2 = π 2 6 . Consequently,
W = O N 2 2 2 γ α ε ,
where ε is a small positive constant accounting for the integers v 0 that are not coprime with ψ ( N ) . This bound is significantly larger than the number of exponents e of size N α that are vulnerable to the attack proposed by Feng et al. [10].

7. Experimental Results

In this section, we verify the validity of our proposed attacks. The experiments were carried out employing the HPC-MARWAN cluster [18] using SageMath software (version 10.0). HPC-MARWAN is a cluster dedicated to scientific research, providing computational power and storage capacity, deployed by the National Center for Scientific and Technical Research (CNRST) in Morocco.
We showcase the effectiveness of our method in breaking the RSA variant. We also highlight that our attack retains its efficacy even when both the public exponent e and the private exponent d are approximately of size N 2 , which is a scenario in which existing attacks fail to be effective.

7.1. A Detailed Example for Theorem 4 with the Equation e v x ( y 2 + A y + B ) = z

Here are the following public values to consider: N 2 512 and e 2 1022 . Furthermore, we have
N = 92090123703993789630510505576230736645889095247500414117819966585 56971851097330338536685771902273569360378931731016906043849221198 198036935744123832820589 , e = 38934215691607950469120012361006013254362521225220257539170479082 12083936845833181184061443322817097190960300942509047121150184684 05032721615209801105248073935917468358425919934056305719304476900 13326171658830108432349387168622831806240456963423109366659468641 767246402078467795775932869025626858492401033951 , p 0 = 11579208923731619542357098500868790785326998466564056403945758400 7913129619533 .
This yields e = N α , with α 1.9978 . Additionally, q 0 = N p 0 provides an approximation of q, where
q 0 = 795305830567188711157306392157135729480301169187366271932474635603 00498156651 .
The objective is to find a small solution ( x 0 , y 0 , z 0 ) to the equation
x ( y 2 + A y + B ) + z 0 ( mod   ψ ( N ) ) ,
where the coefficients A and B are given by
A = N + 2 ( p 0 + q 0 ) + 1 , B = N 2 + ( p 0 + q 0 1 ) N + ( p 0 + q 0 ) 2 + p 0 + q 0 + 1 .
Specifically, we have
A = 920901237039937896305105055762307366458890952475004141178199 665855697185109772098388127384203535217260882773469261950624 7017975580663447030880551088372958 , B = 848059088381687885222692986094202987557717535259737888174631 414964151825948608250261552605339973955837573041297651395427 901782819579679100678550541876230524170482254615158647078480 961854145040641977050073093968898196703731531353503540263696 969306755617743680388039859455084301800191045010714420214587 73916749 .
To apply the method outlined in Theorem 4, we assume the following conditions | v 0 |   N ζ , | z 0 |   N μ , and | p p 0 |   N γ , where ζ = 0.4 , μ = 0.3899 , and γ = 0.49179 . It is important to note that these conditions satisfy the requirements of Theorem 4, specifically
2 γ 0.983 < α < 2 γ 4.066 , ζ + 1 2 μ 0.594 < 2 2 α γ 0.598 .
This establishes the bounds
X 2 N α + ζ 2 = 35317824933741584086703883528303714266057154312907285 271150592 , Y 2 N γ = 104493754698738524355519630920748673797643265656422206730 78934100159252922368 , Z N μ = 1073116001210012283097441801606665179596152191451837623173 120 , W = 2 X Y 2 7712667597695774572677877060455265161976005042632523391603 081758469953781216418569262422889415953648852041303652656380831526898 381818448408398069785728319647442138981910138399151181599383434359979 090634271952470016 .
Setting m = 4 and t = 2 , we construct a lattice L of dimension ω = 65 using the previous methodology. By applying the LLL algorithm, we obtain a reduced matrix from which we derive 65 polynomials. Utilizing the Gröbner basis method, we then select the first four polynomials and solve them over the integers. This process yields the following solution
x 0 = 476922631141216258824709909224364166021406637821749589143304 , y 0 = 14656198973036963042431218136491400396244274845042329427279636 15709918784714 , z 0 = 996433282843842715907804162488388224532595860353406044735637 , w 0 = 1024449691435018776340137001955759656771048969519650439490082783 2638023125347661454319655031429861706474386683637844031231494225 2383581431101959647815237784662331111283958734677149072166350844 5598874271464349621 .
Consequently, v 0 can be computed using
v 0 = x 0 ( y 0 2 + A y 0 + B ) + z 0 e , = 1038825528162352588568000862000004781199865632503791922107619 .
Note that the conditions | z 0 |   < e v 0 and | z 0 |   < x 0 y 0 2 are satisfied, as stipulated in Theorem 4. The LLL algorithm and the Gröbner basis method were executed in less than nine seconds. Using N = p q and y 0 + p 0 + q 0 = p + q , we obtain
p = 110571356571418192257697953498072866373701107402166325725750051578 456079806541 , q = 832856958253131779773605489126794743879745666977066325642270323740 47629184929 .
Observe that e is of the form
e z 0 v 0   mod   ψ ( N ) .
Then, we can compute d 1 e   mod   ψ ( N ) , and obtain
d = 1655976650302073678969564407000807245248647342975144716132433455575 6230549465308393591474500785245363456869535980499891838145913408081 2794424287762914612538408390381867384695524863158843802553880691450 9112178358072157830766955945445112918958250946247537278292973203123 5295228023136388116354527659154529687250 .
Hence d = N δ with δ 1.995 .
Notice that the condition established by Feng [10] for breaking the system is given by δ < 2 2 γ α , which represents the optimal bound for partial prime exposure attacks. In this numerical example, we find that 2 2 γ α 0.598 < δ . This shows that their method cannot be broken the system in this particular instance.
Similarly, the optimal bound for small private exponent attacks is given by δ < 2 α , as presented by Zheng et al. [17]. In our example, we find that 2 α 0.586 < δ . This indicates that their method is insufficient to break the system in this instance.

7.2. Experiments for Theorem 4 for Large Public Keys

We implement the method outlined in Theorem 4 using large values for the public key N , e . Through a series of experiments, we solve the equation e v x R ( y ) = z , where
R ( y ) = y 2 + N + 2 ( p 0 + q 0 ) + 1 y + N 2 + ( p 0 + q 0 1 ) N + ( p 0 + q 0 ) 2 + p 0 + q 0 + 1 .
This approach allows us to factor the modulus N = p q efficiently when the most significant bits of p are known. The experiments are summarized in Table 2 where the parameters in each column are defined as follows.
  • nb ( x ) stands for the number of bits of x.
  • ζ is a parameter satisfies | v | N ζ .
  • δ is the parameter for which d = N δ .
  • α is defined by e = N α .
  • μ is a parameter such that | z | N μ .
  • γ is the parameter defined by γ = log | p p 0 | log ( N ) .
  • nbk ( p ) stands for the number of known bits of p.
  • m and t are parameters for constructing the lattice L with dimension ω .
  • Time is specified for the time in seconds required to perform both the LLL algorithm and the Gröbner basis method.
Table 2 presents our experimental results when the most significant bits of the prime factor p of N = p q are known. These results are related to the method of Theorem 4.

8. Conclusions

In this paper, we introduced a novel approach for solving the generalized equation e v x ( y 2 + A y + B ) = z , where x, y, and z are small unknown integers, and A , B Z . Our method builds upon Coppersmith’s technique and leverages lattice basis reduction. Furthermore, we applied this approach to the cryptanalysis of the RSA variants based on the cubic Pell equation, represented as e u x ( p 2 + p + 1 ) ( q 2 + q + 1 ) = z . Notably, our technique enabled the factorization of the RSA modulus N = p q in polynomial time. This research demonstrates that our attack improves upon previous methods such the method of Feng et al. [10] and the method of Zheng et al. [17], targeting small private exponent attacks and partial prime exposure attacks.

Author Contributions

Conceptualization, M.R., A.T. and A.N.; methodology, M.R., A.T. and A.N.; software, M.R.; validation, M.R., A.T., A.N. and M.Z.; formal analysis, M.R. and A.N.; investigation, M.R. and A.N.; resources, M.R. and A.N.; data curation, M.R. and A.N.; writing—original draft preparation, M.R. and A.N.; writing—review and editing, M.R., A.T. and A.N.; visualization, M.R., A.T. and A.N.; supervision, M.R., A.N. and M.Z.; project administration, M.R., A.N., A.T. and M.Z. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Data Availability Statement

Data sharing is not applicable.

Acknowledgments

This work utilized the computational resources of HPC-MARWAN, made available by the National Center for Scientific and Technical Research (CNRST) in Rabat, Morocco.

Conflicts of Interest

The authors declare no conflicts of interest.

Abbreviations

The following abbreviations are used in this manuscript:
RSARivest, Shamir, Adleman
LLLLenstra, Lenstra, and Lovász
CNRSTCentre National de la Recherche Scientifique et Technique
φ ( N ) Euler’s totient function φ ( N ) = ( p 1 ) ( q 1 )
ψ ( N ) cubic totient function ψ ( N ) = ( p 2 + p + 1 ) ( q 2 + q + 1 )

References

  1. Rivest, R.; Shamir, A.; Adleman, L. A Method for Obtaining digital signatures and public-key cryptosystems. Commun. ACM 1978, 21, 120–126. [Google Scholar] [CrossRef]
  2. Wiener, M. Cryptanalysis of short RSA secret exponents. IEEE Trans. Inf. Theory 1990, 36, 553–558. [Google Scholar] [CrossRef]
  3. Boneh, D.; Durfee, G. Cryptanalysis of RSA with private key d less than N0.292. In Advances in Cryptology—Eurocrypt’99; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 1999; Volume 1592, pp. 1–11. [Google Scholar]
  4. Coppersmith, D. Small solutions to polynomial equations, and low exponent RSA vulnerabilities. J. Cryptol. 1997, 10, 233–260. [Google Scholar] [CrossRef]
  5. Murru, N.; Saettone, F.M. A Novel RSA-Like Cryptosystem Based on a Generalization of the Rédei Rational Functions. In Number-Theoretic Methods in Cryptology; Kaczorowski, J., Pieprzyk, J., Pomykala, J., Eds.; NuTMiC 2017; Lecture Notes in Computer Science; Springer: Cham, Switzerland, 2018; Volume 10737, pp. 91–103. [Google Scholar]
  6. Castagnos, G. An efficient probabilistic public-key cryptosystem over quadratic fields quotients. Finite Fields Their Appl. 2007, 13, 563–576. [Google Scholar] [CrossRef]
  7. Elkamchouchi, H.; Elshenawy, K.; Shaban, H. Extended RSA cryptosystem and digital signature schemes in the domain of Gaussian integers. In Proceedings of the 8th International Conference on Communication Systems, ICCS 2002, Singapore, 28 November 2002; IEEE: Piscataway, NJ, USA, 2002; Volume 1, pp. 91–95. [Google Scholar]
  8. Kuwakado, H.; Koyama, K.; Tsuruoka, Y. A New RSA-Type Scheme Based on Singular Cubic Curves with equation y2x3 + bx2 (mod N). IEICE Trans. Fundam. 1995, 78, 27–33. [Google Scholar]
  9. Said, M.R.M.; Loxton, J. A cubic analogue of the RSA cryptosystem. Bull. Aust. Math. Soc. 2003, 68, 21–38. [Google Scholar] [CrossRef]
  10. Feng, Y.; Nitaj, A.; Pan, Y. Partial prime factor exposure attacks on some RSA variants. Theor. Comput. Sci. 2024, 999, 114549. [Google Scholar] [CrossRef]
  11. Nitaj, A. Another generalization of Wiener’s attack on RSA. In Africacrypt 2008; Vaudenay, S., Ed.; LNCS; Springer: Berlin/Heidelberg, Germany, 2008; Volume 5023, pp. 174–190. [Google Scholar]
  12. Lenstra, A.K.; Lenstra, H.W.; Lovász, L. Factoring polynomials with rational coefficients. Math. Ann. 1982, 261, 513–534. [Google Scholar] [CrossRef]
  13. May, A. New RSA Vulnerabilities Using Lattice Reduction Methods. PhD Thesis, University of Paderborn, Paderborn, Germany, 2003. [Google Scholar]
  14. Howgrave-Graham, N. Finding small roots of univariate modular equations revisited. In Proceedings of the IMA International Conference on Cryptography and Coding, Cirencester, UK, 17–19 December 1997; LNCS. Springer: Berlin/Heidelberg, Germany, 1997; Volume 1355, pp. 131–142. [Google Scholar]
  15. Jochemsz, E.; May, A. A strategy for finding roots of multivariate polynomials with new applications in attacking RSA variants. In Proceedings of the ASIACRYPT 2006, Shanghai, China, 3–7 December 2006; LNCS. Springer: Berlin/Heidelberg, Germany, 2006; Volume 4284, pp. 267–282. [Google Scholar]
  16. Peng, L.; Hu, L.; Lu, Y.; Wei, H. An improved analysis on three variants of the RSA cryptosystem. In Proceedings of the International Conference on Information Security and Cryptology, Beijing, China, 4–6 November 2016; Springer: Berlin/Heidelberg, Germany, 2016; Volume 10143, pp. 140–149. [Google Scholar]
  17. Zheng, M.; Kunihiro, N.; Yao, Y. Cryptanalysis of the RSA variant based on cubic Pell equation. Theor. Comput. Sci. 2021, 889, 135–144. [Google Scholar] [CrossRef]
  18. HPC-MARWAN, National Center for Scientific and Technical Research (CNRST), Rabat, Morocco. Available online: http://hpc.marwan.ma/index.php/en/ (accessed on 5 May 2025).
Table 1. The lattice basis matrix for ( m , t ) = ( 2 , 1 ) , where a ★ represents an nonzero entry.
Table 1. The lattice basis matrix for ( m , t ) = ( 2 , 1 ) , where a ★ represents an nonzero entry.
H k , e x , e y , e z z 2 xz xyz x 2 x 2 y zw yzw xw xyw w 2 yw 2 y 2 w 2
H 0 , 0 , 0 , 2 e 2 Z 2 00000000000
H 0 , 1 , 0 , 1 0 e 2 X Z 0000000000
H 0 , 1 , 1 , 1 00 e 2 X Y Z 000000000
H 0 , 2 , 0 , 0 000 e 2 X 2 00000000
H 0 , 2 , 1 , 0 0000 e 2 X 2 Y 0000000
H 1 , 0 , 0 , 1 000 e Z W 000000
H 1 , 0 , 1 , 1 000 e Y Z W 00000
H 1 , 1 , 0 , 0 00000 e X W 0000
H 1 , 1 , 1 , 0 00000 e X Y W 000
H 2 , 0 , 0 , 0 0000 W 2 00
H 2 , 0 , 1 , 0 000 Y W 2 0
H 2 , 0 , 2 , 0 00 Y 2 W 2
Table 2. Experimental results by exposing the most significant bits of p.
Table 2. Experimental results by exposing the most significant bits of p.
nb (N)nb (e) ζ δ α μ γ nbk (p)mt ω Time (s)
70013990.356572.0001.99930.3900.4967650426513.48
80015980.362851.9941.99960.3900.4971088426516.59
89917980.377161.9991.99910.3900.49734138426518.47
100019990.369421.9981.99880.3900.49558250426522.04
129925980.345972.0002.00000.3900.49879177426535.99
149929960.373371.9991.99920.3900.49986250426542.20
199939970.349991.9991.99930.3900.49904270335209.06
249949970.367882.0001.99970.3900.49948282335212.59
300059990.366502.0001.99980.3900.49895532335217.37
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Rahmani, M.; Nitaj, A.; Tadmori, A.; Ziane, M. An Improved Attack on the RSA Variant Based on Cubic Pell Equation. Cryptography 2025, 9, 40. https://doi.org/10.3390/cryptography9020040

AMA Style

Rahmani M, Nitaj A, Tadmori A, Ziane M. An Improved Attack on the RSA Variant Based on Cubic Pell Equation. Cryptography. 2025; 9(2):40. https://doi.org/10.3390/cryptography9020040

Chicago/Turabian Style

Rahmani, Mohammed, Abderrahmane Nitaj, Abdelhamid Tadmori, and Mhammed Ziane. 2025. "An Improved Attack on the RSA Variant Based on Cubic Pell Equation" Cryptography 9, no. 2: 40. https://doi.org/10.3390/cryptography9020040

APA Style

Rahmani, M., Nitaj, A., Tadmori, A., & Ziane, M. (2025). An Improved Attack on the RSA Variant Based on Cubic Pell Equation. Cryptography, 9(2), 40. https://doi.org/10.3390/cryptography9020040

Article Metrics

Back to TopTop