Security and Performance Analyses of Post-Quantum Digital Signature Algorithms and Their TLS and PKI Integrations ^†

Abstract

Quantum computing challenges the mathematical problems anchoring the security of the classical public key algorithms. For quantum-resistant public key algorithms, the National Institute of Standards and Technology (NIST) has undergone a multi-year standardization process and selected the post-quantum cryptography (PQC) public key digital signatures of Dilithium, Falcon, and SPHINCS⁺. Finding common ground to compare these algorithms can be difficult because of their design differences, including the fundamental math problems (lattice-based vs. hash-based). We use a visualization model to show the key/signature size vs. security trade-offs for all PQC algorithms. Our performance analyses compare the algorithms’ computational loads in the execution time. Building on the individual algorithms’ analyses, we analyze the communication costs and implementation overheads when integrated with Public Key Infrastructure (PKI) and with Transport Layer Security (TLS) and Transmission Control Protocol (TCP)/Internet Protocol (IP). Our results show that the lattice-based algorithms of Dilithium and Falcon induce lower computational overheads than the hash-based algorithms of SPHINCS⁺. In addition, the lattice-based PQC can outperform the classical algorithm with comparable security strength; for example, Dilithium 2 and Falcon 512 outperform RSA 4096 in the TLS handshake time duration.

1. Introduction

Public key digital signatures provide authentication and integrity protection and are critical in securing digital systems. The security of the most-used present-day digital signature standards like Rivest–Shamir–Adleman (RSA) [1] and the Elliptic Curve Digital Signature Algorithm (ECDSA) [2] are based on the computational hardness problems such as prime factorization and the discrete logarithm problem. Shor’s polynomial-time algorithm effectively solves these problems when equipped with a powerful quantum computer [3]. Securing digital communication against attackers that have access to quantum computing resources [4] requires new public key cryptographic algorithms that can withstand such quantum attackers.

Recent advancements in quantum computing and quantum computers (Section 3) yield a need for transitioning to post-quantum cryptography (quantum-resistant cryptography). This involves identifying the relevant hardness problems and designing and constructing quantum-resistant algorithms for securing digital communications.

The National Institute of Standards and Technology (NIST) launched the Post-Quantum Cryptography (PQC) standardization project to establish and standardize quantum-resistant algorithms. The NIST has a track record of preparing for the impending cryptanalysis and breaks on cryptographic algorithms and is preparing for the post-quantum era before the emergence of practical quantum computer implementations capable of breaking current systems. The NIST’s involvement in cryptography has global and lasting impacts on digital systems, as demonstrated by their involvement in standardizing DES in the 1970s and AES in the late 1990s. The standardization process starts with an open public call that lists the requirements of the algorithms. All the submission algorithms are openly published and subjected to analyses, and, after the analyses, an algorithm is selected for standardization. The PQC standardization project follows the same process and requests that the interested parties provide submissions for quantum-resistant cryptographic candidates.

We study the security and performances of the NIST digital signature algorithms selected for standardization. The selected three digital signature algorithms are Crystals–Dilithium (Dilithium), Falcon, and SPHINCS⁺. The design principles and hardness problems underlying these digital signature algorithms come from lattice-based and hash-based cryptography (Section 4).

Contributions and Paper Organization

We compare the PQC digital signature schemes in both security and performance. We first focus on the algorithms before their integration into the TCP/TLS protocol and Public Key Infrastructure (PKI). Our security analyses of the PQC algorithms include the adoption of a model for visual representations to compare the size–security trade-offs of digital signature algorithms (Section 5). These include the security offered by the digital signature algorithms with respect to public key length and signature length. We then analyze the individual algorithm performances in the execution time and its scalability in message sizes for key-pair generation, signature generation, and signature verification (Section 6). Our work additionally extends beyond the analyses of the individual algorithms and to the integration into the TCP/TLS protocol and PKI. We analyze the computational overheads involved in integration with PKI in terms of certificate generation and certificate verification times (Section 7) and the communication/handshake overheads for TLS 1.3 and TCP/IPv4 connection when integrating the signature algorithms (Section 8). Our analyses show that, with comparable security strengths, the lattice-based PQC algorithms (Dilithium and Falcon) can even outperform the classical RSA algorithm, while the hash-based SPHINCS⁺ generally takes longer than Dilithium and Falcon.

2. Materials and Methods

Methodologies and Approaches

For the performance analyses of the PQC schemes, we implement prototypes and empirically measure performances between the schemes. While the post-quantum era prepares for adversaries equipped with quantum computing, the PQC schemes are designed to be implemented on classical computers to defend common users. We build on the algorithm implementations from liboqs and openssl by Open Quantum Safe [5] and deploy them on a virtual machine with 4 processing cores and 8 GB of RAM on a computer equipped with an 8-core 16-thread AMD Ryzen 7 1700X processor with a 3.4 GHz processor frequency and 32 GB RAM. While the absolute performance costs vary depending on the computer platform and its hardware specifications, we focus on the comparison results in this paper so that our results and insights are applicable to classical non-quantum computers beyond our platform. We run each experiment to collect 1000 data values and use the average value for our analysis. We compare the performance costs of the algorithms by themselves in Section 6, when integrated into PKI in Section 7, and when integrated with TLS 1.3 and TCP/IP in Section 8.

3. Background: PQC History and NIST

We describe the PQC history and the NIST involvement to motivate our work in this section, and Figure 1 shows the timeline of events regarding cryptography and quantum computing. Cryptography provides the backbone to secure digital systems in our society. While practical quantum computers currently only support a small number of qubits and remain at proof-of-concept stages, theoretical algorithms building on quantum computing have emerged to expedite the solving of computational hardness problems anchoring the security of current public–key algorithms. Shor’s algorithm [3], invented in 1994, provides a polynomial-time algorithm in quantum computing for solving the prime factorization and discrete log problem, threatening the security of public key algorithms such as RSA and Diffie–Hellman Key Exchange. Grover’s algorithm [6], invented in 1996, expedites the brute-force search of the general problems, such as the original database search problem and hash collision finding. The first successful implementation of quantum searching was performed, in 1998, on a two-qubit Nuclear Magnetic Resonance (NMR) quantum computer. It used Grover’s algorithm to search for a system that has four states [7]. Followed by Grover’s implementation, in 2001, a seven-qubit NMR quantum computer [8] used Shor’s algorithm to find prime factors for the number 15.

Inspired by the extraordinary opportunities in quantum computing, major tech giants started research into quantum computers. The first was D-Wave Systems, using quantum annealing concepts, which cannot run Shor’s algorithm and is built for specific applications like combinatorial optimization problems. In 2019, D-Wave unveiled a 5000 qubit processor [9]. On the other hand, Google and International Business Machines (IBM) follow the universal quantum gate model, which can run Shor’s algorithm. In 2018, Google announced a 72-qubit quantum processor [10]. IBM showed significant progress from a 5-qubit processor [11] to a 127-qubit processor and announced details on futuristic 433- and 1121-qubit processors. IBM’s quantum experience provides access to up to 32-qubit processor quantum computers for registered users free of cost [12]. With steady growth in practical quantum computers and raising concerns in cryptography, the NIST initiated the PQC standardization project. The NIST requested nominations in December 2016 for public key post-quantum cryptographic algorithms [13]. The NIST’s PQC standardization project aims to replace the present recommended digital signature standards of RSA and ECDSA [14] with post-quantum algorithms. Instead of measuring the security in bits, all these post-quantum algorithms are referenced with security categories defined by the NIST (

Table 1. NIST security categories, where X is the MAXDEPTH.

Security Category	Reference Algorithm	Classical Bit Cost	Qubit Security	Circuit Size to Break the Algorithm [13]
1	AES 128	128 (key search)	64 (Grover [6])	$2^{170}/X$ quantum gates or $2^{143}$ classical gates
2	SHA3-256	128 (collision)	85 (Brassard [15])	$2^{146}$ classical gates
3	AES 192	192 (key search)	96 (Grover [6])	$2^{233}/X$ quantum gates or $2^{207}$ classical gates
4	SHA3-384	192 (collsion)	128 (Brassard [15])	$2^{210}$ classical gates
5	AES 256	256 (key search)	128 (Grover [6])	$2^{298}/X$ quantum gates or $2^{272}$ classical gates
6	SHA3-512	256 (collision)	170 (Brassard [15])	$2^{274}$ classical gates

). Each of these categories sets the minimum required computational resources to break well-known symmetric block cipher or hash functions. Breaking a symmetric block cipher indicates a successful brute-force key search attack, and breaking a hash function means a successful brute-force collision attack. Computational resources can be restricted by a new parameter, defined by NIST, called MAXDEPTH. It can be used to limit the quantum attacks to a fixed run time or circuit depth. PQC standardization is a multi-year process and involves multiple rounds of analyses and scrutiny for the maturity of algorithm design before standardization.

In response to the open public call for the PQC standardization proposal [13], the NIST received 82 submissions, and only 69 of the submissions satisfied the minimum required conditions. The first round of the PQC standardization project started in December 2017 and selected 26 candidate algorithms. With fewer algorithms to analyze, the second round of the PQC standardization project started in January 2019 and selected 15 algorithms for the third round [16]. In July 2022, the NIST announced the end of the third round and selected candidates for standardization. A public key encryption/KEM and three digital signatures were selected for standardization. In August 2024, the NIST released three standards including (i) KEM: FIPS 203 based on Kyber, and (ii) digital signatures: FIPS 204 based on Dilithium and 205 based on SPHINCS+ for public comments. The NIST has yet to release FIPS 206 based on FALCON. The PQC standardization effort has continued to round 4 for identifying additional KEM standards. In September 2022, the NIST started a new search for digital signatures and released an additional call for proposals. This effort is currently in its round 2 and is analyzing fourteen candidates for digital signatures [17]. Notable exceptions of the NIST finalist algorithm family candidates from the third round that did not get selected were the multivariate-based algorithms. While surviving the first three rounds, during the third round, the key-recovery attack [18], rectangular MinRank attack [19], and hybrid combinatorial/algebraic attack [20] were discovered as being able to break the multivariate-based mathematical problem in polynomial time (e.g., Rainbow—I got broken and the key compromised in days).

We focus on the digital signature algorithms in this paper, while the NIST solicits and plans to standardize both key exchange/key encapsulation mechanisms and digital signature algorithms. Our work focuses on the NIST-selected signature schemes/algorithms: Dilithium, Falcon, and SPHINCS⁺.

4. PQC Signature Schemes

The NIST-selected digital signature schemes for standardization are Dilithium, Falcon, and SPHINCS⁺. These families of algorithms can be categorized into two different schemes, listed in

Table 2. NIST-selected digital signature algorithms and respective parameter sets with security levels.

Algorithm	Scheme	Parameters	Security Level
Dilithium	Lattice-based	Dilithium 2	1
		Dilithium 3	3
		Dilithium 5	5
Falcon	Lattice-based	Falcon 512	1
		Falcon 1024	5
SPHINCS+	Hash-based	SPHINCS+-128s SPHINCS+-128f	1
		SPHINCS+-192s SPHINCS+-192f	3
		SPHINCS+-256s SPHINCS+-256f	5

, based on the hardness problems on which they rely. These include lattice-based signature schemes (Dilithium and Falcon) described in Section 4.1 and hash-based signature schemes (SPHINCS⁺) described in Section 4.2.

Within each algorithm family (Dilithium, Falcon, and SPHINCS⁺), there are multiple algorithms depending on the parameter choices for the security level control. The parameters affect the public key length and the private-key length as well as the signature length for the PQC signature algorithms, and the security strengths increase as the length increase (using greater number/options to yield greater entropy), as we study in Section 5. The public key and the signature lengths for the different algorithms are also listed in the horizontal axis in Figure 2. For example, Dilithium 2 has a public key length of 1312 Bytes, Dilithium 3 with 1952 Bytes, and Dilithium 5 with 2592 Bytes. Thus, Dilithium 5 is designed to have greater security strength than Dilithium 3, and Dilithium 3 greater than Dilithium 2. This section describes the overview of the PQC schemes without the scheme-specific details, such as the actual parameters/variables to control for the different algorithms within the family; we refer readers interested in the scheme-specific design details to the design documents for Dilithium [21], Falcon [22], and SPHINCS⁺ [23].

4.1. Lattice-Based Signature Schemes

Lattice-based signature schemes are based on a set of points in an n-dimensional space with a periodic structure [24]. The security of lattice-based cryptography comes from the use of NP-hard problems such as (i) Short Vector Problems (SVPs), which involve finding the shortest non-zero vector, (ii) Closed Vector Problems (CVPs), which involve finding the shortest vector, (iii) Learning With Errors (LWE), which is computationally intensive as it requires a linear function over a finite ring of given samples, (iv) Short Integer Solutions (SISs), which are based on Ajtai’s theorem, where if polynomial-time algorithm A solves the SIS problem, then there exists an Algorithm B that can solve the Short Vector Integer Problem (SVIP), (v) Learning With Rounding (LWR), which is the non-rounding variant of LWE. LWR is more efficient than LWE as it removes LWE’s complex randomization elements [25]. Both Dilithium and Falcon are lattice-based algorithms selected for standardization.

4.1.1. Dilithium

Dilithium relies on the Fiat–Shamir and Aborts framework and SVPs for its security [21]. Dilithium introduces three algorithms (Table 2) in Dilithium 2, Dilithium 3, and Dilithium 5 corresponding to 2, 3, and 5 of the NIST’s post-quantum security categories, respectively.

4.1.2. Falcon

Falcon is based on fast Fourier lattice-based compact signatures over an N-th Degree Truncated Polynomial Ring (NTRU) [22]. Falcon relies on the NTRU for key generation, encryption and decryption data, Short Integer Problems (SISs), Floating-Point arithmetic, and Gaussian sampling floating-point arithmetic for its security. Falcon parameter sets (Table 2) include Falcon 512 and Falcon 1024 algorithms corresponding to 1 and 5 of the NIST’s post-quantum security strengths, respectively.

4.2. Hash-Based Signature Schemes

Hash-based signatures are based on one-time signatures [26] or many-time signatures [27]. Their security relies on the collision and preimage resistance of the underlying hash function. Previously available standards like LMS and XMSS are stateful where the state of the private key needs to be remembered for single use and does not meet ideal digital signature requirements [13]. SPHINCS⁺ is stateless and builds on both one-time signature schemes as well as few-time signature schemes.

SPHINCS⁺

SPHINCS⁺ uses Winternitz One-Time Signature Plus (WOTS⁺) and a Forest of Random Subsets (FORS) for signature generation. SPHINCS⁺ parameter sets are designed to be small/size-optimized (s) or fast/speed-optimized (f). Each of the SPHINCS⁺ parameter sets in Table 2 has 3 more variants based on the underlying hash function. For example, SPHINCS⁺128s can be based on the SHA-256 (SPHINCS⁺-SHA-256-128s) or SHAKE256 (SPHINCS⁺-SHAKE256-128s) or Haraka (SPHINCS⁺-Haraka-128s) hash functions. Each of these can again be sub-categorized based on the type of construction as ’robust’, where XORs message with pseudo-random bitmasks, or ’simple’, where bitmasks are omitted [23]. For example, SPHINCS⁺-SHA-256-128s is available as SPHINCS⁺-SHA-256-128s-robust and SPHINCS⁺-SHA-256-128s-simple.

5. Security Analysis Using Visualization Model

In this section, we analyze the size–security trade-offs of the NIST-selected signature schemes using the attacker’s security cost in qubits. The metric qubit used to measure the security strength is relatively new and depends on the quantum computer physics and hardware architecture, which are in active research and development.

5.1. Metrics: Qubits

In classical computing, the state of a particular bit is always known. In the quantum case, before the measurement is carried out, the state of a qubit is unknown. Although a qubit resembles a classical bit after the measurement, it takes two possible values and additionally can exploit the interference effects; before the measurement, a qubit can be in a superposition of these two states described by the wave function. The classical brute-force attacker searches blindly and cannot distinguish if some particular value is closer or further from the key being searched, while checking if the key is found may only generate a binary result of “true” or “false”. In the quantum case, a brute-force attacker always has an overlap of the wave function of the current state of qubits and the key, and this overlap is bigger if some particular state of qubits is closer to the key being searched. Even if the algorithm’s structure is unknown to the attacker, interference helps to move in the direction towards the key, enabling a faster search.

Qubit Cost

Qubit cost indicates the number of qubits the attacker needs to break the algorithm assuming a sufficient number of gates. For Dilithium, qubit cost represents the cost of solving an SVP, which is exponential relative to Block–Korkine–Zolotarev (BKZ) algorithm’s block-size (b) [21]. For Falcon, core SVP hardness indicates the cost for one call to the SVP oracle in dimension b [22,28].

5.2. Lattice-Based Signatures Security and Length Trade-Off

Figure 2 shows the classical and quantum security cost trade-offs with respect to the public key length and the signature length of lattice-based signature algorithms. Public key length and signature length are important parameters controlling the trade-off between security strength and communication overhead in digital communications. For example, when Alice sends a signed message to Bob, Bob uses Alice’s public key and signature to verify the message. The public key and signature are the overheads in the communication. Many real-world applications use the same private/public key-pair to sign/verify multiple messages rather than using One Time Pad/Ephemeral Keys, resulting in the transfer of signatures more often than public keys. A low signature size and low public key lengths help in reducing communication costs. In Figure 2, an ideal algorithm will have a small communication cost (left in the plot) and provide strong security (top).

Dilithium and Falcon, the two lattice-based NIST finalists schemes, display the security vs. overhead/length trade-off, and the security costs measured in (classical) bits or qubits increase as the public key length or the signature length increases. Figure 2a analyzes the classical security cost to public key length trade-offs. With 123 and 120 bit costs targeting level 1 security, Dilithium’s public key size is 46.26% bigger than Falcon. Falcon at 1793 bytes offers level 5 security with 91 bits more security and has an 8.14% smaller public key size than Dilithium, offering level 3 security at 1952 bytes. Falcon with 273 bit cost targeting level 5 security has a 30.82% smaller public key size than Dilithium with 252 bit cost targeting level 5 security. As discussed before, low signature sizes are preferable and from Figure 2b,d it is evident that Falcon has low signature sizes compared to Dilithium for both classical and quantum security levels. Figure 2b analyzes the classical security cost to signature length trade-offs. With 123 and 120 bit costs targeting level 1 security, Dilithium’s signature size is 263.36% bigger than Falcon. Falcon with a 273 bit cost targeting level 5 security has a 72.14% smaller signature size than Dilithium with a 252 bit cost targeting level 5 security. Figure 2c analyzes the quantum security cost to public key length trade-offs. Falcon at a 248 qubit cost targeting level 5 security has a 99.88% bigger signature size than its level 1 parameter set with a 108 qubit cost.

For applications targeting security levels 2 and 3, Falcon does not have any parameter sets and yet it is advantageous to use the Falcon 1024 parameter set as it provides better security and less overhead compared to Dilithium parameter sets.

5.3. Hash-Based Signatures Security and Length Trade-Off

Figure 3 shows the classical and quantum security trade-offs with respect to the public key length and the signature length of the hash-based signature algorithm SPHINCS⁺. Hash-based signature algorithms generate smaller public keys and larger signatures compared to the lattice-based schemes in Section 5.2.

Figure 3a analyzes the classical bit cost to public key length trade-offs. Both SPHINCS⁺s and SPHINCS⁺f have the same bit cost for the same public key sizes. This is due to the fact that in the classical case, the underlying hash function determines the security costs of the algorithm. Figure 3b analyzes the classical bit cost to signature length trade-offs. At a 128 bit cost, SPHINCS⁺s generates a 54.02% smaller signature than SPHINCS⁺f. As the security costs increases, this difference remains significant and is 40.24% at a 256 bit cost. Figure 3c analyzes the quantum bit cost to public key length trade-offs. For the same public key lengths of 32 bytes, SPHINCS⁺s provides 3.9% more security than SPHINCS⁺f. Figure 3d analyzes the quantum bit cost to signature length trade-offs. SPHINCS⁺f with a 1 qubit cost more than SPHINCS⁺s at 193 produces 119% more in signature length. Overall, SPHINCS⁺s parameter sets with smaller signature lengths are preferable over SPHINCS⁺f parameter sets.

6. Performance Analysis of PQC Digital Signature Algorithms

In this section, we analyze the execution times of each selected algorithm for key-pair generation, signature generation (signing), and signature verification (verifying). We use the liboqs version 0.7.2 library to analyze the performance of the algorithms. Using our benchmark software, each of the algorithms is sampled to calculate the average time duration for key-pair generation, signing a message, and verifying the messages. We also vary the message lengths with random data to measure the algorithm performances with respect to message scalability. Each data point is averaged over 1000 runs, and the results are plotted.

6.1. Algorithm Performances

Figure 4 shows the computing costs for every algorithm of each family. While we experiment with all algorithms, our presentation/plot focuses on the slowest from each algorithm family (outlined bar) and the fastest from each algorithm family to show the performance span across Dilithium (D), Falcon (F), and SPHINCS⁺ (S).

We analyze the performances within each PQC family of algorithms in order to compare the performances between the algorithms and their respective parameter sets. We introduce the intra-family performance ratio, $R_i$ where i specifies the algorithm family, i.e., $i\in \{D,F,S\}$. For example, $R_D$ is the ratio between the execution time for the slowest Dilithium algorithm and the time for the quickest Dilithium algorithm. By definition, $R_i>1,\forall i$. By dividing the slowest algorithm’s duration with the fastest for each family of algorithms, the key generation data (Figure 4a) shows that $R_D=1.9$, $R_F=2.74$, and $R_S=864.61$. For the message signing phase (Figure 4b), $R_D=1.73$, $R_F=1.71$, and $R_S=385.73$. For message verification (Figure 4c), $R_D=2.8$, $R_F=2$, and $R_S=141.66$. This makes it clear that Dilithium, when compared to Falcon and SPHINCS⁺, shows optimal behavior with regards to the time taken to run the algorithms from each family.

While the intra-family comparison analysis helps in understanding how each algorithm in a family performs independently, an inter-family comparison analysis provides additional insights when comparing the algorithms from other families. For key generation, our data shows that Dilithium is at least 43.68 times faster than Falcon and 1.78 times faster than SPHINCS⁺ algorithms. Similar to key generation, Dilithium outperforms Falcon by at least 1.27 times and SPHINCS⁺ by 20.72 times in message signing. For message verification, both Dilithium and Falcon parameter sets targeting level 1 security are the most effective with the same verification time, which is 2.4 times faster than the SPHINCS⁺ parameter set targeting the same security level. Falcons’ slowest algorithm (Falcon 1024) is 1.4 times faster than Dilithium’s slowest algorithm (Dilithium 5). Dilithium 5 and Falcon 1024 are 121.42 and 170 times faster than the slowest SPHINCS⁺ algorithm. From our intra-family and inter-family comparisons, we conclude that lattice-based algorithms are efficient compared to hash-based algorithms in key generation, message signing, and message verification. Figure 5 plots all the parameter sets used in our analysis. It focuses on the signing vs. verifying trade-off and an ideal algorithm with the lowest overheads is positioned at the left bottom of the plot. Figure 5 shows that lattice-based algorithms are closer to this point than hash-based algorithms. In hash-based algorithms, SPHINCS⁺ algorithms with Haraka as the underlying hash function are closer to the ideal position. Overall, Dilithium is the fastest in execution times across the NIST-selected digital signature algorithms. We extend the analysis to algorithm scalability by using variable message lengths to compare how the performance changes when the input size increases.

6.2. Message Scalability Performances

We measure the signing and verifying performances of the PQC algorithms when the message length varies from 1 Byte to 100 MB to analyze the message scalability. Using our results in Figure 6, the dots represent our empirically measured data values while the line estimates the polynomial line of best fit (LoBF) based on minimizing the mean squared error to enable greater precision in our analysis.

Table 3. The line of best fit (LoBF) estimations for the signing time and verifying time in milliseconds, where x is the message length in Bytes. Table lists the default enabled SPHINCS⁺-128f-robust variant algorithms of SPHINCS⁺ only.

Algorithm	LoBF (Signing Time)	LoBF (Verifying Time)
Dilithium 2	$2.275\times {10}^{-15}{x}^2+5.045\times {10}^{-6}x+0.1974$	$2.294\times {10}^{-15}{x}^2+5.043\times {10}^{-6}x+0.05107$
Dilithium 3	$2.262\times {10}^{-15}{x}^2+5.037\times {10}^{-6}x+0.2897$	$2.276\times {10}^{-15}{x}^2+5.035\times {10}^{-6}x+0.08682$
Dilithium 5	$2.217\times {10}^{-15}{x}^2+5.045\times {10}^{-6}x+0.3375$	$2.170\times {10}^{-15}{x}^2+5.041\times {10}^{-6}x+0.1377$
Falcon 512	$2.276\times {10}^{-15}{x}^2+5.039\times {10}^{-6}x+0.4087$	$2.230\times {10}^{-15}{x}^2+5.036\times {10}^{-6}x+0.05265$
Falcon 1024	$2.207\times {10}^{-15}{x}^2+5.039\times {10}^{-6}x+0.698$	$2.237\times {10}^{-15}{x}^2+5.035\times {10}^{-6}x+0.104$
SPHINCS+-Haraka	$2.260\times {10}^{-15}{x}^2+2.752\times {10}^{-6}x+7.664$	$2.286\times {10}^{-15}{x}^2+1.396\times {10}^{-6}x+0.448$
SPHINCS+-SHA-256	$2.207\times {10}^{-15}{x}^2+1.213\times {10}^{-6}x+36.93$	$2.196\times {10}^{-15}{x}^2+6.317\times {10}^{-7}x+2.847$
SPHINCS+-SHAKE256	$5.689\times {10}^{-17}{x}^2+1.029\times {10}^{-5}x+83.9$	$1.831\times {10}^{-15}{x}^2+5.122\times {10}^{-6}x+10.93$

includes the LoBF estimations in their equation form, and Figure 6 includes the plot for each equation.

From this data, we find that the fastest algorithm for message signing is Dilithium 2 when message sizes are less than 3.256 MB. For message sizes greater than 3.256 MB but less than 19 MB, we find that SPHINCS⁺-Haraka-128f-robust yields the fastest signing time, and for message sizes greater than 19 MB, SPHINCS⁺-SHA-256-128f-robust is the fastest at signing. For verification, we find that Dilithium 2 is similar to Falcon 512, with Dilithium 2 being just 0.1339 ms faster, on average. For message sizes less than 0.1088 MB, Dilithium 2 is fastest, but for message sizes greater than 0.1088 MB and less than 3.138 MB, we find that SPHINCS⁺-Haraka-128f-robust has the fastest verification, and for message sizes greater than 3.138, SPHINCS⁺-SHA-256-128f-robust is fastest.

6.3. Application Dependency

The choice of the PQC algorithm depends on its application since the application determines the usage frequencies of the signing vs. verifying and the message payload size. The frequency discrepancy between signing and verifying a message varies significantly according to the application. For example, in cryptocurrency applications, once a transaction is created, it gets signed a single time. In contrast, as the transaction propagates across the network, every node verifies that the message is genuine [29], thus the signing-to-verifying ratio is close to zero in this case. We aim to prioritize the algorithm with fast message verification. In addition, the message payload size provided by the application requirement affects the performance prioritization and the algorithm selection. Our results show that Dilithium 2 is the most execution-time-efficient if the application message length is short and SPHINCS⁺-Haraka-128f-robust and SPHINCS⁺-SHA-256-128f-robust are the fastest for verifying when the message lengths are larger.

7. Performance Analysis of Integration with Public Key Infrastructure

Public key infrastructure (PKI) is critical in digital networking systems since it provides trust in the public key, which can be used for security protocols or for establishing symmetric keys. Because of its importance for digital and internet networking, we study the effects of transitioning to post-quantum algorithms in PKI and how it affects the performance of digital networking systems. In this section, we focus on the PKI integration of the post-quantum algorithms and conduct an empirical implementation-based study for our analyses of the NIST post-quantum algorithms. More specifically, we study and analyze the feasibility and the performances of the NIST PQC standardization project’s digital signature algorithms when integrated into PKI. Our work focuses on computational efficiency for the following reasons. First, transitioning to post-quantum PKI implementations affects computing the most. Second, we generalize the computational overheads caused by the post-quantum algorithms across the PKI architecture and implementation scenarios.

Figure 7 shows the PKI certification process. In Step 1, the Key-Pair Generation for the public–private key-pair is achieved locally for the subject end-entity, End-Entity 1. The private key does not leave the local node, while the public key is shared with the others, including the CA for certification and later End-Entity 2 for communicating data. In Step 2, End-Entity 1 creates a Certificate Signing Request (CSR) using the public key and sends it to the CA in Step 3. In Step 4, the CA verifies the identity of End-Entity 1 and generates a certificate in Step 5. This certificate contains the public key of End-Entity 1 and a signature generated by the CA using its private key. In Step 6, the certificate is transferred to End-Entity 1. In Step 7, End-Entity 1 verifies and configures the certificate for future communications. In Step 8, the certificate is transferred from End-Entity 1 to End-Entity 2 for authentication. End-Entity 2 obtains the CA and End-Entity 1 certificates in Steps 9 and 10, respectively, for verification. In Step 11, End-Entity 2 verifies the certificate of End-Entity 1. This verification includes the use of the CA’s public key from the CA certificate to verify the signature on the End-Entity 1 certificate. Once verification is successful, a session is established between End-Entity 1 and End-Entity 2 for communicating data.

The functions described in this section and shown in Figure 7 are the essential functions for the PKI process. In addition to its popular applications to provide the root of trust for authenticating web servers for more conventional internet applications, PKI and digital certificate management are applied for ad hoc networking where the data networking between the end-entities does not go through traditional internet core domain networking, e.g., a Security Credential Management System (SCMS) for vehicular/V2X/VANET networking [30,31,32]. Our work is applicable in such ad hoc networking contexts in principle because the PKI designs for such applications are often more complex due to the requirements of the end-entities and the additional security/privacy objectives, which build on the PKI functions targeted for our analyses.

Figure 8a plots the average time taken for certificate generation using classical RSA and NIST-selected post-quantum digital signature algorithms. All Dilithium and Falcon algorithms outperform RSA in certificate generation times. Dilithium 2 and Falcon 512 are 18.98% and 16.24% faster than RSA 2048, while they are 65.56% and 64.39% faster than RSA 4096, respectively. Dilithium 2 is the fastest post-quantum algorithm and is 3.27% faster than Falcon 512, which is the second fastest post-quantum algorithm. At security level 5, Dilithium 5 is 7.32% faster than Falcon 1024. Hash-based SPHINCS⁺ algorithms take longer time than any other classical or post-quantum algorithms. SPHINCS⁺-Haraka-128f-robust takes 2.9%, 198.95%, and 189.16% more certificate generation time than RSA 4096, Dilithium 2, and Falcon 512, respectively. SPHINCS⁺-Haraka-128f-robust is the most efficient in the SPHINCS⁺ family and is 72.39% and 87.21% faster than SPHINCS⁺-SHA-256-128f-robust and SPHINCS⁺-SHAKE256-128f-robust, respectively. Overall, Dilithium algorithms are the most effective in terms of certificate generation time.

Figure 8b plots the average time taken for certificate verification using classical RSA and NIST-selected post-quantum digital signature algorithms. In contrast to the certificate generation times, classical RSA is the most effective in terms of certification verification times. SPHINCS⁺ algorithms continue to remain the heaviest in terms of certification verification times, too. Dilithium 2 and Falcon 512 take 25% and 37.29% longer than RSA 2048, while they take 15.96% and 27.37% longer than RSA 4096, respectively. Dilithium 2 is 15% and 18.23% faster than Dilithium 3 and Dilithium 5, respectively. Falcon 1024 takes 5.37% longer than Falcon 512 for the certificate verification. Similar to certificate generation, with certificate verification, SPHINCS⁺-Haraka-128f-robust is the most efficient in the SPHINCS⁺ family and is 53.78% and 80.64% faster than SPHINCS⁺-SHA-256-128f-robust and SPHINCS⁺-SHAKE256-128f-robust, respectively. Overall, post-quantum algorithms increase the certificate verification times compared to classical RSA. Lattice-based Dilithium and Falcon are efficient compared to the hash-based SPHINCS⁺ algorithms.

8. Performance Analysis of Integration with TCP/IP and TLS

In this section, we analyze the communication/handshake overhead of the PQC algorithms at the packet level when integrated with TLS 1.3 and TCP/IPv4. We focus on handshake duration time because PQC authentication impacts TLS 1.3 only during the handshake process and helps in deriving session keys that are used for transferring the payload. The handshake connection involves multiple transmissions between the client and the server, where the client initiates the connection by sending a client hello packet to the server and the server responds with server hello carrying the certificate signed by the Certificate Authority (CA), which contains the public key (post-quantum) of the server and the signature (post-quantum). The client then verifies the signature and sends a finished message to the server indicating the end of the handshake. After a successful handshake, application data is securely transferred using the derived session key. Our analysis focuses on packet-level overhead as opposed to the broader networking overhead between the two hosts. We use local virtual machines loaded with OQS-OpenSSL_1_1_1 [5] acting as a client and server to establish a TLS 1.3 connection using PQC authentication. We use tcpdump [33] to capture the TLS and TCP/IP handshake packets and Wireshark [34] to analyze the packet data. More details about the experimental setup are provided below.

We established the TLS 1.3 connection 1000 times for the experimental samples and ran and compared the performances with RSA (not quantum-resistant) to provide a reference.

Table 4. TLS performance with different digital signature algorithms, including CPU usage with a 95% confidence interval, certificate size, TCP segment size, number of server hello packets, server hello size, and number of handshake packets. Table lists the default enabled SPHINCS⁺-128f-robust variant algorithms of SPHINCS⁺ only.

Algorithm	Time (ms)	CPU ± CI	CS (B)	TSS (B)	SHP	SHS (B)	HP	CS/SHS	SHP/HP
Dilithium 2	3.71	23.90% ± 2.14%	5340	6607	2	6541	8.78	0.816	0.22
Dilithium 3	3.98	27.33% ± 2.10%	7391	8754	2	8688	9.49	0.851	0.211
Dilithium 5	4.31	31.29% ± 0.51%	10,020	12,237	3	12,171	10.58	0.823	0.283
Falcon 512	3.83	29.84% ± 0.47%	2358	2645	1	2579	8.06	0.914	0.124
Falcon 1024	4.54	31.43% ± 0.61%	4426	4788	2	4722	8.16	0.937	0.245
SPHINCS+-Haraka	14.34	35.38% ± 0.85%	23,442	35,149	8	34,621	21.65	0.677	0.369
SPHINCS+-SHA-256	53.73	36.08% ± 0.96%	23,442	35,137	7	34,621	20.48	0.677	0.341
SPHINCS+-SHAKE256	110.02	36.63% ± 0.87%	23,442	35,083	7	34,612	20.38	0.677	0.341
RSA 2048	3.46	17.16% ± 0.58%	1000	1319	1	1253	8	0.798	0.115
RSA 4096	11.64	20.22% ± 0.63%	1696	2006	1	1940	8.02	0.874	0.124

shows the algorithms, time, CPU usage, Certificate Size (CS), total TCP Segment Size (TSS), number of Server Hello Packets (SHPs), Server Hello Size (SHS), and number of Handshake Packets (HPs). Time refers to the average handshake time elapsed for a connection, CPU usage represents the highest percentage logged during connection establishment, and CS provides the size of the certificate generated using the algorithms listed. We observed that Server Hello with large certificates uses TCP segmentation. SHP represents the number of packets used to transfer the Server Hello message. TSS provides the total data transferred by TCP segments. SHS represents the total Server Hello size in bytes that contains the certificate and TLS extensions. HP represents the total number of packets used to establish the connection (counted to client’s finished message).

8.1. Packet-Level Handshake Analysis: Time Overhead

This section analyzes the handshake overheads in the unit of average connection time and processing (CPU utilization). Our results in Table 4 show that Dilithium 2 outperforms the other quantum-resistant algorithms in average handshake time. Falcon 512 is the next fastest pqc algorithm in handshake establishment and takes 3.23% more time than Dilithium 2. Dilithium 2 and Falcon 512 are 7.22% and 10.6% slower than RSA 2048, while they are 68.12% and 67.09% faster than RSA 4096, respectively.

SPHINCS⁺-Haraka-128f-robust takes 286.52% and 274.41% more than Dilithium 2 and Falcon 512, respectively. SPHINCS⁺-SHAKE256-128f-robust takes the longest time for a TLS connection establishment and is two orders larger in magnitude than Dilithium or Falcon’s connection establishment times.

8.2. Packet-Level Handshake Analysis: Certificate Size and Server Hello

This section analyzes the overhead caused by the CS and SHS on the connection handshake. Falcon 512’s CS (2358 B) is 55.94% smaller than Dilithium 2 (5340 B). Falcon 512 is the only post-quantum algorithm that transfers its Server Hello message in a single packet and is suitable for devices capable of handling only small certificate sizes due to small buffer sizes. We compute the fraction that the CS takes up inside the Server Hello with ${\displaystyle \frac{\mathrm{C}\mathrm{S}}{\mathrm{S}\mathrm{H}\mathrm{S}}}$. This provides the additional extension overhead that each algorithm enforces on the TLS connection. Falcon 1024 has a ${\displaystyle \frac{\mathrm{C}\mathrm{S}}{\mathrm{S}\mathrm{H}\mathrm{S}}}$ percentage of 93.7%, causing only 6.3% overhead. Except for Falcon 512, all other post-quantum algorithms use TCP segmentation for their Server Hello message, indicating the additional overhead within the handshake. Fraction ${\displaystyle \frac{\mathrm{S}\mathrm{H}\mathrm{P}}{\mathrm{H}\mathrm{P}}}$ implicates the effect of the post-quantum certificate in the Server Hello of the handshake. A handshake using Falcon 512 is composed of 12.4% Server Hello, while the other post-quantum algorithms have a percentage of 20% or more.

8.3. PQC Algorithm Choices with TLS Integration

From our analysis, Dilithium 2 has the fastest connection time among the PQC algorithms. Falcon 512 is a better alternative to the Dilithium family due to its low CS and similar packet overheads to RSA.

9. Takeaways and Discussion

We analyze the PQC algorithms in security and performance in this paper and summarize our choices and recommendations based on the analyses in this section. In security costs, Falcon 512/1024 incurs the most computational effort against a quantum-equipped cryptanalyst among the lattice-based algorithms in qubits (Section 5.2). Our performance analyses focusing on the algorithms only (Section 6) yield that Dilithium 2 is the quickest for signing for messages shorter than 3.256 MB and that SPHINCS⁺-Haraka-128f-robust is the quickest for signing messages longer than 3.256 MB. When the PQC algorithm is integrated with TLS and TCP/IP (Section 8), Dilithium 2 is the best both in connection handshake time and in CPU processing, while Falcon 512 has the shortest certificate size affecting the payload and memory.

The choice of the PQC digital signature algorithm depends on both the PQC application requirements and the R&D in quantum computing and cryptanalysis. Our PQC recommendations are based on standard practices in networking security. For example, the key exchange occurs more sporadically than the message communication frequency (if they are comparable, using a one-time pad can be an option for information-theoretic security resistant against (quantum-)computationally capable adversaries), and therefore we prioritize the recommendations based on the signature length as opposed to the public key length in Section 5.2. Other choices depend on the application domains and properties, including the application-layer message size affecting the efficiency performance comparison in the PQC algorithms in Section 6 and the signing frequency vs. verifying frequency providing different prioritization. For example, the cryptocurrency blockchain utilizing digital signatures for the transaction integrity would prioritize the verifying efficiency (since, for every signing to generate a transaction, numerous miners would verify the signature/transaction) and has a message input size less than 460 kB (e.g., Bitcoin has transaction sizes in the order of hundreds to thousands of Bytes), so the performance-focused PQC choice would be Dilithium 2. For networking-constrained applications, the algorithm can also be chosen in order to minimize the number of transmitted packets, which depends on the algorithm’s certificate size and the networking protocol’s packet specification including the field length. For example, if the PQC is used for TLS and TCP/IP in constrained environment, then Falcon 512 would be the choice as studied in Section 8.

10. Related Work in PQC Analyses

In this section, we discuss more work related to our research, including post-quantum algorithm performance studies, cryptanalysis, and security studies.

10.1. PQC Digital Signature Performance Analyses in TLS and TCP/IP and Beyond

Sikeridis et al. [35] provided a performance study on the post-quantum digital signature algorithm candidates of the NIST’s PQC standardization project. The few selected parameters of seven out of the nine algorithms in the second round were integrated with TLS 1.3 and analyzed for networking latency for the respective algorithms. The authors also proposed a scheme to use different post-quantum algorithms regarding Certificate Authority (CA) and Intermediate Certificate Authority (ICA) to improve the overall handshake speed and throughput. Our work is comparable to this work in that it includes performance analyses when the PQC algorithms are integrated with TLS. However, our work provides the performance analyses with finer granularity at the packet level, enabling richer analyses (e.g., analyzing the required number of packet transmissions to capture the relationship between the certificate size and the protocol’s segmentation).

Basu et al. conducted a hardware evaluation study on the signature candidates, including Dilithium, in [36]. Out of the three signature algorithms they analyzed, only Dilithium advanced to the third round of the NIST’s PQC standardization project. Based on implementations in Field Programmable Gate Arrays (FPGAs) and Application-Specific Integrated Circuits (ASICs), their analysis recommends the use of Dilithium in server implementations with low latency.

Beyond TLS and TCP/IP, other research showed that post-quantum digital signatures are feasible and provided performance analyses in widely used networking protocols, including QUIC (a more advanced protocol to replace TCP/TLS) [37,38,39,40], power analysis [41], email or S/MIME [42,43], DNS/DNSSEC [44,45], and 4G/5G eSIM provision [46,47].

10.2. Cryptanalysis and Security Analyses

Lattice-based cryptanalysis is provided in [48,49]. The authors in [48] provided a software toolkit named Sage 9.0 to perform side-channel attacks on lattice-based cryptography. They also proposed a cryptanalysis framework that can take advantage of side information or hints to perform lattice reduction attacks. Their analysis shows a significant cost reduction in performing cryptanalysis that utilizes hints. In [49], the authors performed cryptanalysis based on skip-addition fault attacks. They made use of the determinism in the signature algorithm and injected a single fault targeting the signing operation. A portion of the secret key was extracted and used by the proposed forgery algorithm to generate signatures. Their analyses included skip-addition attacks on Dilithium and zero-cost mitigations.

10.3. Hardware-Based PQC Analyses

As lattice-based and hash-based cryptography differ in the underlying design, many researchers have looked at developing special-purpose hardware to speed up the underlying arithmetic operations and improve the algorithm performances. These include special hardware design implementations or accelerators for Dilithium [50,51,52,53,54,55], Falcon [56,57,58,59], and SPHINCS+ [60,61,62,63,64].

Our PQC implementations are software-based—i.e., the different cryptographic ciphers are implemented in software, while fixing the hardware. The hardware is fixed—i.e., we use the same device platform to test across the PQC algorithms, for fair comparison. Furthermore, our work does not aim to advance or optimize the efficiency of the cipher computations, in contrast to the aforementioned previous work using hardware to improve efficiency.

11. Conclusions

This paper presents security comparisons and performance analyses of NIST-selected post-quantum digital signature algorithms. In our security comparisons, we use a visualization model to analyze the trade-off between the key/signature size vs. security. In our performance analyses, we analyze the performances of the NIST-selected PQC digital signature schemes for key generation, signing, and verifying signatures. To measure the PQC implementation costs and the overheads in communication, time, and processing, we also integrate the PQC algorithms with PKI and TLS 1.3. Our analyses results show that the lattice-based algorithms are computational efficient compared to the hash-based algorithms. Our results from integration with TLS 1.3 show that, at comparable security strengths, Dilithium 2 and Falcon 512 outperform RSA 4096 in handshake time duration by 68.12% and 67.09%, respectively.