Quantum-Resistant Mutual Authentication Scheme for Edge Computing Environments

Abstract

Edge computing has emerged as a distributed computing technology to mitigate the cloud computing overload caused by the rapid increase in connected devices. However, because communications between devices and edge servers are conducted over public channels, authentication and secure session key establishment are imperative to protect against various security attacks. In this paper, we show that Kenioua et al.’s authentication scheme for edge computing is vulnerable to several attacks such as impersonation, offline password guessing, and stolen verifier attacks, and also lacking quantum resistance against the emerging threat posed by quantum computing. To overcome these limitations, we propose a quantum-resistant authentication scheme by adopting module lattice-based key encapsulation mechanism (ML-KEM). We demonstrate the robustness of the proposed scheme through “the Burrows–Abadi–Needham (BAN) logic”, “Quantum Random Oracle Model (QROM)”, “Automated Validation of Internet Security Protocols and Application (AVISPA) tool”, and “Scyther tool”, and show that the proposed scheme achieves security with efficient communication and computation costs by comparing it with related studies.

1. Introduction

With the rapid growth in the number of connected devices and the increasing complexity of cloud-based services, centralized cloud computing architectures face significant challenges in processing massive volumes of data and computation [1,2]. In particular, if all data and control tasks are transmitted to remote cloud servers, the system may suffer from excessive network traffic, scalability limitations, and nonnegligible latency. To overcome these issues, edge computing has emerged as a distributed computing architecture that places computation, storage, and control functions closer to end devices, particularly in mobile network environments where low latency, mobility support, and reliable real-time communication are essential [3]. By reducing the volume of data transmitted to centralized cloud servers, edge computing alleviates network congestion and enables faster real-time responses [4]. Therefore, edge computing is well suited for latency-sensitive applications and offers practical benefits such as improved productivity, lower communication and infrastructure costs, and efficient remote data collection in geographically isolated industrial environments [5].

A typical edge computing system consists of three layers: the device, edge, and cloud layers [6]. The device layer contains resource-constrained end devices, such as sensors, actuators, and mobile nodes. The edge layer performs local processing and provides rapid service responses near data sources. In many practical deployments, these layers are interconnected through mobile network infrastructures, where edge servers are placed close to radio access points or base stations to support real-time services [7]. The cloud layer supports global coordination, large-scale storage, and high performance computation [8]. Owing to these characteristics, edge computing has been widely adopted in diverse environments, including internet of things systems, smart cities, healthcare, robotic manufacturing, autonomous vehicular networks, and other mobile network services [9]. Despite these advantages, edge computing also introduces serious security challenges. Because most devices and edge servers communicate over public channels, they are exposed to impersonation, replay, man-in-the-middle (MITM), and other security attacks [10,11]. In particular, authentication becomes a fundamental security requirement because only legitimate devices and users should be allowed to access edge services and exchange sensitive data. This requirement becomes even more important in large-scale edge computing systems, where numerous devices continuously exchange information over public channels, and the absence of proper authentication may directly lead to service disruption, privacy leakage, or unauthorized control of distributed resources [12].

In addition to conventional security threats, edge computing systems must also address the emerging challenge of quantum computing [13]. Existing authentication protocols often depend on public key cryptography such as elliptic curve cryptography (ECC) and RSA for entity authentication and key establishment. However, these schemes rely on mathematical problems that can be efficiently solved by Shor’s algorithm once sufficiently large quantum computers become available [14]. As noted by the National Institute of Standards and Technology (NIST), this makes currently deployed public key mechanisms unsuitable for long-term protection in the quantum era [15]. Moreover, the threat is not limited to the time when quantum computers become available. Under the harvest now, decrypt later model, an adversary can intercept communications today and store them for future decryption [16]. This is particularly dangerous for edge computing systems, where authentication and data exchange may involve information that must remain secure for a long period. From the perspective of Mosca’s inequality, if the required security lifetime of the data is longer than the time remaining until practical quantum attacks emerge, then classical public key cryptography can no longer provide sufficient assurance [17]. Therefore, it is necessary to incorporate post quantum cryptography (PQC) into authentication protocols for edge computing environments.

Recently, Kenioua et al. [18] proposed a password-based authentication scheme for edge computing environments. However, our analysis shows that their scheme suffers from several security weaknesses. Their scheme is susceptible to offline password guessing, privileged insider, stolen verifier, impersonation, and ephemeral secret leakage (ESL) attacks and does not achieve anonymity and untraceability. Moreover, the scheme does not provide resistance against quantum attacks. To address these limitations, we propose a quantum-resistant authentication scheme for edge computing by incorporating ML-KEM, a representative PQC standardized by NIST [19]. Unlike conventional schemes based solely on classical cryptographic assumptions, the proposed scheme is designed to withstand advanced attackers equipped with quantum computational capabilities. Therefore, the proposed scheme provides enhanced security for edge computing environments while addressing the vulnerabilities of the existing scheme and the urgent need for quantum resistance. After authentication, the device and the edge server can securely communicate over a public channel using the established session key.

1.1. Contributions

This study makes the following main contributions.

• We conduct cryptanalysis of Kenioua et al.’s scheme to reveal vulnerabilities such as impersonation, offline password guessing, and stolen verifier attacks that do not guarantee anonymity, untraceability, and mutual authentication.
• We propose a quantum-resistant authentication scheme for edge computing environments by integrating ML-KEM into the authentication process, thereby enabling post-quantum secure authentication.
• We illustrate the security of the proposed scheme for various attacks through informal analysis, and demonstrate the robustness of the proposed scheme through BAN logic [20], QROM [21], AVISPA [22,23], and Scyther [24].
• We compare the proposed scheme with existing studies and show that the proposed scheme is efficient from a communicative and computational perspective and has high security.

1.2. Organization

Section 2 introduces related studies including the classic authentication scheme and the authentication scheme considering quantum resistance. Section 3 explains background knowledge including ML-KEM, the system model, and the adversary model to facilitate understanding of authentication scheme. Section 4 and Section 5 briefly explain the scheme of Kenioua et al. and analyze the weaknesses of the scheme. Section 6 proposes a ML-KEM-based authentication scheme for an edge computing environment. Section 7 demonstrates the robustness of the proposed scheme, and Section 8 verifies the efficiency through comparative analysis with existing studies. Finally, Section 9 summarizes this study.

2. Related Work

In recent years, various authentication protocols have been proposed for session key establishment between users and servers in edge computing. Jia et al. [25] proposed an identity-based authentication scheme for mobile edge computing and claimed that it could ensure anonymity, untraceability, and resistance against various attacks. However, Xie et al. [26] showed that Jia et al.’s scheme is vulnerable to offline key guessing and insider attacks and fails to preserve user anonymity. To overcome these weaknesses, Xie et al. proposed an ECC-based anonymous authentication scheme for edge computing. Nevertheless, as shown in [27], this scheme is still vulnerable to ESL and impersonation attacks. Later, Jia et al. [28] redesigned the anonymous authentication scheme and claimed improved security. However, Ma et al. [27] revealed that the redesigned scheme remains vulnerable to offline key guessing, ESL, and impersonation attacks. Kenioua et al. [18] proposed a password-based authentication scheme for edge computing architectures. Their scheme relies only on modular computations and hash functions and achieves low communication cost. Nevertheless, our analysis shows that Kenioua et al.’s scheme cannot defend against various attacks, including offline password guessing, impersonation, and privileged insider attacks. Moreover, it fails to provide user anonymity, untraceability, and mutual authentication.

In addition to conventional classical threat models, quantum-resistant authentication protocols have also been studied to prepare for the advent of quantum computers. Ding et al. [29] suggested a lattice-based anonymous authentication protocol. Their scheme is based on ring learning with errors (RLWE)-based key exchange and was designed as an improvement over an earlier scheme of Ding et al. to address signal leakage attacks. Chen et al. [30] introduced a multi-factor authentication scheme for cloud-assisted Internet of medical things systems. Their scheme adopted an error reconciliation function in lattice-based cryptography. However, Ahmad et al. [31] pointed out that the scheme in [29] strongly depends on user identity information without biometric support, whereas the scheme in [30] does not preserve biometric privacy and resist replay and MITM attacks. To overcome these limitations, Ahmad et al. suggested a quantum-safe user authentication protocol based on a post-quantum fuzzy commitment scheme. Nevertheless, their scheme does not resist denial-of-service (DoS) attacks and preserve user anonymity. Palaniswamy et al. [32] introduced an authentication scheme with post quantum security. Their scheme relies on the RLWE problem and adopts a signcryption scheme with the key encapsulation mechanism SETLA-KEM. Although SETLA-KEM provides both signature and encryption simultaneously, it incurs high communication overhead and is not well suited for general deployment. In addition, the scheme does not achieve user anonymity and cannot prevent DoS attacks, as discussed in [33]. Wen et al. [33] proposed a secure multi-factor authentication scheme for multi-server architectures, in which the ML-KEM was employed to achieve quantum resistance. These studies show that lattice-based computation and post-quantum cryptographic mechanisms have been actively incorporated into authentication protocols to achieve quantum resistance. Motivated by that, in this paper we identify the weaknesses of Kenioua et al.’s scheme and propose a quantum-resistant authentication and key agreement scheme for edge computing based on ML-KEM.

3. Preliminaries

Before the analysis of Kenioua et al.’s scheme and proposal of our scheme, we describe preliminaries. These preliminaries include module learning with errors (MLWE), ML-KEM, the system model, the adversary model, and notation for understanding the schemes.

3.1. Module Learning with Errors

Lattice-based cryptography is a major foundation of PQC because no efficient quantum algorithms exist for solving lattice problems [34]. Among lattice-based assumptions, the learning with errors (LWE) problem is one of the most fundamental hardness assumptions and underlies many practical PQC schemes [35].

Definition 1 

(LWE). Let $q\ge 2$ be an integer modulus, let $n,m\in \mathbb{N}$, and let χ be an error distribution over ${\mathbb{Z}}_q$. For a secret vector $\mathbf{s}\in {\mathbb{Z}}_q^n$, the LWE distribution is defined by samples

$$(\mathbf{a},b)\in {\mathbb{Z}}_q^n\times {\mathbb{Z}}_q,$$

where

$$\mathbf{a}\leftarrow {\mathbb{Z}}_q^n,b=\langle \mathbf{a},\mathbf{s}\rangle +e\left(\mathrm{mod}q\right),e\leftarrow \chi .$$

The search-LWE problem is to recover $\mathbf{s}$ from sufficiently many samples, whereas the decision-LWE problem is to distinguish such samples from uniform ones.

Although LWE provides a strong security foundation, it often incurs large key sizes and high computational costs [36]. To improve efficiency, RLWE extends LWE to polynomial rings, where

$$R_q={\mathbb{Z}}_q\left[x\right]/\left(f\left(x\right)\right),$$

with $f\left(x\right)$ is commonly set as a cyclotomic polynomial, such as $x^n+1$.

Definition 2 

(RLWE). Let χ be an error distribution over $R_q$. For a secret polynomial $s\in R_q$, the RLWE distribution is defined by samples

$$(a,b)\in R_q\times R_q,$$

where

$$a\leftarrow R_q,b=a·s+e\in R_q,e\leftarrow \chi .$$

The search-RLWE problem is to recover s, whereas the decision-RLWE problem is to distinguish $(a,b)$ from uniformly random pairs in $R_q\times R_q$.

The RLWE offers compact keys and efficient polynomial arithmetic, but its single-ring structure introduces stronger algebraic regularity. To balance efficiency and structural generality, the MLWE was introduced.

Definition 3 

(MLWE). Let $R_q={\mathbb{Z}}_q\left[x\right]/\left(f\left(x\right)\right)$, let $k,m\in \mathbb{N}$, and let χ be an error distribution over $R_q$. For a secret vector $\mathbf{s}\in R_q^k$, the MLWE distribution is defined by samples

$$(\mathbf{A},\mathbf{b})\in R_q^{m\times k}\times R_q^m,$$

where

$$\mathbf{A}\leftarrow R_q^{m\times k},\mathbf{b}=\mathbf{A}\mathbf{s}+\mathbf{e}\in R_q^m,\mathbf{e}\leftarrow {\chi }^m.$$

The search-MLWE problem is to recover $\mathbf{s}$, whereas the decision-MLWE problem is to distinguish $(\mathbf{A},\mathbf{b})$ from uniform samples.

MLWE can be viewed as an intermediate assumption between LWE and RLWE. This approach retains the efficiency of polynomial ring arithmetic while reducing the structural rigidity of RLWE. Hence, MLWE provides a practical balance between security and efficiency in lattice-based PQC [37].

3.2. Key Encapsulation Mechanism Using ML-KEM

To establish a quantum-resistant key in the proposed scheme, we adopt ML-KEM, standardized in FIPS 203 by NIST [19]. As a post quantum key encapsulation mechanism, ML-KEM is designed to enable two parties to derive a shared secret over a public channel while maintaining security against both classical and quantum adversaries. It serves as a practical post quantum replacement for conventional public key-based key establishment mechanisms, such as RSA and ECDH.

ML-KEM’s security relies on the hardness of the MLWE problem. The ML-KEM applies structured polynomial operations and the number theoretic transform to achieve efficient key generation, encapsulation, and decapsulation [38]. This makes ML-KEM especially suitable for practical secure communication systems, where both strong security and implementation efficiency are required.

The ML-KEM-based key establishment procedure in the proposed scheme is composed of the following three algorithms:

$$\begin{array}{cc}\hfill (pk,sk)& \leftarrow KeyGen\left(\right)\hfill \\ \hfill (k,c)& \leftarrow Encap\left(pk\right)\hfill \\ \hfill k& \leftarrow Decap(sk,c)\hfill \end{array}$$

where $pk$, $sk$, c, and k denote the public key, secret key, ciphertext, and derived shared key, respectively.

In Figure 1, Party A first generates a private key $sk$ and a public key $pk$ using the key generation algorithm, keeping $sk$ secret and publishing $pk$. Then, party B uses A’s public key $pk$ to encapsulate a shared secret key k and generate a ciphertext c, which is transmitted to A. Upon receiving c, A decapsulates it using $sk$ and obtains the same shared secret key k. In this way, A and B establish a common secret key. In this work, we employ ML-KEM-512, corresponding to the security level of AES-128 [39]. For ML-KEM-512, the sizes of the private key, public key, and ciphertext are 1632, 800, and 768 bytes, respectively, and the generated shared key is 32 bytes [40].

3.3. System Model

Figure 2 illustrates the system model of the proposed protocol. The system consists of a cloud server, an edge server, and a device. The edge server and the device register with the cloud server and establish session keys through the authentication process. The detailed roles of each entity are described below.

• Cloud server ($CS$): The $CS$ acts as a registration center, and data center. The device $D_i$ and the edge server $ES$ must register with the $CS$ before the authentication phase. In addition, the $CS$ processes and stores high-level data transmitted by the edge server.
• Edge server ($ES$): The $ES$ handles computation tasks to reduce the traffic on the $CS$. It analyzes the data transmitted by devices and provides corresponding services. Only an authenticated edge server can access device data through the established session key.
• Device ($D_i$): The $D_i$ denotes a low-level electronic device that collects data in a specific environment. Systems including such devices may be deployed in various applications (e.g., smart homes, smart health care, vehicle area networks, and industrial IoT). After authentication, $D_i$ encrypts the collected data with the session key and transmits it to the $ES$.

3.4. Adversary Model

We adopt the “Dolev–Yao (DY) model [41]” and “Canetti–Krawczyk (CK) model [42]” to analyze Kenioua et al.’s scheme and the proposed scheme. These adversary models characterize the capabilities of an attacker over a public channel and have been widely used in many studies. In addition, we consider a quantum adversary model to account for the potential emergence of quantum computers. The specific assumptions regarding the adversary are given as follows.

• The adversary $\mathcal{A}$ can eavesdrop, forge, capture, and replay messages transmitted over a public channel [43].
• $\mathcal{A}$ can steal a device and extract stored parameters using a power analysis attack.
• After session termination, $\mathcal{A}$ can obtain the ephemeral secrets of the session and the long-term private keys of $CS$ and $ES$.
• $\mathcal{A}$ can store messages transmitted over the public channel, later recovering their contents after quantum computers become available.
• $\mathcal{A}$ can launch various attacks including insider, impersonation, and MITM attacks.

3.5. Notation

The notation for Kenioua et al.’s and the proposed scheme is presented at

Table 1. Notation.

Notation	Description
$D_i$	Device
$ES$	Edge server
$CS$	Cloud server
$I{D}_d$	Real identity of $D_i$
$P{W}_d$	Password of $D_i$
$PI{D}_d$	Pseudo identity of $D_i$
$pk,sk$	Public and secret keys
$Encap(·),Decap(·)$	ML-KEM encapsulation and decapsulation
$c_k$	ML-KEM ciphertext
$k_k$	Secret key from ML-KEM
$SK$	Session key
⊕	Exclusive-OR operation
$\left|\right|$	Concatenation operation
$h(·)$	Hash function
$a,x,y,z$	Random number
$T_k$	Timestamp

.

4. Review of Kenioua et al.’s Scheme

Before discussing the weaknesses of Kenioua et al.’s scheme, we first briefly describe its primary authentication procedure. Kenioua et al. proposed a password-based authentication protocol for secure data communication in an edge computing environment. In their scheme, a device first registers with the cloud server during the registration phase. Subsequently, during the authentication phase, the device and the edge server authenticate each other using the device’s password.

4.1. Registration Phase

During the registration phase, the device submits its identity and password to the cloud server. The cloud server verifies the submitted credentials and forwards the device’s password to the edge server for use in the subsequent authentication phase. The following steps describe the procedure in detail.

Step 1: 

$D_i$ selects $I{D}_d$ and $P{W}_d$ and sends $\{I{D}_d,P{W}_d\}$ to $CS$ via a secure channel.

Step 2: 

$CS$ checks whether $I{D}_d$ is registered in the database, validates that the length of $P{W}_d$ is suitable for the modular computation, and sends a confirmation message to $D_i$.

Step 3: 

$D_i$ stores $I{D}_d$ and $P{W}_d$ securely. Moreover, $CS$ also transmits $I{D}_d$ and $P{W}_d$ to $ES$ through a secure channel for later authentication. Then, $ES$ stores the values received in its verification table.

4.2. Login and Authentication Phase

During the authentication phase, the device and the edge server perform mutual authentication based on the device’s password shared in the registration phase. Each party validates the received message by confirming that the corresponding nonce can be correctly recovered using the stored password. After mutual verification, a session key is established from the exchanged nonces x, y, and z. The detailed procedures are described below and illustrated in Figure 3.

Step 1: 

$D_i$ generates a timestamp $T_d$ and computes $H_d=h(I{D}_d\left|\right|P{W}_d)$. $D_i$ transmits $\{I{D}_d,T_d\}$ to $ES$ via a public channel.

Step 2: 

After receiving the authentication request message, $ES$ first validates the timestamp $T_d$. Then, $ES$ retrieves $P{W}_d$ against its $I{D}_d$ and generates a timestamp $T_e$ and random numbers x and y. Then, $ES$ computes $\alpha =x+y\times P{W}_d$ and $H_e=h(I{D}_d\left|\right|P{W}_d)$, and sends $\{\alpha ,H_e,T_e\}$ to $D_i$.

Step 3: 

$D_i$ validates $T_e$ and checks whether $H_e$ and $h\left(I{D}_d\right|\left|P{W}_d\right)$ match. If they match, $ES$ is authenticated. Then, $D_i$ generates z and $T_d$ and computes $x^{\prime }=\alpha \mathrm{mod}P{W}_d$ and ${\alpha }^{\prime }=z+x^{\prime }\times P{W}_d$. Finally, $D_i$ sends $\{{\alpha }^{\prime },T_d\}$ to $ES$.

Step 4: 

$ES$ validates $T_d$ and computes $z={\alpha }^{\prime }\mathrm{mod}P{W}_d$ and $x^{\prime }=({\alpha }^{\prime }-z)÷P{W}_d$. Then, $ES$ verifies whether $x^{\prime }$ equals the generated x. If they match, $D_i$ is authenticated.

5. Cryptanalysis of Kenioua et al.’s Scheme

In this section, we present the cryptanalysis of the scheme of Kenioua et al. and identify its security weaknesses. The results reveal that the scheme is vulnerable to offline password guessing, impersonation, DoS, stolen verifier, and ESL attacks and fails to preserve user anonymity and untraceability. The details of these attacks are described below.

5.1. Offline Password Guessing Attacks

If a malicious adversary $\mathcal{A}$ obtains the hash value derived from the device identity $I{D}_d$ and password $P{W}_d$, then $\mathcal{A}$ can attempt an offline password guessing attack. In the authentication phase of Kenioua et al.’s scheme, $I{D}_d$ and $H_e$ appear in the authentication request message $\left\{I{D}_d\right\}$ and the edge server message $\{\alpha ,H_e,T_e\}$, respectively. These values are transmitted over a public channel; hence, $\mathcal{A}$ can eavesdrop and obtain them. The detailed attack procedure is presented as follows.

Step 1: 

$\mathcal{A}$ obtains $I{D}_d$ and the hash value $H_e=h(I{D}_d\parallel P{W}_d)$ from the public channel.

Step 2: 

Using $I{D}_d$ and $H_e$, $\mathcal{A}$ performs an offline brute-force attack to guess $P{W}_d$. For each password candidate $P{W}_d^{\prime }$, $\mathcal{A}$ computes $h(I{D}_d\parallel P{W}_d^{\prime })$ and compares it with the observed value $H_e$. Once a match is found, $\mathcal{A}$ recovers $P{W}_d$.

Step 3: 

After obtaining the correct password, $\mathcal{A}$ can launch further malicious attacks, such as impersonating $D_i$ or attempting credential reuse attacks on other systems if the same password is reused.

5.2. Impersonation Attacks

As described in the previous attack, the real identity $I{D}_d$ of $D_i$ is exposed over a public channel, and $I{D}_d$ is included in the authentication request message and reused in the following authentication sessions without an update process. Therefore, if an adversary $\mathcal{A}$ obtains $I{D}_d$, $\mathcal{A}$ can impersonate $D_i$ by sending an authentication request containing the exposed identity with a newly generated timestamp $T_a$. Then, $ES$ cannot distinguish such a forged request from one generated by a legitimate $D_i$, and the scheme of Kenioua et al. is susceptible to impersonation attacks.

5.3. Privileged Insider Attacks

This attack assumes that $\mathcal{A}$ can access the registration request message $I{D}_d,P{W}_d$. Users often reuse identical or similar identities and passwords across multiple websites; therefore, a privileged insider may exploit these credentials to attempt unauthorized logins on other systems. Therefore, the scheme cannot resist privileged insider attacks.

5.4. Stolen Verifier Attacks

Assume that the adversary $\mathcal{A}$ has obtained the verification table of $ES$. In the registration phase of Kenioua et al.’s scheme, the edge server stores $I{D}_d$ and $P{W}_d$ in plaintext in its verification table without any additional encryption. Once $\mathcal{A}$ acquires this table, $\mathcal{A}$ can immediately acquire $I{D}_d$ and $P{W}_d$, and use them to impersonate $D_i$ or launch further attacks. Therefore, the scheme is vulnerable to stolen verifier attacks.

5.5. Ephemeral Secret Leakage Attacks

In the authentication phase of Kenioua et al.’s scheme, the session key between $D_i$ and $ES$ is established solely from the random values x, y, and z generated after that session. In such a setting, if an adversary $\mathcal{A}$ obtains the ephemeral random numbers, the session key can be directly derived. Therefore, Kenioua et al.’s scheme is susceptible to ESL attacks, since the derivation of the session key does not incorporate long-term secrets.

5.6. Quantum Attacks

Kenioua et al.’s scheme mainly relies on hash functions and classical modular arithmetic operations; therefore, the scheme cannot maintain long-term security against quantum adversaries. As discussed in Section 3.4, an adversary $\mathcal{A}$ may adopt a strategy of harvest now, decrypt later. The encrypted messages transmitted today are collected and stored even if they cannot be immediately decrypted. The messages can later be recovered once quantum computing capabilities become sufficiently advanced.

5.7. Anonymity and Untraceability

In authentication, device anonymity is critical because the privacy of the device owner must be protected, and communicating parties should not be able to trace repeated authentication sessions over a public channel. However, in Kenioua et al.’s scheme, the device’s real identity $I{D}_d$ is transmitted in plaintext within the authentication request message. Therefore, if $\mathcal{A}$ eavesdrops on the public channel, the scheme cannot preserve device anonymity. Moreover, $I{D}_d$ is reused without update in every session; hence, $\mathcal{A}$ can link multiple authentication sessions of $D_i$ and trace its activity over time. Hence, the scheme fails to provide anonymity and untraceability.

5.8. Mutual Authentication

In the above attacks, Kenioua et al.’s scheme cannot resist various security threats. Moreover, an adversary $\mathcal{A}$ can impersonate a device and send forged yet acceptable messages to $ES$. In addition, $\mathcal{A}$ can successfully guess $P{W}_d$ by exploiting information transmitted over the public channel. Although $D_i$ and $ES$ attempt to verify each other during the authentication process, such verification cannot be considered trustworthy under these attacks. Therefore, the scheme of Kenioua et al. fails to achieve mutual authentication.

6. Proposed Scheme

We present a quantum-resistant authentication and key agreement scheme using ML-KEM for edge computing systems. In this paper, we assume that clock synchronization has already been established among the participating entities. Such synchronization can be maintained within an acceptable tolerance window through lightweight periodic time updates at the system level [44,45]. The proposed scheme enables the device and the edge server to establish a shared session key and then perform secure communication over a public channel. The scheme consists of three phases: the initialization phase, the registration phase, and the authentication and key agreement phase.

6.1. Initialization Phase

In the initialization phase, $CS$ selects a master key $msk$ and a hash function $h(·)$ to be employed in the authentication process. Furthermore, $CS$ keeps $msk$ secret and publishes $h(·)$ to the system.

6.2. Registration Phase

Both $D_i$ and $ES$ must register with $CS$ and obtain the values required for authentication. The registration phase is conducted over a secure channel. During this phase, $CS$ only facilitates the establishment of the required authentication parameters and does not store the shared secret between $D_i$ and $ES$ after registration. Figure 4 illustrates the detailed procedure.

Step 1: 

$D_i$ chooses $I{D}_d$ and $P{W}_d$. $D_i$ sends $\left\{I{D}_d\right\}$ to $CS$. In the meantime, $ES$ generates a public key and secret key using $keyGen\left(\right)$, computes $(c_e,k_e)\leftarrow Encap\left(p{k}_e\right)$, and transmits $p{k}_e$ to $CS$.

Step 2: 

$CS$ generates a pseudo-identity $PI{D}_d$ and saves the parameters $I{D}_d$ and $PI{D}_d$. Then, $CS$ computes a shared secret key $DS{K}_d$ between $D_i$ and $ES$ using $msk$ from $CS$ and transmits $\{PI{D}_d,DS{K}_d,p{k}_e\}$ and $\{PI{D}_d,DS{K}_d\}$ to $D_i$ and $ES$, respectively.

Step 3: 

$D_i$ generates a random number a and computes $L_d=h(I{D}_d\left|\right|P{W}_d\left|\right|a)$ and $A_d=\left(a\right||DS{K}_d)\oplus h(I{D}_d\left|\right|P{W}_d)$. Then, $D_i$ securely stores $\{PI{D}_d,L_d,A_d\}$. In addition, $ES$ computes $EDS{K}_d=DS{K}_d\oplus k_e$ and saves $\{PI{D}_d,EDS{K}_d\}$ in their database.

6.3. Authentication and Key Agreement Phase

In this phase, $D_i$ and $ES$ perform authentication using ML-KEM and establish a session key. By incorporating ML-KEM, the proposed scheme achieves quantum-resistant security. After completing this phase, $D_i$ and $ES$ can communicate securely over a public channel using the established session key. Figure 5 presents the detailed procedure.

Step 1: 

$D_i$ enters $I{D}_d$ and $P{W}_d$ for login, where $D_i$ computes $\left(a\right||DS{K}_d)=A_d\oplus h(I{D}_d\left|\right|P{W}_d)$ and verifies whether $L_d\stackrel{?}{=}h(I{D}_d\left|\right|P{W}_d\left|\right|a)$. If they match, $D_i$ successfully completes the login. Then, $D_i$ generates a timestamp $T_d$ and computes $(p{k}_d,s{k}_d)\leftarrow Keygen\left(\right)$, $(c_1,k_1)\leftarrow Encap\left(p{k}_e\right)$, $H_d=h(P{W}_d\left|\right|a\left|\right|k_1)$, $DI{D}_d=(PI{D}_d\left|\right|H_d)\oplus k_1$, and $V_d=h(k_1\parallel |PI{D}_d\left|\right|$$DS{K}_d\left|\right|H_d\left|\right|p{k}_d\left|\right|T_d)$. Next, $D_i$ sends $\{DI{D}_d,c_1,p{k}_d,V_d,T_d\}$ to $ES$ via a public channel.

Step 2: 

$ES$ validates $T_d$. If valid, $ES$ computes $k_1\leftarrow Decap(s{k}_e,c_1)$ and $(PI{D}_d\left|\right|H_d)=DI{D}_d\oplus k_1$, and retrieves $EDS{K}_d$. Then, $ES$ computes $k_e\leftarrow Decap(s{k}_e,c_e)$ and $DS{K}_d=EDS{K}_d\oplus k_e$ and checks $V_d\stackrel{?}{=}h(k_1\parallel |PI{D}_d\left|\right|DS{K}_d\left|\right|H_d\left|\right|p{k}_d\left|\right|T_d)$. If they match, the authentication request is accepted. Next, $ES$ generates $T_e$, computes $(c_2,k_2)\leftarrow Encap\left(p{k}_d\right)$, $SK=h\left(k_1\right|\left|k_2\right|\left|H_d\right|\left|T_e\right)$, and $V_e=h(DS{K}_d\left|\right|SK\left|\right|k_2\left|\right|T_e)$ and transmits $\{c_2,V_e,T_e\}$ to $D_i$.

Step 3: 

After the message from $ES$ is received, $D_i$ checks the validity of $T_e$. If valid, $D_i$ computes $k_2\leftarrow Decap(s{k}_d,c_2)$ and $SK=h\left(k_1\right|\left|k_2\right|\left|H_d\right|\left|T_e\right)$, and $ES$ checks $V_e\stackrel{?}{=}h(DS{K}_d\left|\right|SK\left|\right|k_2\left|\right|T_e)$. If they match, the authentication is completed.

7. Security Analysis

We conduct the security analyses of the proposed scheme informally and formally. The informal analysis reveals that the proposed scheme resists various security attacks, including insider, impersonation, offline password guessing, and stolen verifier attacks. The formal analysis further demonstrates the robustness of the authentication scheme using BAN logic, QROM, AVISPA, and the Scyther tool.

7.1. Informal Analysis

This section presents the informal analysis of the proposed scheme. The analysis reveals that the proposed scheme resists various security attacks and achieves user anonymity, perfect forward secrecy, and mutual authentication. The sections below discuss the details of each security property.

7.1.1. Stolen Device Attacks

We assume that the adversary $\mathcal{A}$ obtains a device and then attempts to derive the session key from the information stored in its memory. Although $D_i$ stores $PI{D}_d,L_d,A_d$ in its local memory, $\mathcal{A}$ cannot recover the authentication parameter $DS{K}_d$ without knowing $I{D}_d$ and $P{W}_d$. Moreover, the session key $SK=h\left(k_1\right|\left|k_2\right|\left|H_d\right|\left|T_e\right)$ is derived from $k_1$ and $k_2$, which are generated using ML-KEM. As $\mathcal{A}$ cannot discover $k_1$ and $k_2$ without the secret keys of $D_i$ and $ES$, the session key cannot be reconstructed. In addition, the key generated during the authentication phase rather than the registration phase; thus, the secret key $s{k}_d$ is not permanently stored in the device memory or reused across sessions. Therefore, even if the device is stolen, $\mathcal{A}$ cannot obtain the secret key or derive the session key.

7.1.2. Impersonation Attacks

A malicious adversary $\mathcal{A}$ attempts to masquerade as $D_i$ or $ES$ by sending forged messages during the authentication phase. To impersonate $D_i$, $\mathcal{A}$ must determine $P{W}_d$ and the login-related parameters a. To impersonate $ES$, $\mathcal{A}$ must obtain the authentication parameter $DS{K}_d$ shared between $D_i$ and $ES$. However, these values are protected in masked form during the authentication phase and are transmitted only over the public channel. Therefore, $\mathcal{A}$ cannot recover the required values solely by eavesdropping on the public channel.

7.1.3. Offline Guessing Attacks

An adversary $\mathcal{A}$ attempts to guess $I{D}_d$ and $P{W}_d$ of $D_i$. However, no information related to $I{D}_d$ or $P{W}_d$ is directly exposed in authentication messages. Thus, $\mathcal{A}$ cannot recover $PI{D}_d$ from exchanged messages. Although $\mathcal{A}$ may obtain the login parameter $L_d=h(I{D}_d\left|\right|P{W}_d\left|\right|a)$, recovering the individual values is computationally infeasible because $\mathcal{A}$ must simultaneously guesses $I{D}_d$, $P{W}_d$, and a. Therefore, the proposed scheme resists offline guessing attacks.

7.1.4. Insider Attacks

As described in Section 3.4, an insider adversary $\mathcal{A}$ may register as a valid device with $CS$ and take part in the authentication process with $ES$. During this process, $\mathcal{A}$ may attempt to obtain the authentication-related security parameters of another device. However, the exchanged authentication messages are protected via ML-KEM-based encryption. Although $\mathcal{A}$ can act as an insider in the protocol, $\mathcal{A}$ cannot obtain the secret keys of $ES$ or those of other devices.

7.1.5. Privileged Insider Attacks

An adversary $\mathcal{A}$ is assumed to function as a privileged insider who can access the device registration message containing $I{D}_d$. In practice, users often reuse the same $I{D}_d$ and $P{W}_d$ across multiple websites. Exploiting this tendency, a privileged insider $\mathcal{A}$ may attempt to guess $P{W}_d$ from the exposed $I{D}_d$ and use the recovered credentials to access other services. However, in the registration phase, $D_i$ transmits only $I{D}_d$ to $CS$ and does not send any information related to the password. Therefore, even a privileged insider cannot recover $P{W}_d$ from the registration message.

7.1.6. ESL Attacks

Assume that an adversary $\mathcal{A}$ obtains the session specific temporary parameters and attempts to derive the session key $SK=h\left(k_1\right|\left|k_2\right|\left|H_d\right|\left|T_e\right)$. In the authentication phase of the proposed scheme, the temporary parameters correspond to the ML-KEM-derived values $k_1$ and $k_2$. However, even if $\mathcal{A}$ acquires these values, the session key cannot be computed without $H_d$, which is derived from the device’s password $P{W}_d$. Therefore, the session key cannot be recovered from session specific temporary values alone.

7.1.7. Replay and MITM Attacks

In replay and MITM attacks, the adversary $\mathcal{A}$ eavesdrops on messages exchanged during the authentication phase, replays previously transmitted messages, or forges message parameters. In the proposed scheme, each authentication message contains a ML-KEM ciphertext, a verification parameter, and a timestamp. A replayed message is rejected because the timestamp is freshly generated in every session, and the corresponding verification parameter is computed based on that timestamp. In addition, message forgery is infeasible because the adversary must know secret values, such as $PI{D}_d$ and $h_d$, to generate valid parameters. These values are not available to $\mathcal{A}$; hence, forged messages cannot pass the verification process.

7.1.8. Stolen Verifier Attacks

An adversary $\mathcal{A}$ can steal the verification table maintained by $ES$ and then attempts to derive the session key $SK=h\left(k_1\right|\left|k_2\right|\left|H_d\right|\left|T_e\right)$ shared between each $D_i$ and $ES$. In the registration phase, $ES$ stores $PI{D}_d$ and $EDS{K}_d$. However, the shared value $DS{K}_d$ is not stored directly. Instead, it is protected as $EDS{K}_d$ through an XOR operation with the secret value $k_e$ of $ES$. Therefore, even if $\mathcal{A}$ obtains the verification table, the original value $DS{K}_d$ cannot be recovered without knowledge of $k_e$. Hence, the proposed scheme resists stolen verifier attacks.

7.1.9. Denial-of-Service Attacks

By sending numerous authentication request messages, DoS attacks aim to overload the edge server $ES$. However, in the proposed scheme, each authentication request message is generated using the secret values of $D_i$ and is subject to multiple verification steps at $ES$, including timestamp verification, $PI{D}_d$ validation, and $V_d$ verification. The adversary does not know the valid values of $PI{D}_d$ and $H_d$; thus, the forged $DI{D}_d$ cannot pass the verification process. Therefore, $\mathcal{A}$ cannot succeed in DoS attacks.

7.1.10. Quantum Attacks

The proposed scheme incorporates ML-KEM, a post quantum cryptographic primitive whose security is founded on the hardness of the MLWE problem. Although an adversary $\mathcal{A}$ may possess quantum computational capabilities, no efficient algorithm is currently known to solve the MLWE problem. Hence, the proposed scheme provides robustness against quantum attacks and ensures quantum-resistant security.

7.1.11. User Anonymity and Untraceability

The identity $I{D}_d$ is never transmitted over a public channel in the proposed scheme, preserving device anonymity. In addition, the temporary identity $PI{D}_d$, which is generated during the registration phase, is not sent directly, but is masked with the ML-KEM-derived secret value $k_1$ and transmitted in the form of $DI{D}_d$. As $k_1$ is freshly generated in each authentication session, the corresponding $DI{D}_d$ also changes from session to session. Furthermore, the device’s public key $p{k}_d$ included in the authentication request is regenerated by $Keygen\left(\right)$ for every authentication. Therefore, a third party observing only the public channel cannot link the consecutive authentication sessions of a specific device.

7.1.12. Perfect Forward Secrecy

In this attack, the adversary $\mathcal{A}$ aims to derive the session key $SK=h\left(k_1\right|\left|k_2\right|\left|H_d\right|\left|T_e\right)$ by compromising the long-term secret keys of the server. In the proposed scheme, the keys are $msk$ for $CS$ and the ML-KEM secret key $s{k}_e$ for $ES$. However, $msk$ is used only during the registration phase and is not involved in deriving the session key. Although $s{k}_e$ is employed to derive $k_1$ and $H_d$, which contribute to computing the session key, the derivation of the final session key also requires the device’s secret key $s{k}_d$ to obtain $k_2$. As $s{k}_d$ is freshly generated in every session, $\mathcal{A}$ cannot recover past session keys even if the long-term secret keys are compromised. Similarly, even if long-term values derived from the user’s password are compromised, past session keys cannot be derived, because the session key computation always involves session-specific parameters that are valid only for the corresponding session. Therefore, the proposed scheme achieves perfect forward secrecy.

7.1.13. Single Point of Failure

The proposed protocol mitigates the single point of failure issue by preventing the cloud server $CS$ from acting as a permanent repository of device–edge server shared secrets. The $CS$ only assists during the registration phase and is not involved in subsequent session key establishment. Since the session key $SK=h\left(k_1\right|\left|k_2\right|\left|H_d\right|\left|T_e\right)$ is established directly between $D_i$ and $ES$ using session-specific values, compromising or disabling the $CS$ after registration does not directly expose the established session keys. Therefore, the proposed protocol reduces dependency on $CS$ and limits the impact of cloud server compromise.

7.1.14. Mutual Authentication

As discussed, the proposed scheme can defend against multiple security attacks. An adversary cannot masquerade as $D_i$ and $ES$, replay previously transmitted messages, or forge valid authentication messages. In addition, during the authentication phase, $D_i$ and $ES$ verify the authenticity of each other through the validation parameters $V_d$ and $V_e$. Therefore, the proposed scheme enables secure communication through the established session key and achieves mutual authentication.

7.2. Formal Analysis Under BAN Logic

By capturing the trust assumptions and beliefs of the involved entities, “BAN logic [20]” offers an approach for formally analyzing authentication protocols. This approach is applied to verify mutual authentication by reasoning about the freshness, message origin, and shared key beliefs [46]. In the BAN logic analysis, the protocol messages are represented in an idealized and abstract form. BAN logic does not aim to reproduce every implementation-level element of the actual protocol and focuses on the message components necessary for reasoning about authentication, freshness, and trust [47]. By applying the rules of BAN logic, we show that $D_i$ and $ES$ can reach agreement on a session key in the proposed scheme. The BAN logic notation used in this analysis is summarized in

Table 2. Notation of BAN logic.

Notation	Descriptions
$P_1,P_2$	Principals
$Ms{g}_1,Ms{g}_2$	Statements
$SK$	Session key
$P_1\mid \sim Ms{g}_1$	$P_1$ once said $Ms{g}_1$
$P_1\mid \equiv Ms{g}_1$	$P_1$ believes $Ms{g}_1$
$P_1⊲Ms{g}_1$	$P_1$ receives $Ms{g}_1$
$P_1\Rightarrow Ms{g}_1$	$P_1$ controls $Ms{g}_1$
${\left\{Ms{g}_1\right\}}_{key}$	$Ms{g}_1$ is encrypted with a key
$P_1\stackrel{key}{⟷}{P}_2$	$P_1$ and $P_2$ share a key
$\#\left(Ms{g}_1\right)$	$Ms{g}_1$ is fresh

.

7.2.1. Rules

The BAN logic rules in this analysis are presented below.

• Message meaning rule (MMR):

  $${\displaystyle \frac{P_1\mid \equiv P_1\stackrel{K}{⟷}{P}_2,P_1⊲{\left\{Ms{g}_1\right\}}_K}{P_1\mid \equiv P_2\mid \sim Ms{g}_1}}$$
• Nonce verification rule (NVR):

  $${\displaystyle \frac{P_1\mid \equiv \#\left(Ms{g}_1\right),P_1\mid \equiv P_2\mid \sim Ms{g}_1}{P_1\mid \equiv P_2\mid \equiv Ms{g}_1}}$$
• Jurisdiction rule (JR):

  $${\displaystyle \frac{P_1\mid \equiv P_2\Rightarrow Ms{g}_1,P_1\mid \equiv P_2\mid \equiv Ms{g}_1}{P_1\mid \equiv Ms{g}_1}}$$
• Belief rule (BR):

  $${\displaystyle \frac{P_1\mid \equiv (Ms{g}_1,Ms{g}_2)}{P_1\mid \equiv Ms{g}_1}}$$
• Freshness rule (FR):

  $${\displaystyle \frac{P_1\mid \equiv \#\left(Ms{g}_1\right)}{P_1\mid \equiv \#(Ms{g}_1,Ms{g}_2)}}$$

7.2.2. Goals

In BAN logic analysis, the main goal is to reveal that the communicating entities can mutually authenticate each other and share a valid session key. The goals for the scheme are listed as follows.

Goal 1: 

$D_i\mid \equiv D_i\stackrel{SK}{⟷}ES$

Goal 2: 

$D_i\mid \equiv ES\mid \equiv U_i\stackrel{SK}{⟷}ES$

Goal 3: 

$ES\mid \equiv D_i\stackrel{SK}{⟷}ES$

Goal 4: 

$ES\mid \equiv D_i\mid \equiv D_i\stackrel{SK}{⟷}ES$

7.2.3. Idealized Forms

The exchanged messages are transformed into their idealized BAN logic forms to evaluate timestamp freshness and session key agreement. In this form, each message is simplified to retain only the crucial security relevant components, namely which entity protects which information and under what keying material. The idealized messages exchanged in the proposed scheme are given below.

$Ms{g}_1$:

$D_i\to ES:{\{PI{D}_d,k_1,T_d\}}_{DS{K}_d}$

$Ms{g}_2$:

$ES\to D_j:{\{k_2,T_e\}}_{DS{K}_d}$

7.2.4. Assumptions

In BAN logic analysis, each entity is assumed to believe in the freshness of the exchanged messages, and $D_i$ is assumed to share secret key information with $ES$ after the registration phase. Based on these assumptions, the initial assumptions in BAN logic are defined as follows.

$A_1$:

$ES\mid \equiv \#\left(T_d\right)$

$A_2$:

$D_i\mid \equiv \#\left(T_e\right)$

$A_3$:

$D_i\mid \equiv D_i\stackrel{DS{K}_d}{⟷}ES$

$A_4$:

$ES\mid \equiv D_i\stackrel{DS{K}_d}{⟷}ES$

$A_5$:

$D_i\mid \equiv ES\Rightarrow (D_i\stackrel{SK}{⟷}ES)$

$A_6$:

$ES\mid \equiv D_i\Rightarrow (D_i\stackrel{SK}{⟷}ES)$

7.2.5. BAN Logic Proof

The detailed BAN logic proof of the proposed scheme is presented below.

Step 1: 

From $Ms{g}_1$, $S_1$ is obtained.

$$S_1:ES⊲{\{PI{D}_d,k_1,T_d\}}_{DS{K}_d}$$

Step 2: 

$S_2$ is derived from $S_1$ and $A_4$ using the MMR.

$$S_2:ES\mid \equiv D_i\mid \sim (PI{D}_d,k_1,T_d)$$

Step 3: 

$S_3$ is derived from $S_2$ and $A_1$ using the FR.

$$S_3:ES\mid \equiv \#(PI{D}_d,k_1,T_d)$$

Step 4: 

$S_4$ is derived from $S_2$ and $S_3$ using the NVR.

$$S_4:ES\mid \equiv D_i\mid \equiv (PI{D}_d,k_1,T_d)$$

Step 5: 

From $Ms{g}_2$, $S_5$ is obtained.

$$S_5:D_i⊲{\{k_2,T_e\}}_{DS{K}_d}$$

Step 6: 

$S_6$ is derived from $S_5$ and $A_3$ using the MMR.

$$S_6:D_i\mid \equiv ES\mid \sim (k_2,T_e)$$

Step 7: 

$S_7$ is derived from $S_6$ and $A_2$ using the FR.

$$S_7:D_i\mid \equiv \#(k_2,T_e)$$

Step 8: 

$S_8$ is derived from $S_6$ and $S_7$ using the NVR.

$$S_8:D_i\mid \equiv ES\mid \equiv (k_2,T_e)$$

Step 9: 

$S_9$ and $S_{10}$ can be derived from $S_4$ and $S_8$.

$$\begin{array}{c}\hfill S_9:D_i\mid \equiv ES\mid \equiv (D_i\stackrel{SK}{⟷}ES)\mathbf{(}\mathbf{Goal}\mathbf{2}\mathbf{)}\\ \hfill S_{10}:ES\mid \equiv D_i\mid \equiv (D_i\stackrel{SK}{⟷}ES)\mathbf{(}\mathbf{Goal}\mathbf{4}\mathbf{)}\end{array}$$

Step 10: 

Upon applying the JR to $S_9$ and $A_5$, and $S_{10}$ and $A_6$, $S_{11}$ and $S_{12}$ can be derived, respectively.

$$\begin{array}{c}\hfill S_{11}:D_i\mid \equiv (D_i\stackrel{SK}{⟷}ES)\mathbf{(}\mathbf{Goal}\mathbf{1}\mathbf{)}\\ \hfill S_{12}:ES\mid \equiv (D_i\stackrel{SK}{⟷}ES)\mathbf{(}\mathbf{Goal}\mathbf{3}\mathbf{)}\end{array}$$

7.3. Formal Analysis Under QROM

We employ “QROM [21]” to prove the session key security of the proposed scheme. In QROM, the adversary is modeled as a quantum polynomial time adversary and is allowed to make superposition queries to the $Hash$ and $KEM$ oracle [48]. Under this security model, the adversary can attempt attacks using $Send$, $Execute$, $CorruptD$, $Hash$, $KEM$, $Reveal$, and $Test$ queries. In the proposed scheme, ${\mathcal{P}}_D^{t_d}$ and ${\mathcal{P}}_{ES}^{t_e}$ denote the device and the edge server, respectively. The detailed definitions of each query are listed below.

• $Send({\mathcal{P}}^t,Msg)$: This query models an active adversary that is allowed to send a message $Msg$ to a participant and obtain the corresponding response.
• $Execute({\mathcal{P}}_D^{t_d},{\mathcal{P}}_{ES}^{t_e})$: This query models a passive adversary that is allowed to eavesdrop on a session of the protocol between ${\mathcal{P}}_D^{t_d}$ and ${\mathcal{P}}_{ES}^{t_e}$, and obtain the exchanged messages.
• $CorruptD\left({\mathcal{P}}_D^{t_d}\right)$: This query allows the adversary to compromise the device ${\mathcal{P}}_D^{t_d}$ and obtain its stored secret credentials.
• $Hash\left(\phi \right)$: This query models the hash table that stores each input-output pair. The input is given in a superposition state, and the output corresponds to the hash value. Whenever the adversary queries, the oracle $\phi$ checks whether the queried input exists in its table. If a matching record is found, the previously stored output is returned. Otherwise, a fresh random value is generated, returned, and recorded in the table.
• $KEM\left(\gamma \right)$: This query models the key encapsulation mechanism. The input and output are represented in superposition states. The operation of the oracle $\gamma$ is defined similarly to that of the hash oracle.
• $Reveal\left({\mathcal{P}}^t\right)$: This query allows the adversary to obtain the session key of an established session, except for the designated test session.
• $Test\left({\mathcal{P}}^t\right)$: This query can evaluate the security of the session key. For the test session, the adversary is given either the real session key or a random string of the same length, and its objective is to distinguish between them.

Theorem 1. 

Let $\mathcal{A}$ be a quantum polynomial time adversary against the session key security of the proposed protocol in QROM, where the session key is derived as $SK=h\left(k_1\right|\left|k_2\right|\left|H_d\right|\left|T_e\right)$. In addition, let $q_h$, $q_k$, and $q_s$ denote the numbers of $Hash$, $KEM$, and $Send$ queries issued by $\mathcal{A}$, respectively, and let λ be a security parameter. Then, the distinguishing advantage of $\mathcal{A}$ between the real session key and a random string is bounded as follows:

$$Ad{v}_{\mathcal{A}}\le {\displaystyle \frac{4q_h^2+2q_k+3q_s}{\sqrt{2^{\lambda }}}}+Ad{v}_{\mathcal{A}}^{sk},$$

where $Ad{v}_{\mathcal{A}}^{sk}$ denotes the advantage of $\mathcal{A}$ in compromising the long-term secret.

Proof. 

The above equation is proved using a sequence of games. Let $\mathrm{Pr}\left[G_i\right]$ denote the probability that $\mathcal{A}$ wins Game $G_i$ by determining whether the session key is real or randomly generated. The session key distinguishing the advantage of $\mathcal{A}$ is defined as

$$Ad{v}_{\mathcal{A}}=\left|\mathrm{Pr}\left[G_0\right]-{\displaystyle \frac{1}{2}}\right|.$$

(1)

Game 0: 

$\mathcal{A}$ performs a real attack against the proposed protocol in QROM. In this game, $\mathcal{A}$ can issue $Send$, $Execute$, $CorruptD$, $Hash$, $KEM$, $Reveal$, and $Test$ queries based on the security model. All protocol messages, the ML-KEM-derived secrets $k_1$ and $k_2$, and the final session key $SK=h\left(k_1\right|\left|k_2\right|\left|H_d\right|\left|T_e\right)$ are generated as specified in the proposed protocol.

Game 1: 

Through the $Send$ query, $\mathcal{A}$ actively injects, replays, or modifies protocol messages to produce a valid forged session transcript. Let $F_s$ denote a session-related failure event, comprising the following cases: (i) a replayed message is accepted as fresh, (ii) a transcript mismatch results in an unintended matching session, or (iii) a forged transcript is accepted as valid. If $F_s$ does not occur, then Games 0 and 1 are identical from view point of $\mathcal{A}$. By the Difference Lemma [49], we have

$$|\mathrm{Pr}\left[G_1\right]-\mathrm{Pr}\left[G_0\right]|\le \mathrm{Pr}\left[F_s\right].$$

(2)

For each $Send$ query, the probability of causing any of the three session-related undesirable events is bounded by $2^{-\lambda /2}$ in QROM [50]. Therefore, by the union bound, the probability of $F_s$ is bounded by

$$\mathrm{Pr}\left[F_s\right]\le 3q_s·2^{-\lambda /2}={\displaystyle \frac{3q_s}{\sqrt{2^{\lambda }}}}.$$

(3)

Game 2: 

In this game, the ML-KEM-derived secrets $k_1$ and $k_2$ associated with the test session are replaced with independently sampled random strings of the same lengths. This game transition is justified by the QROM security of ML-KEM. ML-KEM is constructed from an MLWE-based public-key encryption scheme using an Fujisaki–Okamoto transform, which provides chosen-ciphertext security in the QROM [51]. Therefore, the ML-KEM-derived shared secrets are computationally indistinguishable from uniformly random strings unless the adversary breaks the underlying MLWE-based hardness or the QROM security of the Fujisaki–Okamoto transform [52].

In other words, if $\mathcal{A}$ can distinguish the real ML-KEM-derived secrets $k_1$ and $k_2$ from random strings in this game transition, then $\mathcal{A}$ can also be used to distinguish the outputs of the underlying ML-KEM construction from random values. Thus, distinguishing Games 1 and 2 implies the ability to distinguish the ML-KEM shared secret from a random string, which contradicts the QROM security of ML-KEM based on the decisional MLWE assumption.

Let $F_k$ denote the event that $\mathcal{A}$ detects the replacement of either $k_1$ or $k_2$ through its interaction with the $KEM$ oracle. If $F_{kem}$ does not occur, then Games 1 and 2 are computationally indistinguishable from the viewpoint of $\mathcal{A}$. Hence,

$$|\mathrm{Pr}\left[G_2\right]-\mathrm{Pr}\left[G_1\right]|\le \mathrm{Pr}\left[F_k\right].$$

(4)

Since two independent ML-KEM-derived secrets, $k_1$ and $k_2$, are used in the test session, the KEM-related failure probability is bounded by

$$\mathrm{Pr}\left[F_k\right]\le {\displaystyle \frac{2q_k}{\sqrt{2^{\lambda }}}}.$$

(5)

Here, the factor 2 comes from the two ML-KEM-derived secrets used in the session, and the term $q_k/\sqrt{2^{\lambda }}$ captures the QROM distinguishing probability associated with the adversary’s $KEM$ oracle queries.

Game 3: 

In this game, the session key of the test session is no longer computed as $SK=h\left(k_1\right|\left|k_2\right|\left|H_d\right|\left|T_e\right)$ but is replaced with an independent random string of the same length. Let $F_h$ denote a hash-related failure event, comprising the following cases: (i) $\mathcal{A}$ queries the hash oracle on the exact test-session input $k_1\parallel k_2\parallel H_d\parallel T_e$, (ii) a hash collision occurs among the hash oracle queries, (iii) an inconsistency occurs in the random-oracle simulation, or (iv) the simulated random oracle becomes distinguishable under quantum superposition queries. If $F_h$ does not occur, then Games 2 and 3 are indistinguishable from the viewpoint of $\mathcal{A}$. We have

$$|\mathrm{Pr}\left[G_3\right]-\mathrm{Pr}\left[G_2\right]|\le \mathrm{Pr}\left[F_h\right].$$

(6)

Since $k_1$ and $k_2$ have been replaced with uniformly random strings in Game 2, the exact test-session input $k_1\parallel k_2\parallel H_d\parallel T_e$ remains hidden from $\mathcal{A}$. Under the QROM, the probability that a quantum polynomial-time adversary making $q_h$ quantum hash queries includes this exact input is conservatively upper-bounded by $q_h^2/\sqrt{2^{\lambda }}$ [53].

In addition, by the birthday bound, the probability of a hash collision among the $q_h$ hash queries is conservatively upper-bounded by $q_h^2/\sqrt{2^{\lambda }}$. The remaining oracle-simulation failure events, including oracle inconsistency and distinguishability under quantum superposition queries, are bounded by the standard QROM simulation bounds [54]. Therefore, by applying the union bound to the four hash-related sub-events, we obtain

$$\mathrm{Pr}\left[F_h\right]\le {\displaystyle \frac{4q_h^2}{\sqrt{2^{\lambda }}}}.$$

(7)

Game 4: 

In this final game, the challenger consistently responds to the $Test$ query with a uniformly random string of the same length as the session key. As $k_1$ and $k_2$ have been replaced with random strings and as the final hash output has also been replaced with a random value, the test-session response is independent of the actual session key. Therefore,

$$\mathrm{Pr}\left[G_4\right]={\displaystyle \frac{1}{2}}.$$

(8)

By the triangle inequality, we have

$$\begin{array}{cc}\hfill \left|\mathrm{Pr}\left[G_0\right]-\mathrm{Pr}\left[G_4\right]\right|& \le \left|\mathrm{Pr}\left[G_1\right]-\mathrm{Pr}\left[G_0\right]\right|+\left|\mathrm{Pr}\left[G_2\right]-\mathrm{Pr}\left[G_1\right]\right|+\left|\mathrm{Pr}\left[G_3\right]-\mathrm{Pr}\left[G_2\right]\right|\hfill \\ \hfill & \le \mathrm{Pr}\left[F_s\right]+\mathrm{Pr}\left[F_k\right]+\mathrm{Pr}\left[F_h\right].\hfill \end{array}$$

(9)

Substituting (3), (5), and (7) into (9) yields

$$\left|\mathrm{Pr}\left[G_0\right]-\mathrm{Pr}\left[G_4\right]\right|\le {\displaystyle \frac{3q_s+2q_k+4q_h^2}{\sqrt{2^{\lambda }}}}.$$

(10)

From (8), it follows that

$$\left|\mathrm{Pr}\left[G_0\right]-{\displaystyle \frac{1}{2}}\right|\le {\displaystyle \frac{3q_s+2q_k+4q_h^2}{\sqrt{2^{\lambda }}}}.$$

(11)

Long-term secret compromise: Let $\mathcal{L}$ denote the set of long-term secrets stored by the protocol entities, including password-derived authentication values, device-specific credentials, and server-side long-term secrets. We define $Ad{v}_{\mathcal{A}}^{sk}$ as the probability that $\mathcal{A}$ distinguishes the test-session key from a random key using only the compromised long-term secrets in $\mathcal{L}$, while the test session remains fresh.

Under the freshness condition, a test session is considered unfresh if the relevant long-term secrets are compromised before session establishment. In contrast, post-session compromise is allowed to capture forward secrecy. Such compromise alone does not reveal the established session key, because $SK=h(k_1\parallel k_2\parallel H_d\parallel T_e)$ also depends on the fresh ML-KEM-derived secrets $k_1$ and $k_2$. These values are generated independently for each session and erased after the session key is established. Therefore, an adversary who obtains only long-term secrets after session completion cannot reconstruct the past session key unless it also obtains the corresponding ephemeral ML-KEM-derived secrets or breaks the underlying ML-KEM security.

Therefore, considering the residual advantage caused by long-term secret compromise, the overall advantage is bounded as

$$Ad{v}_{\mathcal{A}}\le {\displaystyle \frac{4q_h^2+2q_k+3q_s}{\sqrt{2^{\lambda }}}}+Ad{v}_{\mathcal{A}}^{sk}.$$

Therefore, the proposed scheme achieves session key security in QROM. □

7.4. Formal Analysis Under AVISPA

The formal simulation tool “AVISPA [22,23]” employed to evaluate whether an authentication protocol is secure against attacks such as replay and MITM attacks under the DY channel model [55]. Therefore, the protocol is described in “High-Level Protocol Specification Language (HLPSL)”, and transformed into the “Intermediate Format (IF)” using the HLPSL2IF translator. The resulting IF specification can be analyzed using four backend tools: “On-the-Fly Model-Checker (OFMC)”, “SAT-based Model-Checker (SATMC)”, “Constraint Logic based Attack Searcher (CL-AtSe)”, and “Tree Automata based on Automatic Approximations for the Analysis of Security Protocols (TA4SP)”. Among these backends, we employ OFMC and CL-AtSe because they support the algebraic properties of XOR operations required in the protocol. The protocol specification is then verified using these two backends. The AVISPA verification was carried out using SPAN 1.6 in Oracle VirtualBox with an Ubuntu 10.10 (32-bit) virtual machine. Figure 6 presents the corresponding simulation results. The left side of the figure shows the OFMC result, and the right side shows the CL-AtSe result. In both cases, the summary indicates “SAFE”, indicating that no attack was found on the DY channel. Therefore, the proposed protocol is secure against replay and MITM attacks.

7.5. Formal Analysis Under Scyther

We further analyzed the authentication phase of the proposed scheme using “Scyther [24]”. Scyther provides an automated analysis of security protocols based on five security claim types: “Alive”, “Weak-agreement (Weakagree)”, “Non-injective agreement (Niagree)”, “Non-injective synchronization (Nisynch)”, and “Secret”. The Alive claim confirms that the intended peer has participated in the protocol run. The Weakagree claim verifies that both parties agree on the identity of the communication partner. The Niagree claim checks whether the participants agree on the relevant session data, whereas Nisynch ensures that both parties follow the same protocol flow in the correct order. Finally, the Secret claim verifies that designated values remain confidential from the adversary. In this paper, the Scyther analysis was performed using Scyther v1.2.1 on Windows 10, and compromise scenarios were evaluated using the Scyther compromise-analysis version. For the analysis, we model the authentication phase of the proposed scheme in the “Security Protocol Description Language (SPDL)” and verifies it using Scyther. As shown in Figure 7, all security claims are reported with the status “OK” and the result “no attack”. These results indicate that the proposed scheme preserves the confidentiality of the protected data and offers mutual authentication.

8. Performance Analysis

We conduct a performance analysis of the proposed scheme in terms of the computational costs, communication costs, and security properties by comparing it with authentication protocols for edge computing [18,25,26,28] and post quantum authentication protocols [29,30,31,32,33]. The analysis demonstrates that the proposed scheme offers an effective balance between efficiency and security.

8.1. Computational Costs

We compare the computational cost of the authentication phase with those of related studies [18,25,26,28,29,30,31,32,33] to validate the execution efficiency of the proposed scheme.

For the measurement of each execution time, the experiments were conducted in two different environments considering the computational characteristics of each entity. The execution time of $ES$ was measured on an Ubuntu-based environment, while the execution time of the device layer was measured on a Raspberry Pi 5 to reflect an embedded computing environment. Since devices in an edge computing system generally have lower computational capability than server-side nodes, the Raspberry Pi 5 was used to provide a practical evaluation environment for resource-constrained devices.

We implement each cryptographic function with the NTL and MIRACL libraries. The execution times of XOR and concatenation operations were ignored due to their negligible computational overhead. The multiplication and scalar addition operations used in the protocol of Kenioua et al. [18] are also negligible. In addition, the time required for the fuzzy extractor was assumed to be equal to that for ECC point multiplication [56]. We estimated the ML-KEM key generation time by applying the ratio $T_{KE}/T_{En}\approx 0.804$, which was calculated from the execution times reported in [33]. The notation, description, and execution time for each operation on $D_i$ and $ES$ are presented in

Table 3. Execution time.

Notation	Description	Device (ms)	Server (ms)
$T_h$	SHA-256 hash operation	0.0012	0.0004
$T_{gen}$	ML-KEM-512 key generation	0.4151	0.2201
$T_{en}$	ML-KEM-512 encapsulation	0.5163	0.2737
$T_{de}$	ML-KEM-512 decapsulation	0.6329	0.0344
$T_{em}$	ECC point multiplication	0.6879	0.2278
$T_{ea}$	ECC point addition	0.0021	0.0007
$T_{lenc}$	lattice encryption	8.6879	5.3705
$T_{denc}$	lattice decryption	8.5786	5.3104
$T_{senc}$	symmetric encryption	0.0005	0.0002
$T_{sdec}$	symmetric decryption	0.0001	0.00006
$T_{fe}$	fuzzy extractor	0.6879	0.2278

. In the authentication phase of the proposed scheme, $D_i$ requires $1T_{gen}+1T_{en}+1T_{de}+5T_h=1.2146$ ms and $ES$ requires $1T_{en}+1T_{de}+3T_h=0.3093$ ms. Hence, the total computational cost of the proposed scheme is 1.5239 ms. The computational costs of the related schemes are derived in the same way, and the results are given in

Table 4. Computational costs.

Scheme	Device	Edge Server	Total (ms)
Kenioua et al. [18]	$1T_h$	$1T_h$	0.0016
Jia et al. [25]	$5T_h+5T_{em}+3T_{ea}$	$5T_h+5T_{em}+3T_{ea}$	4.5949
Xie et al. [26]	$5T_h+6T_{em}+3T_{ea}$	$4T_h+4T_{em}+3T_{ea}$	5.0546
Jia et al. [28]	$8T_h+4T_{em}+2T_{ea}$	$5T_h+3T_{em}+2T_{ea}$	3.4522
Ding et al. [29]	$5T_h+2T_{lenc}+2T_{ldec}$	$3T_h+2T_{lenc}+2T_{ldec}$	55.902
Chen et al. [30]	$1T_{fe}+7T_h+1T_{lenc}+1T_{ldec}$	$4T_h+1T_{lenc}+1T_{ldec}$	28.6453
Ahmad et al. [31]	$6T_h+1T_{lenc}$	$4T_h+1T_{ldec}$	14.0071
Palaniswamy et al. [32]	$6T_h+1T_{senc}+2T_{lenc}$	$3T_h+1T_{sdec}+2T_{ldec}$	28.00556
Wen et al. [33]	$1T_{gen}+1T_{en}+1T_{de}+1T_{fe}+7T_h$	$1T_{en}+1T_{de}+6T_h$	2.2154
Proposed	$1T_{gen}+1T_{en}+1T_{de}+5T_h$	$1T_{en}+1T_{de}+3T_h$	1.5239

. Although Kenioua et al.’s scheme has a very low computational cost, requiring only two hash operations, its protocol design mainly relies on simple addition and multiplication operations. This construction may be insufficient for providing strong security guarantees and could be vulnerable to various attacks. As shown in Table 4, the proposed scheme achieves a low computational cost among the compared schemes. This result demonstrates that the proposed scheme offers improved computational efficiency in the authentication phase compared with existing approaches. In particular, even though the proposed scheme employs ML-KEM to ensure quantum resistance, the scheme imposes less computational overhead than the others. These results demonstrate that the proposed scheme is suitable for edge computing environments, where lightweight operation and strong security must be satisfied.

8.2. Communication Costs

We evaluate the communication cost of the proposed scheme compared with those of related studies [18,25,26,28,29,30,31,32,33]. Assuming an ideal communication environment, we calculate the number of bytes exchanged during the authentication phase. As SHA-256 is employed in the scheme, the hash output size is assumed to be 32 bytes. In addition, ML-KEM-512 is adopted; thus, the sizes of the public key, ciphertext, and ML-KEM key are assumed to be 800, 768, and 32 bytes, respectively. Furthermore, the sizes of the timestamp, identity, and elliptic curve point are assumed to be 4, 8, and 40 bytes, respectively. Following the parameter settings in [30], where n = 512 and q = 7,557,773, the size of one RLWE polynomial is assumed to be $n\lceil {\log }_{2}q\rceil$ = 512 × 23 = 11,776 bits (i.e., 1472 bytes).

In the authentication phase of the proposed scheme, $D_i$ transmits $DI{D}_d,c_1,p{k}_d,V_d,T_d$, whereas $ES$ sends $c_2,V_e,T_e$. The message size sent by $D_i$ is calculated as $32+768+800+32+4=1612$ bytes, and the message size sent by $ES$ is $768+32+4=804$ bytes. Therefore, the total communication cost of the proposed scheme is 2416 bytes. The communication costs of the other schemes are calculated in the same way, and

Table 5. Communication costs.

Scheme	Communication Cost (Bytes)	Number of Messages
Kenioua et al. [18]	148	3
Jia et al. [25]	192	2
Xie et al. [26]	184	2
Jia et al. [28]	192	2
Ding et al. [29]	3152	3
Chen et al. [30]	3144	2
Ahmad et al. [31]	3232	4
Palaniswamy et al. [32]	3040	2
Wen et al. [33]	2540	3
Proposed	2416	2

summarizes the comparison results. Although the schemes of Sahoo et al., Shukla et al., and Kenioua et al. realized lower communication costs, they were not designed for the quantum system. Therefore, they cannot offer security against future quantum attacks and remain vulnerable to the harvest now, decrypt later threat. Therefore, the proposed scheme is efficient in that it provides quantum resistance while maintaining a low communication cost.

8.3. Security Features

We evaluate the security of the proposed scheme via a comparison with related studies [18,25,26,28,29,30,31,32,33]. The considered security properties include robustness against attacks (i.e., quantum, impersonation, DoS, and stolen device attacks), and realize anonymity, untraceability, perfect forward secrecy, and mutual authentication.

Table 6. Security features.

Security Feature	[18]	[25]	[26]	[28]	[29]	[30]	[31]	[32]	[33]	Proposed
Stolen device attacks	○	○	○	○	×	○	○	○	○	○
Impersonation attacks	×	×	○	×	○	○	○	○	○	○
Offline guessing attacks	×	×	○	×	×	○	○	○	○	○
Insider attacks	○	×	○	○	×	○	○	○	○	○
Privileged insider attacks	×	○	○	○	○	○	○	×	○	○
Replay and MITM attacks	○	×	○	○	○	×	○	○	○	○
Stolen verifier attacks	×	○	○	○	×	-	○	○	-	○
ESL attacks	×	○	×	×	○	○	○	○	○	○
DoS attacks	×	×	○	○	×	○	×	○	○	○
Quantum attacks	×	×	×	×	○	○	○	○	○	○
Anonymity	×	×	○	○	○	○	×	×	○	○
Untraceability	×	○	○	○	×	○	×	×	○	○
Perfect forward secrecy	-	×	○	○	-	○	-	○	○	○
Mutual authentication	×	○	×	×	×	○	○	○	○	○

×: No guarantee, ○: Guarantee, -: Not considered.

summarizes the comparison results. In the table, the proposed scheme achieves quantum resistance while maintaining efficiency and provides enhanced security properties.

Therfore, the proposed protocol provides an efficient and secure design for edge computing environments by reducing computational and communication costs while maintaining strong security properties. Unlike the schemes in [18,25,26,28], which rely on conventional cryptographic primitives, the proposed protocol adopts ML-KEM to provide quantum-resistant key establishment. Although the schemes in [29,30,31,32] employ lattice-based cryptographic mechanisms, they require costly lattice encryption operations involving polynomial multiplications, noise sampling, and modular arithmetic. The proposed protocol also differs structurally from the scheme in [33]. While Ref. [33] employs an ML-KEM-based mechanism, it requires a three-round authentication procedure. In contrast, the proposed protocol completes mutual authentication and session key establishment within two communication rounds. In addition, the use of a dynamic pseudonym-based identifier enables the edge server to retrieve the corresponding device record while preserving user anonymity.

9. Conclusions

In this paper, we propose a quantum safe mutual authentication scheme for edge computing systems. The proposed scheme was designed to address the security weaknesses of Kenioua et al.’s scheme, which is vulnerable to various attacks including impersonation, offline password guessing, stolen verifier, and privileged insider attacks and fails to guarantee anonymity and untraceability. We also incorporate ML-KEM, a standardized PQC mechanism to provide quantum resistance. This approach is critical because many existing authentication schemes still rely on classical public key cryptography, which may not be secure against quantum attacks. Therefore, edge computing systems require authentication schemes that are secure in the quantum era. The proposed scheme is resistant against quantum attacks and various classical attacks, including stolen device, MITM, replay, and ESL attacks. In addition, the proposed scheme achieves anonymity, untraceability, perfect forward secrecy, and mutual authentication. To validate its security, we analyzed the session key agreement between the device and the edge server using BAN logic and verified the security of the proposed scheme in QROM. Furthermore, we employed AVISPA and Scyther to show that the proposed scheme is robust against replay and MITM attacks on the DY channel. Performance comparisons with existing related schemes showed that the proposed scheme improves security, including quantum resistance, while maintaining communication and computational efficiency. Although the proposed scheme provides secure mutual authentication against various attacks, side-channel resistance is not considered in the current design. In edge computing environments, side-channel attacks can be launched by collecting traffic patterns, power consumption traces, and using inference techniques to extract sensitive information. For future work, we plan to extend the proposed protocol by considering side-channel-resistant design techniques and implementing the scheme in a practical edge computing testbed with multiple devices and edge servers. Such experiments can facilitate evaluating the scalability, computation time, communication overhead, and applicability under realistic network conditions. We also plan to investigate the extension of our scheme to more dynamic edge environments, such as device mobility and multi-edge server scenarios.