Universal Encryption of Individual Sequences Under Maximal Information Leakage

Abstract

We consider the Shannon cipher system in the framework of individual sequences and finite-state encrypters under the metric of maximal information leakage. A lower bound and an asymptotically matching upper bound on the leakage are derived, which lead to the conclusion that asymptotically minimum leakage can be attained by Lempel–Ziv compression followed by one-time pad encryption of the compressed bitstream.

1. Introduction

The information-theoretic approach that combines individual-sequence modeling with finite-state encoders and decoders has been extensively developed, representing a notable departure from the conventional reliance on probabilistic models traditionally used in source and channel modeling. This paradigm shift has gained traction across multiple areas of information theory, including lossless and lossy source coding [1,2,3,4,5,6]; source/channel simulation [7]; hypothesis testing [8,9]; prediction and decision making [10,11,12]; filtering [13]; and even error correction coding [14,15,16]. A concise overview of this expanding body of work can be found in [17], though these citations represent only a small fraction of the broader literature. By sharp contrast, the domain of information-theoretic security has remained largely anchored in probabilistic methods from Shannon’s foundational contributions [18] to more contemporary developments [19,20,21,22,23]. While these examples are far from exhaustive, they underscore the field’s persistent adherence to probabilistic frameworks.

To the author’s knowledge, only two significant departures from the dominant probabilistic approach in information-theoretic security exist: one is an unpublished technical report by Ziv [24] and the other is a subsequent development documented in [25]. In his report, Ziv proposes a novel framework in which the plaintext, destined for encryption using a secret key, is modeled as an arbitrary individual sequence. Within this setup, the encrypter functions as a general block encoder, while the eavesdropper is equipped with a finite-state machine (FSM) designed to distinguish between potential candidates for estimating the plaintext. A central aspect of Ziv’s model is the assumption that the eavesdropper possesses partial prior knowledge about the plaintext, formalized as a set of “acceptable messages”, which he defines as the acceptance set. Before intercepting the ciphertext, the eavesdropper’s uncertainty is characterized by the possibility that the plaintext could be any member of this set. Encryption is deemed perfectly secure if the ciphertext provides no additional information, i.e., if it does not reduce the size of the acceptance set, and thus renders the eavesdropper’s uncertainty unchanged. Accordingly, the size of the acceptance set quantifies uncertainty: the larger the set, the less the eavesdropper knows. The FSM attempts to rule out unacceptable sequences by testing them across different key sequences. Perfect secrecy is thus defined by the ciphertext’s inability to eliminate any members of the acceptance set. Ziv’s key result is that the minimum asymptotic key rate required for perfect secrecy, under this definition, is lower-bounded by the Lempel–Ziv (LZ) complexity of the plaintext sequence [6]. Furthermore, this lower bound is asymptotically tight. Perfect security can be achieved by applying a one-time pad (bitwise XOR with key bits) to the LZ-compressed version of the plaintext. This mirrors Shannon’s classical finding that the key rate must match the entropy rate of the source. More recent work [26] has expanded and sharpened Ziv’s original ideas in several important respects.

The subsequent work [25] offers a different take on the modeling approach and on achieving perfect secrecy for individual sequences. Instead of focusing on a finite-state eavesdropper with predefined knowledge, this approach models the encrypter itself as a finite-state machine (FSM), which processes both the plaintext and a stream of random key bits in a sequential manner. Central to this framework is the introduction of a new notion called finite-state encryptability, in the footsteps of finite-state compressibility introduced in [6]. Finite-state encryptability is defined as the minimum key rate required by any FSM-based encrypter to ensure that a particular measure of normalized empirical mutual information between the plaintext and ciphertext converges to zero as the block length increases. One of the main theoretical results in [25] asserts that the finite-state encryptability of a given individual sequence is lower-bounded by its finite-state compressibility. Stated differently, no finite-state encrypter can use a key rate below the sequence compressibility without compromising security, as defined in this setting. Once again, this lower bound is not merely theoretical; it can be asymptotically achieved through the same two-step process that was mentioned above: first compressing the plaintext using the Lempel–Ziv (LZ) algorithm and then applying one-time pad encryption to the resulting compressed bitstream. This mirrors earlier results in both compression and security, illustrating a deep connection between the two domains.

In this paper, we adopt the same model setting as in [25], but with a different security metric: the maximum leakage of information, which was first introduced by Issa, Wagner, and Kamath in [27] and then further explored in several more recent works, including [28,29,30,31,32], among others. This metric is closely related to, and similarly motivated by, the earlier security measure proposed in [33], which defines security as a scenario where the correct decoding exponent of the plaintext is not improved by the availability of the ciphertext, compared to that of blind guessing. For more details, see the last paragraph of Section 2.2.1. The maximum leakage metric is defined in a more general form and has a relatively straightforward expression, as demonstrated in [27] and further clarified in the following sections. As will be discussed in the sequel, the maximum leakage metric is particularly well suited for the individual-sequence setting considered here, as it is weakly dependent on the probability distribution of the plaintext, depending only on its support.

We derive both a lower bound and an asymptotically matching upper bound on the leakage, leading yet again to the conclusion that asymptotically optimal performance can be achieved by applying LZ compression followed by one-time pad encryption of the compressed bitstream. Thus, considering the above-mentioned earlier works, refs. [24,25,26], one of the messages of this work is that one-time pad encryption on top of LZ compression forms an asymptotically optimal cipher system from many aspects. Therefore, we believe that the deeper and more interesting contribution of this work is the converse theorem (Theorem 1 in the sequel) and its proof, asserting that the key rate that must be consumed to encrypt an individual sequence cannot be much smaller than the LZ complexity of the sequence minus the allowed normalized maximal information leakage.

This paper is structured as follows: In Section 2, we establish notation conventions, provide some necessary background, and formulate the problem studied in this work. In Section 3, we assert the main results and discuss them. Finally, in Section 4, we prove Theorem 1, which is the converse theorem.

2. Notation Conventions, Background, and Problem Formulation

2.1. Notation Conventions

In this paper, we adopt the following notation rules: Scalar random variables (RVs) are represented using uppercase letters, while their realizations (sample values) are denoted by the corresponding lowercase letters. The sets of possible values (alphabets) for these variables are indicated using calligraphic letters. This notation extends naturally to random vectors and their realizations. Specifically, an n-dimensional random vector will be denoted by appending a superscript indicating the dimension to the scalar symbol. For instance, $A^n$ (n–positive integer) refers to the random vector $(A_1,\dots ,A_n)$, and $a^n=(a_1,\dots ,a_n)$ denotes a particular instance of this vector, belonging to the set ${\mathcal{A}}^n$, the n-fold Cartesian product of the alphabet $\mathcal{A}$. Segment notations, such as $A_i^j$ and $a_i^j$, are used to represent substrings $(A_i,\dots ,A_j)$ and $(a_i,\dots ,a_j)$, respectively, for integers $i\le j$. When $i=1$, the index is dropped for brevity. If $i>j$, these notations are interpreted as representing the empty string. Additionally, for any real number u, the notation ${\left[u\right]}_{+}$ denotes $max0, u$. Unless otherwise noted, all logarithms and exponential functions throughout this paper are taken to base 2.

Throughout this article, information sources and channels will be generically represented by the letter P, following standard textbook notation. Subscripts will indicate the relevant random variables and any sort of conditioning, when applicable. For example, $P_{X^n}\left(x^n\right)$ denotes the probability mass function of the random vector $X^n$ evaluated at $x^n$, while $P_{Y^n|X^n}\left(y^n\right|x^n)$ represents the conditional probability of $Y^n=y^n$ given $X^n=x^n$. These subscripts may be omitted when the meaning is clear from context. Information-theoretic functionals such as entropy, mutual information, and related quantities will be expressed using standard symbols and conventions widely adopted in the information theory literature. In the remainder of this work, the symbol $x^n=(x_1,\dots ,x_n)$ will refer to a specific input sequence intended for encryption. Each element $x_i$, $i=1,2,\dots ,n$, belongs to a finite input alphabet $\mathcal{X}$, whose size is denoted by $\alpha$.

2.2. Background

Before showing the main results and their proofs, let us revisit key terms and details related to the notion of maximal information leakage and the 1978 version of the LZ algorithm, also known as the LZ78 algorithm [6], which is the central building block in this work.

2.2.1. Maximal Leakage of Information

As mentioned in the introduction, we adopt the maximal leakage [27] as our secrecy metric. For a probabilistic plaintext source, the maximal leakage from a secret random variable X, distributed according to $\{P_X\left(x\right),x\in \mathcal{X}\}$, to another random variable Y, available to an adversary, and which is conditionally distributed given $X=x$ according to $\{P_{Y|X}\left(y\right|x),x\in \mathcal{X},y\in \mathcal{Y}\}$, is defined as

$$\mathcal{L}(X\to Y)\stackrel{▵}{=}\underset{U-X-Y-\widehat{U}}{sup}log{\displaystyle \frac{Pr\{\widehat{U}=U\}}{{max}_{u\in \mathcal{U}}{P}_U\left(u\right)}},$$

(1)

where the supremum is over all finite-alphabet random variables U and $\widehat{U}$, with the Markov structure $U-X-Y-\widehat{U}$. In other words, it is the maximum possible difference between the logarithm of the probability of guessing correctly some (possibly randomized) function of X based on Y and correctly guessing it blindly.

In Theorem 1 of [27], it was asserted and proved that the leakage can be calculated relatively easily using the following formula:

$$\mathcal{L}(X\to Y)=log\left[\sum _{y\in \mathcal{Y}}\underset{\{x:P_X\left(x\right)>0\}}{max}{P}_{Y|X}\left(y\right|x)\right].$$

Clearly, if $P_{Y|X}\left(y\right|x)$ is independent of x for all $y\in \mathcal{Y}$, then $\mathcal{L}(X\to Y)=0$, which is the case of perfect secrecy. In general, the smaller the $\mathcal{L}(X\to Y)$, the more secure the system is. In [27], it is shown that the maximal leakage has many interesting properties; one of them is that it satisfies a data processing inequality (see Lemma 1 of [27]). It is also shown in Section III of [27] that the maximal leakage has several additional operative meanings in addition to the original one explained above.

Note that the dependence on the distribution of the secret random variable, $P_X$, is rather weak, as it depends only on its support. When passing from single variables to vectors of length n, $\mathcal{L}(X^n\to Y^n)$ is defined in the same manner except that x, y, $\mathcal{X}$, $\mathcal{Y}$, $P_X(·)$, and $P_{Y|X}(·|·)$ are replaced by $x^n$, $y^n$, ${\mathcal{X}}^n$, ${\mathcal{Y}}^n$, $P_{X^n}(·)$, and $P_{Y^n|X^n}(·|·)$, respectively. In this case, the weak dependence of $\mathcal{L}(X^n\to Y^n)$ on $P_{X^n}$ makes it natural to use when $P_{X^n}$ is uncertain, or completely unknown, or even non-existent, such as in the individual sequence setting considered here. In this case, we adopt the simple definition

$$\mathcal{L}(x^n\to Y^n)\stackrel{▵}{=}log\left[\sum _{y^n\in {\mathcal{Y}}^n}\underset{x^n\in {\mathcal{X}}^n}{max}{P}_{Y^n|X^n}\left(y^n\right|x^n)\right],$$

(2)

corresponding to the full support ${\mathcal{X}}^n$ for $x^n$, which accounts for a worst-case approach. The operational significance of maximal information leakage in our setting can then be understood in two ways: (i) Considering the definition (1), it allows arbitrary probability distributions (without any assumed structure) on $x^n$, including those that place almost all their mass on a single (unknown) arbitrary sequence, with regard to the individual-sequence setting considered here. (ii) Referring to Formula (2), it is evident that the leakage vanishes whenever $P_{Y^n|X^n}\left(y^n\right|x^n)$ is independent of $x^n$, which is an indisputable characterization for perfect secrecy in the individual-sequence setting too, where no distribution is assumed on $x^n$.

As mentioned in the introduction, a somewhat different security metric was proposed in [33], but it is intimately related to the maximal information leakage considered here. In [33], the idea was to define a system as secure if the probability of guessing X correctly is essentially the same if Y is present or absent. More precisely, if X and Y are random vectors of dimension n, then a system is considered secure if the correct decoding exponent of X in the presence of Y is the same as if Y is absent. Specifically, the correct decoding probability of X based on Y is

$$P_c=\sum _y\underset{x}{max}{P}_{XY}(x,y),$$

(3)

which is closely related to

$$\begin{array}{ccc}\hfill 2^{\mathcal{L}(X\to Y)}& =& \sum _y\underset{x}{max}{P}_{Y|X}\left(y\right|x)\hfill \\ & =& \left|\mathcal{X}\right|·\sum _y\underset{x}{max}{\displaystyle \frac{P_{Y|X}\left(y\right|x)}{\left|\mathcal{X}\right|}}\hfill \\ & \stackrel{▵}{=}& \left|\mathcal{X}\right|·\sum _y\underset{x}{max}{P}_X\left(x\right)P_{Y|X}\left(y\right|x)\hfill \\ & =& {\displaystyle \frac{{\sum }_y{max}_x{P}_X\left(x\right)P_{Y|X}\left(y\right|x)}{1/\left|\mathcal{X}\right|}}\hfill \\ & =& {\displaystyle \frac{P_c^i}{P_c^u}},\hfill \end{array}$$

(4)

where $P_X(·)$ is understood to designate the uniform distribution across $\mathcal{X}$; accordingly, $P_c^i$ stands for the probability of correct decoding of a uniformly distributed X by an informed observer, namely one that has access to Y, whereas $P_c^{\mathrm{u}}=1/\left|\mathcal{X}\right|$ denotes the probability of correctly blind guessing the value of X (in the absence of Y).

2.2.2. Lempel–Ziv Parsing

The incremental parsing process of the LZ78 algorithm is a sequential method applied to an input vector $x^n$ over a finite alphabet. In this process, each new phrase is the shortest substring that has not appeared before as a complete parsed phrase, except possibly for the final (incomplete) phrase. For instance, if one applies incremental parsing to the sequence $x^{15}=\mathrm{abbabaabbaaabaa}$, the outcome is $\mathrm{a},\mathrm{b},\mathrm{b}\mathrm{a},\mathrm{b}\mathrm{a}\mathrm{a},\mathrm{b}\mathrm{b},\mathrm{a}\mathrm{a},\mathrm{a}\mathrm{b},\mathrm{a}\mathrm{a}$. Let $c\left(x^n\right)$ designate the total number of phrases formed from $x^n$ by the incremental parsing process (in the example above, $c\left(x^{15}\right)=8$). Also, let $LZ\left(x^n\right)$ stand for the length of the LZ78 binary compressed representation for $x^n$. Theorem 2 of [6] easily leads to the following inequality:

$$LZ\left(x^n\right)\le c\left(x^n\right)logc\left(x^n\right)+n·ϵ\left(n\right),$$

(5)

where $ϵ\left(n\right)$ tends to zero as $n\to \infty$. In other words, the LZ code-length for $x^n$ cannot exceed an expression whose dominant term is $c\left(x^n\right)logc\left(x^n\right)$. On the other hand, it turns out that $c\left(x^n\right)logc\left(x^n\right)$ is also the dominant term of a lower bound (see Theorem 1 of [6]) to the minimum code-length attainable by any information lossless finite-state encoder with no more than s states, provided that $log\left(s^2\right)$ is very small compared to $logc\left(x^n\right)$. In view of these facts, we will be referring to $c\left(x^n\right)logc\left(x^n\right)$ as the unnormalized LZ complexity of $x^n$, whereas the normalized LZ complexity will be defined as

$${\rho }_{\mathrm{LZ}}\left(x^n\right)\stackrel{▵}{=}{\displaystyle \frac{c\left(x^n\right)logc\left(x^n\right)}{n}}.$$

(6)

2.3. Problem Formulation

Following the approach in [25], we adopt a finite-state model for encryption, described by the sextuple

$$E=(\mathcal{X},\mathcal{Y},\mathcal{Z},f,g,\Delta ),$$

where $\mathcal{X}$ is a finite input alphabet with cardinality $\alpha =\left|\mathcal{X}\right|$, $\mathcal{Y}$ is a finite collection of binary strings of variable length, possibly including the empty string $\lambda$ (with zero length), $\mathcal{Z}$ is a finite set representing the internal states of the encrypter, $f:\mathcal{Z}\times \mathcal{X}\times {0,1}^{*}\to \mathcal{Y}$ is the output function, $g:\mathcal{Z}\times \mathcal{X}\to \mathcal{Z}$ is the state transition function, and $\Delta :\mathcal{Z}\times \mathcal{X}\to 0,1,2,\dots$ indicates the number of key bits used at each step. The encrypter E processes two infinite input sequences: a plaintext sequence $\mathbf{x}=x_1,x_2,\dots$, where each $x_i\in \mathcal{X}$, and a key sequence $\mathbf{u}=u_1,u_2,\dots$, where each $u_i\in 0,1$. Given these inputs, the encrypter generates an infinite ciphertext sequence $\mathit{y}=y_1,y_2,\dots$, with each $y_i\in \mathcal{Y}$, while simultaneously transitioning through a corresponding state sequence $\mathit{z}=z_1,z_2,\dots$, where each $z_i\in \mathcal{Z}$. The evolution of these sequences is governed by a set of recursive equations, applied iteratively for each time step $i=1,2,\dots$

$$\begin{array}{ccc}\hfill t_i& =& t_{i-1}+\Delta (z_i,x_i),t_0\stackrel{▵}{=}0\hfill \end{array}$$

(7)

$$\begin{array}{ccc}\hfill k_i& =& (u_{t_{i-1}+1},u_{t_{i-1}+2},\dots ,u_{t_i})\hfill \end{array}$$

(8)

$$\begin{array}{ccc}\hfill y_i& =& f(z_i,x_i,k_i)\hfill \end{array}$$

(9)

$$\begin{array}{ccc}\hfill z_{i+1}& =& g(z_i,x_i).\hfill \end{array}$$

(10)

Here, the initial state of the encrypter, $z_1$, is fixed and will be referred to as $z_{★}$ throughout. It is understood that when $\Delta (z_i,x_i)=0$, the encrypter uses no key bits at step i. In this case, the key fragment $k_i$ is defined to be the empty string $\lambda$. Similarly, if the output $y_i=\lambda$, then no output is generated at that step; the system effectively idles, meaning only the internal state changes in response to the input. More specifically, at each time step i, given the current state $z_i$ and the incoming plaintext symbol $x_i$, the encrypter consumes the next $\Delta (z_i,x_i)$ unused bits from the key sequence u to form $k_i$. It then produces an output $y_i$ (which may be empty) and transitions to the next state $z_{i+1}$ based on the function g. In summary, the operation at time i proceeds as follows: The encrypter is in state $z_i$, it receives input symbol $x_i$; it consumes $\Delta (z_i,x_i)$ key bits from u, forming $k_i$; it generates output $y_i$; and it updates its state to $z_{i+1}$.

Remark 1. 

Note that the evolution of the state variable $z_i$ depends solely on the source inputs $\left\{x_i\right\}$ and is independent of the key bits. This design choice reflects the intended role of $z_i$, which is to retain memory of the source sequence $x^n$, allowing the encrypter to exploit empirical correlations and repetitive patterns within the plaintext. In contrast, maintaining memory of past key bits—which are assumed to be independent and identically distributed (i.i.d.)—offers no practical benefit and is therefore omitted. Moreover, the model can be naturally extended to include two state variables: one that evolves based only on the source sequence $\left\{x_i\right\}$ (as in the current setup) and another that evolves based on both $\left\{x_i\right\}$ and the consumed key bits $\left\{k_i\right\}$. In such a framework, the first state variable would continue to govern the update of the index $t_i$, while the second could influence the output function, allowing for more expressive or adaptive encryption mechanisms.

An encrypter with s states, or an s-state encrypter, E, is one with $\left|\mathcal{Z}\right|=s$. It is assumed that the plaintext sequence x is deterministic (i.e., an individual sequence), whereas the key sequence u is purely random, i.e., for every positive integer n, $P_{U^n}\left(u^n\right)=2^{-n}$.

A few additional notation conventions will be convenient: By $f(z_i,x_i^j,k_i^j)$, ($i\le j$) we refer to the vector $y_i^j$ produced by E in response to the inputs $x_i^j$ and $k_i^j$ when the initial state is $z_i$. Similarly, the notation $g(z_i,x_i^j)$ will mean that the state $z_{j+1}$ and $\Delta (z_i,x_i^j)$ will designate ${\sum }_{\mathit{\ell }=i}^j\Delta (z_{\mathit{\ell }},x_{\mathit{\ell }})$ under the same circumstances.

As explained in Section 2.2, we adopt the maximal leakage of information as our security metric, given by

$$\mathcal{L}(x^n\to Y^n)\stackrel{▵}{=}log\left[\sum _{y^n}\underset{x^n\in {\mathcal{X}}^n}{max}{P}_{Y^n|X^n}\left(y^n\right|x^n)\right].$$

An encryption system E is said to be perfectly secure if for every positive integer n, $\mathcal{L}(x^n\to Y^n)=0$. If $\mathcal{L}(x^n\to Y^n)\to 0$ as $n\to \infty$, we say that the encryption system is asymptotically secure.

An encrypter is referred to as information lossless (IL) if for every $z_i\in \mathcal{Z}$, every sufficiently large n and all pairs $(x_i^{i+n},k_i^{i+n})$, the quadruple $(z_i,k_i^{i+n},f(z_i,x_i^{i+n},k_i^{i+n}),g(z_i,x_i^{i+n}))$ uniquely determines $x_i^{i+n}$. Given an encrypter E and an input string $x^n$, the encryption key rate of $x^n$ with respect to E is defined as

$${\sigma }_E\left(x^n\right)\stackrel{▵}{=}{\displaystyle \frac{\mathit{\ell }\left(k^n\right)}{n}}={\displaystyle \frac{1}{n}}\sum _{i=1}^n\mathit{\ell }\left(k_i\right),$$

(11)

where $\mathit{\ell }\left(k_i\right)=\Delta (z_i,x_i)$ is the length of the binary string $k_i$ and $\mathit{\ell }\left(k^n\right)={\sum }_{i=1}^n\mathit{\ell }\left(k_i\right)$ is the total length of $k^n$.

Remark 2. 

It is worth noting that the definition of information losslessness used here is more relaxed and, thus, more general than the one given in [6]. In [6], the requirement must hold for every positive integer n, whereas in the present context, it is only required to hold for all sufficiently large n. The absence of information losslessness in the stricter sense of [6] does not contradict the ability of the legitimate decoder to reconstruct the source. Rather, it implies that reconstructing $x^n$ may require more than just the tuple $(z_i,y_i^{i+n},k_i^{i+n},z_{i+n+1})$; for example, some additional data from times later than $i+n+1$ may be needed.

The set of all perfectly secure, IL encrypters $\left\{E\right\}$ with no more than s states will be denoted by $\mathcal{E}\left(s\right)$. The minimum of ${\sigma }_E\left(x^n\right)$ over all encrypters in $\mathcal{E}\left(s\right)$ will be denoted by ${\sigma }_s\left(x^n\right)$, i.e.,

$${\sigma }_s\left(x^n\right)=\underset{E\in \mathcal{E}\left(s\right)}{min}{\sigma }_E\left(x^n\right).$$

(12)

Finally, let

$${\sigma }_s\left(\mathit{x}\right)=\underset{n\to \infty }{lim\; sup}{\sigma }_s\left(x^n\right),$$

(13)

and define the finite-state encryptability of x as

$$\sigma \left(\mathit{x}\right)=\underset{s\to \infty }{lim}{\sigma }_s\left(\mathit{x}\right).$$

(14)

Our purpose is to characterize these quantities and to point out how they can be achieved in principle.

3. Main Results

Our converse theorem, whose proof appears in Section 4, is the following:

Theorem 1. 

For every information lossless encrypter E with no more than s states,

$$\begin{array}{ccc}\hfill {\displaystyle \frac{\mathcal{L}(x^n\to Y^n)}{n}}& \ge & [\underset{x^n\in {\mathcal{X}}^n}{max}\{{\rho }_{LZ}\left(x^n\right)-{\sigma }_E\left(x^n\right)\}-\hfill \\ & & {\delta }_s\left(n\right)-{\displaystyle \frac{(\alpha s-1)log(n+1)}{n}}-{\displaystyle \frac{logs}{n}}{]}_{+},\hfill \end{array}$$

(15)

where ${\delta }_s\left(n\right)\le O\left({\displaystyle \frac{log(logn)}{logn}}\right)$ for every fixed s. Equivalently, if $0\le \mathcal{L}(x^n\to Y^n)\le n\lambda$ for some given constant $\lambda \ge 0$, then for every $x^n\in {\mathcal{X}}^n$ and every information lossless encrypter $E\in \mathcal{E}\left(s\right)$,

$${\sigma }_E\left(x^n\right)\ge {\rho }_{LZ}\left(x^n\right)-\lambda -{\delta }_s\left(n\right)-{\displaystyle \frac{(\alpha s-1)log(n+1)}{n}}-{\displaystyle \frac{logs}{n}}.$$

(16)

As for achievability, consider first an arbitrary lossless compression scheme that compresses $x^n$ at a compression ratio of $\rho \left(x^n\right)=L\left(x^n\right)/n$ and then applies one-time pad encryption to ${[L\left(x^n\right)-n\lambda ]}_{+}$ compressed bits. Let $y^n$ denote the resulting (partially) encrypted compressed representation of $x^n$. Since the ${[L\left(x^n\right)-n\lambda ]}_{+}$ key bits are purely random, the probablity of any $y^n$ that can be obtained from some $x^n$ is exactly $2^{-{[L\left(x^n\right)-n\lambda ]}_{+}}$ and zero if $y^n$ cannot be obtained from any $x^n$. In other words, ${max}_{x^n}{P}_{Y^n|X^n}\left(y^n\right|x^n)\le 2^{-{[L\left(x^n\right)-n\lambda ]}_{+}}$. Obviously, the length of $y^n$, denoted as $L\left(y^n\right)$, is equal to $L\left(x^n\right)$. Therefore, by denoting $L_{max}={max}_{x^n\in {\mathcal{X}}^n}L\left(x^n\right)$, we have the following:

$$\begin{array}{ccc}\hfill {exp}_2\left\{\mathcal{L}(x^n\to y^n)\right\}& =& \sum _{y^n\in {\mathcal{Y}}^n}\underset{x^n}{max}{P}_{Y^n|X^n}\left(y^n\right|x^n)\hfill \\ & =& \sum _{\mathit{\ell }=1}^{L_{max}}\sum _{\{y^n:L\left(y^n\right)=\mathit{\ell }\}}\underset{x^n}{max}{P}_{Y^n|X^n}\left(y^n\right|x^n)\hfill \\ & \le & \sum _{\mathit{\ell }=1}^{L_{max}}\sum _{\{y^n:L\left(y^n\right)=\mathit{\ell }\}}{2}^{-{[\mathit{\ell }-n\lambda ]}_{+}}\hfill \\ & \le & \sum _{\mathit{\ell }=1}^{L_{max}}{2}^{\mathit{\ell }}{2}^{-{[\mathit{\ell }-n\lambda ]}_{+}}\hfill \\ & \le & \sum _{\mathit{\ell }=1}^{L_{max}}{2}^{\mathit{\ell }}{2}^{-(\mathit{\ell }-n\lambda )}\hfill \\ & =& L_{max}·2^{n\lambda },\hfill \end{array}$$

(17)

and, therefore,

$$\mathcal{L}(x^n\to y^n)\le n\lambda +log{L}_{max}.$$

(18)

If $L_{max}=O\left(n\right)$, then the dominant term is clearly $n\lambda$.

Remark 3. 

The condition that $L_{max}=O\left(n\right)$ is always easy to satisfy via a minor modification of any given compression scheme (if it does not satisfy the condition in the first place). First, test whether $L\left(x^n\right)<\lceil nlog\alpha \rceil$ or $L\left(x^n\right)\ge \lceil nlog\alpha \rceil$. If $L\left(x^n\right)<\lceil nlog\alpha \rceil$, add a header bit ‘0’ before the compressed representation of $x^n$; otherwise, add a header bit ‘1’ and then add the uncompressed binary representation of $x^n$ using $\lceil nlog\alpha \rceil$ bits. The resulting code-length would then be $L^{\prime }\left(x^n\right)=min\{L\left(x^n\right),\lceil nlog\alpha \rceil \}+1$ bits.

If the compression scheme is chosen to be the LZ78 algorithm, then

$${\sigma }_E\left(x^n\right)\le {\rho }_{\mathrm{LZ}}\left(x^n\right)-\lambda +O\left({\displaystyle \frac{loglogn}{logn}}\right),$$

(19)

which essentially meets the converse bound (16). We have therefore proved the following direct theorem:

Theorem 2. 

Given $\lambda \ge 0$, there exists a universal encrypter that satisfies

$$\mathcal{L}(x^n\to Y^n)\le n\lambda +logn+O\left(1\right),$$

(20)

and for every $x^n\in {\mathcal{X}}^n$,

$${\sigma }_E\left(x^n\right)\le {\rho }_{LZ}\left(x^n\right)-\lambda +O\left({\displaystyle \frac{loglogn}{logn}}\right).$$

(21)

Discussion.

• A few comments are now in order.

• We established both a lower bound and an asymptotically matching upper bound on the information leakage, leading once again to the conclusion that asymptotically optimal performance can be achieved by applying Lempel–Ziv (LZ) compression followed by one-time pad encryption of the compressed bitstream. As discussed in the introduction, together with earlier works, such as [24,25,26], this reinforces the message that one-time pad encryption applied after LZ compression yields an asymptotically optimal cipher system in several important respects. That said, we believe the deeper and more significant contribution of this work lies in the converse theorem (Theorem 1), which shows that the key rate required to securely encrypt an individual sequence cannot be substantially smaller than its LZ complexity minus the permitted normalized maximal information leakage, no matter what encryption strategy is employed.
• Similarly as in [6], there is formally a certain gap between the converse theorem and the achievability scheme in its basic form, when examined from the viewpoint of the number of states, s, relative to n. While s should be small relative to n for the lower bound to be essentially ${\rho }_{\mathrm{LZ}}\left(x^n\right)$ (see Section 2.2 above), the number of states actually needed to implement LZ78 compression for a sequence of length n is basically exponential in n. In [6], the gap is closed in the limit of $s\to \infty$ (after taking the limit $n\to \infty$) by subdividing the sequence into blocks and restarting the LZ algorithm at the beginning of every block. A similar comment applies here too in the double limit of achieving $\sigma \left(x\right)$.
• As discussed in [26] in a somewhat different context, for an alternative to the use of the LZ78 algorithm, it can be shown that asymptotically optimum performance can also be attained by a universal compression scheme for the class of k-th order Markov sources, where k is chosen to be sufficiently large. In this case, ${\rho }_{\mathrm{LZ}}\left(x^n\right)$ in Theorems 1 and 2 should be replaced by the k-th order empirical entropy of order k, and some redundancy terms should be modified. However, one of these redundancy terms is ${\displaystyle \frac{logs}{k+1}}$, which means that in order to compete with the best encrypter with s states, k must be chosen to be significantly larger than $logs$, so as to make this term reasonably small.
• It is speculated that it may not be difficult to extend our findings in several directions, including lossy reconstruction, the presence of side information at either parties, the combination of both, and successive refinement systems in accordance to [32]. Other potentially interesting extensions are in broadening the scope of the FSM model to larger classes of machines, including FSMs with counters, shift-register machines with counters, and periodically time-varying FSMs with counters, as was carried out in Section III of [26]. Research in some of these directions will be explored in future studies.

4. Proof of Theorem 1

First, observe that

$${\sigma }_E\left(x^n\right)={\displaystyle \frac{1}{n}}\sum _{i=1}^n\Delta (z_i,x_i)=\sum _{x,z}\widehat{P}(x,z)\Delta (z,x),$$

(22)

where $\widehat{P}=\{\widehat{P}(x,z),x\in \mathcal{X},z\in \mathcal{Z}\}$ is the joint empirical distribution of $(x,z)$ derived from $(x^n,z^n)$. It is therefore seen that ${\sigma }_E\left(x^n\right)$ depends on $x^n$ only via $\widehat{P}$. Accordingly, in the sequel, we will also use the alternative notation ${\sigma }_E\left(\widehat{P}\right)$ when we wish to emphasize the dependence on $\widehat{P}$. Let $\mathcal{T}\left(x^n\right)$ denote the set of ${\tilde{x}}^n\in {\mathcal{X}}^n$, which, together with their associated state sequences, share the same empirical PMF $\widehat{P}$ as that of $x^n$ along with its state sequence. Similarly as with ${\sigma }_E(·)$, we also denote it by $\mathcal{T}\left(\widehat{P}\right)$. In the sequel, we will make use of the inequality

$${\displaystyle \frac{log\left|\mathcal{T}\right(x^n\left)\right|}{n}}\ge {\rho }_{LZ}\left(x^n\right)-{\delta }_s\left(n\right),$$

(23)

where ${\delta }_s\left(n\right)\to 0$ as $n\to \infty$ for a fixed s at the rate of ${\displaystyle \frac{log(logn)}{logn}}$. The proof of Equation (23), which appears in various forms and variations in earlier papers (see, e.g., [34]), is provided in the Appendix A for the sake of completeness (see also the related Ziv’s inequality in Lemma 13.5.5 of [35]).

For later use, we also define the following sets:

$$\mathcal{P}\left(y^n\right)=\{\widehat{P}:\mathcal{T}\left(\widehat{P}\right)\cap f^{-1}\left(y^n\right)\ne \varnothing \},$$

(24)

$$\mathcal{Y}\left(\widehat{P}\right)=\{y^n:\mathcal{T}\left(\widehat{P}\right)\cap f^{-1}\left(y^n\right)\ne \varnothing \},$$

(25)

where

$$\begin{array}{ccc}\hfill f^{-1}\left(y^n\right)& =& \{x^n:f(z_{★},x^n,k^n)=y^n\hfill \\ & & \mathrm{for} \mathrm{some}{k}^n\in {\{0,1\}}^{n{\sigma }_E\left(x^n\right)}\}.\hfill \end{array}$$

(26)

In other words, $\mathcal{P}\left(y^n\right)$ is the set of all type classes of plaintext sequences for which some members can be mapped to $y^n$ by some key bit strings, whereas $\mathcal{Y}\left(\widehat{P}\right)$ is the set of ciphertext sequences which can be obtained by some member of type class $\mathcal{T}\left(\widehat{P}\right)$ and some key bit string. Now, observe that

$$\begin{array}{ccc}\hfill \left|\mathcal{Y}\right(\widehat{P}\left)\right|& \ge & \left|\right\{y^n:y^n=f(z_{★},x^n,0^{n{\sigma }_E\left(x^n\right)})\hfill \\ & & \mathrm{for} \mathrm{some}{x}^n\in \mathcal{T}\left(\widehat{P}\right)\left\}\right|\hfill \\ & \ge & \underset{z\in \mathcal{Z}}{max}\left|\right\{y^n:y^n=f(z_{★},x^n,0^{n{\sigma }_E\left(x^n\right)})\hfill \\ & & \mathrm{and}g(z_{★},x^n)=z\mathrm{for} \mathrm{some}{x}^n\in \mathcal{T}\left(\widehat{P}\right)\left\}\right|\hfill \\ & \stackrel{▵}{=}& \underset{z\in \mathcal{Z}}{max}\left|{\mathcal{Y}}_z\left(\widehat{P}\right)\right|\hfill \\ & \ge & {\displaystyle \frac{\left|\mathcal{T}\right(\widehat{P}\left)\right|}{s}},\hfill \end{array}$$

(27)

where the last inequality follows from the following consideration: Let $x^n$ exhaust all members of $\mathcal{T}\left(\widehat{P}\right)$. For each such $x^n$, let $y^n=f(z_{★},x^n,0^{n{\sigma }_E\left(x^n\right)})$. Now, for every $z\in \mathcal{Z}$, let ${\mathcal{T}}_z\left(\widehat{P}\right)$ denote the subset of $\mathcal{T}\left(\widehat{P}\right)$ for which $z_{n+1}=g(z_{★},x^n)=z$, and we have already defined ${\mathcal{Y}}_z\left(P\right)$ to denote the set of corresponding output sequences, $\left\{y^n\right\}$. Obviously, since ${\left\{{\mathcal{T}}_z\left(\widehat{P}\right)\right\}}_{z\in \mathcal{Z}}$ form a partition of $\mathcal{T}\left(\widehat{P}\right)$, then for some $z=z^{*}$, $|{\mathcal{T}}_{z^{*}}\left(\widehat{P}\right)|\ge |\mathcal{T}\left(\widehat{P}\right)|/s$. Therefore,

$$\begin{array}{ccc}\hfill \underset{z}{max}\left|{\mathcal{Y}}_z\left(\widehat{P}\right)\right|& \ge & |{\mathcal{Y}}_{z^{*}}\left(\widehat{P}\right)|\hfill \\ & =& |{\mathcal{T}}_{z^{*}}\left(\widehat{P}\right)|\hfill \\ & \ge & {\displaystyle \frac{\left|\mathcal{T}\right(\widehat{P}\left)\right|}{s}},\hfill \end{array}$$

(28)

where the equality is proved since the mapping between $x^n$ and $y^n$ is one-to-one given that $k^n=0^{n{\sigma }_E\left(x^n\right)}$, $z_1=z_{★}$, and $z_{n+1}=z^{*}$ by the information losslessness postulated, provided that n is sufficiently large as required. Now, let ${\mathcal{Y}}_n^{+}$ denote that set of all $y^n\in {\mathcal{Y}}^n$ for which $P_{Y^n|X^n}\left(y^n\right|x^n)>0$ for some $x^n\in {\mathcal{X}}^n$. Then,

$$\begin{array}{ccc}\hfill {exp}_2\{\mathcal{L}(x^n\to y^n\}& =& \sum _{y^n\in {\mathcal{Y}}_n^{+}}\underset{x^n}{max}{P}_{Y^n|X^n}\left(y^n\right|x^n)\hfill \\ & =& \sum _{y^n\in {\mathcal{Y}}_n^{+}}\underset{x^n\in {\varphi }^{-1}\left(y^n\right)}{max}{P}_{Y^n|X^n}\left(y^n\right|x^n)\hfill \\ & =& \sum _{y^n\in {\mathcal{Y}}_n^{+}}\underset{\widehat{P}\in \mathcal{P}\left(y^n\right)}{max}\underset{x^n\in {\varphi }^{-1}\left(y^n\right)\cap \mathcal{T}\left(\widehat{P}\right)}{max}{P}_{Y^n|X^n}\left(y^n\right|x^n)\hfill \\ & \stackrel{\left(a\right)}{\ge }& \sum _{y^n\in {\mathcal{Y}}_n^{+}}\underset{\widehat{P}\in \mathcal{P}\left(y^n\right)}{max}\underset{x^n\in {\varphi }^{-1}\left(y^n\right)\cap \mathcal{T}\left(\widehat{P}\right)}{max}{2}^{-n{\sigma }_E\left(\widehat{P}\right)}\hfill \\ & =& \sum _{y^n\in {\mathcal{Y}}_n^{+}}\underset{\widehat{P}\in \mathcal{P}\left(y^n\right)}{max}{2}^{-n{\sigma }_E\left(\widehat{P}\right)}\hfill \\ & \ge & {\displaystyle \frac{1}{M_n}}\sum _{y^n\in {\mathcal{Y}}_n^{+}}\sum _{\widehat{P}\in \mathcal{P}\left(y^n\right)}{2}^{-n{\sigma }_E\left(\widehat{P}\right)}\hfill \\ & =& {\displaystyle \frac{1}{M_n}}\sum _{\widehat{P}}\sum _{y^n\in \mathcal{Y}\left(\widehat{P}\right)}{2}^{-n{\sigma }_E\left(\widehat{P}\right)}\hfill \\ & =& {\displaystyle \frac{1}{M_n}}\sum _{\widehat{P}}\left|\mathcal{Y}\left(\widehat{P}\right)\right|·2^{-n{\sigma }_E\left(\widehat{P}\right)}\hfill \\ & \stackrel{\left(b\right)}{\ge }& {\displaystyle \frac{1}{M_{n}s}}\sum _{\widehat{P}}\left|\mathcal{T}\left(\widehat{P}\right)\right|·2^{-n{\sigma }_E\left(\widehat{P}\right)}\hfill \\ & \ge & {\displaystyle \frac{1}{M_{n}s}}·\underset{x^n\in {\mathcal{X}}^n}{max}\left|\mathcal{T}\left(x^n\right)\right|·2^{-n{\sigma }_E\left(x^n\right)}\hfill \\ & \stackrel{\left(c\right)}{\ge }& {\displaystyle \frac{1}{M_{n}s}}·\underset{x^n\in {\mathcal{X}}^n}{max}{2}^{n[{\rho }_{\mathrm{LZ}}\left(x^n\right)-{\delta }_s\left(n\right)]}·2^{-n{\sigma }_E\left(x^n\right)}\hfill \\ & =& {exp}_2\{n·\underset{x^n\in {\mathcal{X}}^n}{max}[{\rho }_{\mathrm{LZ}}\left(x^n\right)-{\sigma }_E\left(x^n\right)-{\delta }_s\left(n\right)-\hfill \\ & & {\displaystyle \frac{log{M}_n}{n}}-{\displaystyle \frac{logs}{n}}\left]\right\}\hfill \\ & \ge & {exp}_2\{n·\underset{x^n\in {\mathcal{X}}^n}{max}[{\rho }_{\mathrm{LZ}}\left(x^n\right)-{\sigma }_E\left(x^n\right)-{\delta }_s\left(n\right)-\hfill \\ & & {\displaystyle \frac{(\alpha s-1)log(n+1)}{n}}-{\displaystyle \frac{logs}{n}}\left]\right\},\hfill \end{array}$$

(29)

where, in (a), we used the fact that $P_{Y^n|X^n}\left(y^n\right|x^n)>0$ implies $P_{Y^n|X^n}\left(y^n\right|x^n)\ge 2^{-n{\sigma }_E\left(x^n\right)}$ (because $P_{Y^n|X^n}\left(y^n\right|x^n)>0$ implies that there is at least one $k^n\in {\{0,1\}}^{n{\sigma }_E\left(x^n\right)}$ such that $f(z_{★},x^n,k^n)=y^n$ and the probability of each such $k^n$ is $2^{-n{\sigma }_E\left(x^n\right)}$), and where $M_n$ is the number of different type classes, $\left\{\widehat{P}\right\}$, which is upper-bounded by ${(n+1)}^{\alpha s-1}$. In (b), we used Equation (27); in (c), we used Equation (23). Finally, the operator ${[·]}_{+}$ that appears in the assertion of Theorem 1 is due to the additional trivial lower bound $\mathcal{L}(x^n\to Y^n)\ge 0$. This completes the proof of Theorem 1.