Generation of Nonlinear Substitutions by Simulated Annealing Algorithm

Abstract

The problem of nonlinear substitution generation (S-boxes) is investigated in many related works in symmetric key cryptography. In particular, the strength of symmetric ciphers to linear cryptanalysis is directly related to the nonlinearity of substitution. In addition to being highly nonlinear, S-boxes must be random, i.e., must not contain hidden mathematical constructs that facilitate algebraic cryptanalysis. The generation of such substitutions is a complex combinatorial optimization problem. Probabilistic algorithms are used to solve it, for instance the simulated annealing algorithm, which is well-fitted to a discrete search space. We propose a new cost function based on Walsh–Hadamard spectrum computation, and investigate the search efficiency of S-boxes using a simulated annealing algorithm. For this purpose, we conduct numerous experiments with different input parameters: initial temperature, cooling coefficient, number of internal and external loops. As the results of the research show, applying the new cost function allows for the rapid generation of nonlinear substitutions. To find 8-bit bijective S-boxes with nonlinearity 104, we need about 83,000 iterations. At the same time, the probability of finding the target result is 100%.

1. Introduction

Nonlinear substitutions (S-boxes) play an important role in symmetric key cryptography [1,2,3]. S-boxes fulfil the role of nonlinear complication in converting plaintext into ciphertext, thereby providing resistance to various types of cryptanalysis [4,5]. For instance, the nonlinearity indicator characterises resistance to linear cryptanalysis [6,7,8]. In this sense, the generation of highly nonlinear substitutions is a relevant field of contemporary research [9,10,11,12].

Another important property of S-boxes is the algebraic immunity [13]. This parameter indicates the complexity of the system of algebraic equations describing the substitution [14,15]. Simple systems of equations are derived for algebraic S-boxes, such as in the AES cipher (FIPS-197) [16]. This is considered a potential disadvantage, as the presence of simple mathematical structures can lead to an effective algebraic attack on the cipher [17]. Random substitutions are devoid of hidden algebraic constructions and, in the majority of cases, their algebraic immunity is maximised [18]. However, the nonlinearity of random S-boxes is not high and needs enhancement [19,20,21].

In order to generate random, highly nonlinear S-boxes, probabilistic combinatorial optimisation algorithms are used [9,10,22]. The generation starts with random substitution. Further, it is modified by randomly changing the values of the substitution. This procedure is iteratively repeated until the desired (target) result is obtained. At each iteration step, a cost function is calculated, which characterises the degree of proximity to the target result. In other words, the algorithm goes through possible alternatives in a discrete search space, focusing on the values of the cost function.

Among the many probabilistic combinatorial optimisation algorithms, the annealing simulation is worth mentioning [23]. This algorithm has been developed by modelling the physical processes involved in the annealing of metals. As long as the temperature of the metal is high, there is a possibility of atoms transferring to other cells in the crystal lattice. As the temperature drops, this probability decreases and the metal solidifies. This process is modelled as a computational search algorithm in a discrete space of possible states. As long as the temperature is high, there is a probability of transition to a worsening state, i.e., for which the value of the cost function is higher. As the temperature decreases, the probability of adopting a worsening state decreases and the process solidifies—only improving decisions are made. The possibility of making worsening decisions is extremely important, as it enables to move out of the local optimum and continue the search [23].

The annealing simulation algorithm has been used to generate S-boxes in several related works [21,24,25]. However, the complexity of computational search is still very high. For instance, our recent work [25] shows that applying optimised algorithm parameters can achieve the target result in about 450,000 iterations. In this paper we propose a new cost function based on the calculation of the Walsh–Hadamard spectrum. We optimise the annealing simulation algorithm for this cost function and show that the target result can be achieved in about 83,000 iterations. Thus, the main contribution of the paper is a new cost function and optimisation of the simulated annealing algorithm for fast generation of highly nonlinear random substitutions.

2. Related Works

The basic mathematical designations and methods for generating nonlinear substitutions are presented in [4,5]. Random generation methods have been investigated in [19,20]. In [21], one of the first WHS cost functions is proposed and an annealing simulation algorithm is investigated. Other cost functions have been reviewed in [11,26,27]. In particular, the study by Picek, S., Cupic, M., and Rotim, L. [11] proposed a novel cost function that demonstrated significant improvements compared to previous research efforts. This innovative approach contributed to advancing the field of combinatorial optimization for S-box generation, emphasizing the importance of exploring alternative cost functions in achieving better results. In more recent works by Freyre Echevarría, A., and Martínez Díaz [26], another new cost function was introduced, which further enhanced the performance compared to the initial improvements obtained in [11]. This progression highlights the ongoing advancements in the field and the potential for achieving even better results through continuous refinement of cost functions and optimization techniques. Interestingly, the best performance in the studies [26,28] was achieved using the well-established hill climbing algorithm [29,30].

The annealing simulation algorithm has been investigated in several related works. For instance, bijective 8-bit substitutions with nonlinearity 104 were generated in [24]. Nevertheless, the computational complexity of this generation turned out to be very high, requiring more than 3 million iterations. The annealing simulation algorithm was also used in [31], but the authors managed to achieve a nonlinearity of 100, which is significantly lower than other well-known results. In [32] the authors achieved only nonlinearity values of 92, which is comparable to random generation. Our latest work [25] managed to achieve nonlinearity 104 with the lowest generation complexity for the annealing simulation algorithm—we needed about 450,000 iterations.

Another thing to note is the probabilistic nature of substitution generation. The modification of the current state at each iteration of the search is carried out randomly Consequently, the target result is not always achieved. As an example, in [19,21] the probability of generating S-boxes with nonlinearity 102 is about 0.5%. More than 50% of the algorithm’s runs yielded the target result in our work [25]. In this paper, we significantly improve on these two important indicators—the complexity of generation and the probability of achieving the target outcome.

3. Background

The annealing simulation algorithm is based on the simulation of the physical process that occurs during the crystallisation of a substance, including metals. At high temperature, the atoms of the substance can move to other cells of the crystal lattice and this is simulated by means of the so-called Gibbs measure [33]. The probability of the system $X$ being in state $x$ (equivalently, that the random variable $X$ has value of $x$) is defined as (Gibbs measure):

$$P(X=x)= \frac{e^{- \beta E(x)}}{Z(\beta )},$$

where $E(x)$ s interpreted as state energy of $x$; $\beta =\frac{1}{k_{B}T}$ is thermodynamic beta; $T$ is the temperature; $k_B$ is Boltzmann constant; $Z(\beta )$ is the normalizing constant.

The annealing simulation algorithm simulates a similar process. The state energy $E(x)$ is interpreted as the value of the cost function $CF(S)$ at some point in the search space. In our case, $S$ is a particular S-box in the discrete space of all possible substitutions. By modifying the state $S$, we get a new state $S*$. In doing so, there are possible cases:

–

The energy has reduced, i.e., $CF(S*)<CF(S)$. Then the substitution $S*$ is taken as the current state and the search continues from this point in the state space; and

–

The energy has not reduced, i.e., $CF(S*)\ge CF(S)$. Then the substitution $S*$ is taken as the current state with probability.

$$P(X=S*)= e^{- \frac{CF(S*)-CF(S)}{T}}.$$

Thus, the probability of adopting a deteriorating state diminishes as the temperature $T$ decreases. The algorithm simulates the cooling of the substance in proportion to the cooling coefficient $\alpha$.

The pseudocode of the annealing simulation algorithm has the form [21,23,34]:

Algorithm 1. The Annealing Simulation Algorithm

$\mathrm{Input}: T_0$; $\alpha$$; S_0$. $T\leftarrow T_0$; $S\leftarrow S_0$; $\mathrm{For} i=1$$\mathrm{to} k_{out}$: $S*\leftarrow \mathrm{Modification}(S)$; $\mathrm{If} \left[CF(S*)<CF(S)\right]\vee \left[RND(0,1)<P(X=S*)\right]$ ○ $S\leftarrow S*$; $T\leftarrow \alpha T$; Output: the final state S.

In [25] we refined this algorithm, in particular we changed the new state acceptance rule:

$$\left[N(S*)>N(S)\right]\vee \left[CF(S*)\le CF(S)\right]\vee \left[RND(0,1)<P(X=S*)\right],$$

i.e., we accept the state $S*$ when:

–

Either the nonlinearity has increased, $N(S*)>N(S)$; or

–

The value of the cost function has not increased, $CF(S*)\le CF(S)$; or

–

With probability of $P(X=S*)$ we accept a worsening solution.

This change has greatly extended the range of accepted values. In addition, as in [35], we use:

–

An additional loop of k_int iterations at the same temperature T; and

–

An additional condition for exiting the outer loop when the number of iterations without improvement k_froz is reached.

4. Materials and Methods

We investigate the problem of generating bijective 8-bit substitutions given by a set of eight Boolean functions $S(x)=(f_0,f_1,\dots ,f_7)$ from eight Boolean variables:

$$f_0(x_0,x_1,\dots ,x_7),f_1(x_0,x_1,\dots ,x_7),\dots ,f_7(x_0,x_1,\dots ,x_7).$$

The nonlinearity of the substitution $S$ is defined through the Walsh–Hadamard spectrum by the expression [5]:

$$N(S)=\underset{v\in {\{0,1\}}^8\backslash {\left\{0\right\}}^8}{\min }\{N(v\cdot S(x))\} = \frac{1}{2}(2^8-\underset{v,u\in {\{0,1\}}^8\backslash {\left\{0\right\}}^8}{\max }\left|WHT(v\cdot S(x),u)\right|),$$

where

$$WHT(f(x),u)={\displaystyle \sum _{x\in {\{0,1\}}^8}{(-1)}^{f(x)\oplus u\cdot x}}.$$

Obviously, to increase the nonlinearity of $N(S)$, the maximum absolute values of the Walsh–Hadamard coefficients must be reduced. In the combinatorial optimisation algorithms under research, this is implemented through the cost function $CF(S)$—an interpretation of the energy of state $E(x)$ during the crystallisation of matter.

One of the best known cost functions used to generate nonlinear substitution is proposed in [21]:

$$C{F}_{WHS}(S)={\displaystyle \sum _{v\in {\{0,1\}}^8}{\displaystyle \sum _{u\in {\{0,1\}}^8}{\left||WHT(v\cdot F(x),u)|-X\right|}^R}},$$

where $X$ and $R$—configurable parameters.

The $C{F}_{WHS}(S)$ function has been used in various related works. For instance, in [11,28,36] it was used to find substitutions with nonlinearity $N(S)=104$. However, the complexity of the search turned out to be extremely high, requiring at least 3 million iterations to find a single S-box. In addition, the probability of achieving the target result did not exceed 1/200 = 0.5% [21]. Another cost function was proposed in [11]:

$$C{F}_{PCF}(S)={\displaystyle \sum _{i=1}^N{2}^{-i}H{(S)}_{k-i}},$$

where $H(S)$—vector of values $\left|WHT(v\cdot F(x),u)\right|$, in which the $i$-th position shows the number of coefficients with values $\left|4i\right|$, $k$—maximum position number with a non-zero value.

In [11], S-boxes with nonlinearity $N(S)=104$ were generated using $C{F}_{PCF}(S)$. The authors of this paper investigated various probabilistic search algorithms: Genetic Algorithm (GA) [37], Genetic and Tree Algorithm (GaT) [36], and our Local Search Algorithm (LSA). The combination of $C{F}_{PCF}(S)$ and GaT turned out to be the most effective [11].

In [26,28] a function was suggested:

$$C{F}_{WCF}(S)={\displaystyle \sum _{v\in {\{0,1\}}^8}{\displaystyle \sum _{u\in {\{0,1\}}^8}{\displaystyle \prod _{z\in C}\left||WHT(v\cdot F(x),u)|-z\right|}}},$$

where $C=\left\{0,4,\dots ,32\right\}$.

The authors of these papers also investigated LSA and GaT, however, the combination of the $C{F}_{WCF}(S)$ cost function and Hill Climbing (HC) Algorithm proved to be the most effective for forming bijective S-boxes with $N(S)=104$ [26]. However, the success rate was only 11/30, i.e., only about 35% of the algorithm runs ended up generating a target S-box with $N(S)=104$ [26].

In this paper we propose a new cost function that is based on a Walsh–Hadamard spectrum calculation:

$$C{F}_{new}=\underset{|WHT(v\cdot F(x),u)|>40}{{\displaystyle \sum _{v\in {\{0,1\}}^8}{\displaystyle \sum _{u\in {\{0,1\}}^8}}}}{2}^{\frac{\left|WHT(v\cdot F(x),u)\right|-20}{2}}.$$

The calculation of our cost function is done through the summation of the $2^{\frac{\left|WHT(v\cdot F(x),u)\right|-20}{2}}$ terms, for which $\left|WHT(v\cdot F(x),u)\right|>40$. This allows focusing only on those $WHT(f(x),u)$ coefficients that can change the nonlinearity value of $N(S)$. Such logic for forming the cost function will, in our view, significantly reduce the computational complexity of the state space search.

The aim of our work was to investigate the behaviour of the annealing simulation algorithm with the new $C{F}_{new}$ cost function under different input data. In particular, we examined the different values:

• The initial temperature T₀;
• The cooling rate α; and
• The number of internal k_int and external k_out loops.

The selection of the initial temperature is important, as it is decisive for calculating the probability of accepting a deteriorating condition $P(X=S*)$. We investigated four initial temperature values $T_0\in \{100, 1000, \mathrm{10,000},\mathrm{100,000}\}$. Figure 1 shows the calculated dependencies of the probability $P(X=S*)$ on the value of $\Delta =CF(S*)-CF(S)$ for different values of temperature $T$.

The probability of accepting deteriorating conditions should not be excessively high and should not be too low. For example, if $P(X=S*)\approx 1$, then almost every deteriorating condition will be accepted. In this case, the algorithm does not search for the optimum state, but only goes through possible alternatives. Conversely, if $P(X=S*)\approx 0$ deteriorating decisions will not be made at all and the algorithm will only find the nearest local minimum of the cost function. As can be seen from Figure 1, all probability values shift to the right as $T$ increases, i.e., for each value of $T$ there is a range of “acceptable” $\Delta =CF(S*)-CF(S)$ values for which the probabilities $P(X=S*)$ are neither too large nor too small.

It should also be noted that the temperature decreases at each iteration of the algorithm in proportion to the cooling factor $\alpha$. This leads to a reduction in the probability $P(X=S*)$, as illustrated in Figure 2 for $T_0=1000$ and $\Delta =CF(S*)-CF(S)=1000$. In our research, we considered $\alpha \in \{0.1;0.2;\dots ;0.9\}$ values and independently tested the search algorithm for each one.

At the same time, we chose the number of internal $k_{\mathrm{int}}$ and external $k_{out}$ loops so that the total number of iterations (in one thread) would be equal to $k_{out}\cdot k_{\mathrm{int}}=\mathrm{100,000}$. We used multi-threaded mode, testing was performed on 8 threads, i.e., the total number of iterations for one algorithm run did not exceed $\mathrm{800,000}$.

In total, we considered three options for starting the algorithm:

• k_out = 10, k_int = 10,000;
• k_out = 100, k_int = 1000; and
• k_out = 1000, k_int = 100.

The first option corresponds to a very slow decline in temperature and it means that worsening conditions will be taken more often. The second and third options use a greater number of external iterations and this means that the probability of accepting a deteriorating condition is reduced rapidly.

We investigated both the total number of search iterations and the probability of finding the target state (an 8-bit bijective S-box with $N(S)=104$ and maximum algebraic immunity).

5. Results

The main results are shown in

Table 1. Findings of search cost investigations for $T_0=100$.

	${\mathit{k}}_{\mathit{o}\mathit{u}\mathit{t}}=10$, ${\mathit{k}}_{\mathbf{int}}=\mathrm{10,000}$	${\mathit{k}}_{\mathit{o}\mathit{u}\mathit{t}}=100$, ${\mathit{k}}_{\mathbf{int}}=1000$	${\mathit{k}}_{\mathit{o}\mathit{u}\mathit{t}}=1000$, ${\mathit{k}}_{\mathbf{int}}=100$
$\alpha =0.1$	76,009.67 (100%)	79,524.40 (100%)	80,626.22 (100%)
$\alpha =0.2$	81,687.52 (100%)	90,611.95 (100%)	78,434.70 (100%)
$\alpha =0.3$	81,329.86 (100%)	88,245.98 (100%)	85,446.05 (100%)
$\alpha =0.4$	80,237.73 (100%)	72,907.94 (100%)	90,480.17 (100%)
$\alpha =0.5$	92,201.78 (100%)	82,822.92 (100%)	85,577.41 (100%)
$\alpha =0.6$	85,267.11 (100%)	80,468.54 (100%)	83,661.40 (100%)
$\alpha =0.7$	86,358.69 (100%)	79,653.47 (100%)	83,008.08 (100%)
$\alpha =0.8$	77,112.92 (100%)	82,100.52 (100%)	85,446.05 (100%)
$\alpha =0.9$	80,084.09 (100%)	84,360.09 (100%)	80,019.55 (100%)

,

Table 2. Findings of search cost investigations for $T_0=1000$.

	${\mathit{k}}_{\mathit{o}\mathit{u}\mathit{t}}=10$, ${\mathit{k}}_{\mathbf{int}}=\mathrm{10,000}$	${\mathit{k}}_{\mathit{o}\mathit{u}\mathit{t}}=100$, ${\mathit{k}}_{\mathbf{int}}=1000$	${\mathit{k}}_{\mathit{o}\mathit{u}\mathit{t}}=1000$, ${\mathit{k}}_{\mathbf{int}}=100$
$\alpha =0.1$	82,530.71 (100%)	84,092.05 (100%)	82,686.53 (100%)
$\alpha =0.2$	83,479.26 (100%)	81,281.64 (100%)	83,755.94 (100%)
$\alpha =0.3$	84,975.59 (100%)	79,385.64 (100%)	88,544.81 (100%)
$\alpha =0.4$	82,686.68 (100%)	83,480.77 (100%)	93,185.31 (100%)
$\alpha =0.5$	82,397.30 (100%)	81,515.59 (100%)	80,625.44 (100%)
$\alpha =0.6$	85,553.65 (100%)	78,870.06 (100%)	82,309.30 (100%)
$\alpha =0.7$	83,126.02 (100%)	89,313.00 (100%)	86,520.43 (100%)
$\alpha =0.8$	78,633.10 (100%)	82,243.44 (100%)	81,308.54 (100%)
$\alpha =0.9$	86,604.24 (100%)	90,141.24 (100%)	83,131.90 (100%)

,

Table 3. Findings of search cost investigations for $T_0=\mathrm{10,000}$.

	${\mathit{k}}_{\mathit{o}\mathit{u}\mathit{t}}=10$, ${\mathit{k}}_{\mathbf{int}}=\mathrm{10,000}$	${\mathit{k}}_{\mathit{o}\mathit{u}\mathit{t}}=100$, ${\mathit{k}}_{\mathbf{int}}=1000$	${\mathit{k}}_{\mathit{o}\mathit{u}\mathit{t}}=1000$, ${\mathit{k}}_{\mathbf{int}}=100$
$\alpha =0.1$	81,975.30 (100%)	79,378.70 (100%)	83,423.39 (100%)
$\alpha =0.2$	88,910.29 (100%)	88,009.86 (100%)	83,627.32 (100%)
$\alpha =0.3$	84,952.69 (100%)	83,338.59 (100%)	77,009.26 (100%)
$\alpha =0.4$	80,722.98 (100%)	82,152.62 (100%)	90,112.30 (100%)
$\alpha =0.5$	75,690.74 (100%)	83,605.75 (100%)	84,112.64 (100%)
$\alpha =0.6$	82,985.41 (100%)	82,453.39 (100%)	80,729.33 (100%)
$\alpha =0.7$	86,407.67 (100%)	82,462.94 (100%)	84,714.74 (100%)
$\alpha =0.8$	82,742.78 (100%)	80,918.05 (100%)	86,715.16 (100%)
$\alpha =0.9$	81,975.30 (100%)	79,378.70 (100%)	83,423.39 (100%)

and

Table 4. Findings of search cost investigations for $T_0=\mathrm{100,000}$.

	${\mathit{k}}_{\mathit{o}\mathit{u}\mathit{t}}=10$, ${\mathit{k}}_{\mathbf{int}}=\mathrm{10,000}$	${\mathit{k}}_{\mathit{o}\mathit{u}\mathit{t}}=100$, ${\mathit{k}}_{\mathbf{int}}=1000$	${\mathit{k}}_{\mathit{o}\mathit{u}\mathit{t}}=1000$, ${\mathit{k}}_{\mathbf{int}}=100$
$\alpha =0.1$	129,340.79 (100%)	82,815.54 (100%)	84,054.20 (100%)
$\alpha =0.2$	153,047.44 (100%)	88,950.34 (100%)	80,751.58 (100%)
$\alpha =0.3$	177,362.26 (100%)	83,268.43 (100%)	79,454.68 (100%)
$\alpha =0.4$	185,580.21 (100%)	86,333.64 (100%)	82,799.83 (100%)
$\alpha =0.5$	226,627.30 (100%)	92,578.85 (100%)	91,702.98 (100%)
$\alpha =0.6$	281,077.52 (100%)	87,348.77 (100%)	80,655.30 (100%)
$\alpha =0.7$	345,715.50 (100%)	94,778.97 (100%)	83,609.55 (100%)
$\alpha =0.8$	503,914.17 (100%)	106,267.47 (100%)	79,737.76 (100%)
$\alpha =0.9$	- (0%)	153,180.84 (100%)	81,359.10 (100%)

.

Table 1 shows the results of the search cost estimate (average number of iterations over all threads) for $T_0=100$. It is observed for all cases that the target result is consistently generated in approximately 83,000 iterations. The search cost is not affected by either the cooling factor or the number of iterations of the outer $k_{out}$ and inner $k_{\mathrm{int}}$ loops of the algorithm. This suggests that the proportion of deteriorating decisions taken is not large and does not contribute significantly to the overall cost of the search. The same conclusions can be drawn from the results shown in Table 2 and Table 3 for $T_0=1000$ and $T_0=\mathrm{10,000}$. In brackets is the fraction of runs of the algorithm in which the target state was found, i.e., an 8-bit bijective S-box with $N(S)=104$ accompanied by maximum algebraic immunity was generated. As can be seen from the results shown in Table 1, Table 2 and Table 3, 100% of the algorithm runs ended up generating the target S-box in all cases.

The initial temperature $T_0=\mathrm{100,000}$ gives slightly different results. As can be seen from the results in Table 4, the cost of searching for some parameters significantly increased. For instance, for $k_{out}=100$, $k_{\mathrm{int}}=1000$ and $\alpha =0.9$ the simulated annealing algorithm needs about twice as many iterations to find the target result (compared to Table 1, Table 2 and Table 3). The situation worsens for $k_{out}=10$, $k_{\mathrm{int}}=\mathrm{10,000}$ and at $\alpha =0.9$ no target result was found (over 800,000 search iterations in each of the 100 runs of the algorithm). The increase in search costs is highlighted in colour in Table 4. The darkest cell colour means the worst result.

6. Discussion of the Results

The stable formation of the target outcome for all parameters in Table 1, Table 2 and Table 3 is due to the low probability of acceptance of a deteriorating condition. For example, Appendix A shows a fragment of the list of cost function changes with the nonlinearity, current temperature and iteration number for $T_0=1000$. The algorithm did not take a worsening condition even once, since for even the smallest observable $\Delta =CF(S*)-CF(S)\approx \mathrm{10,000}$ the corresponding probability is extremely low $P(X=S*)\approx 4.5·{10}^{-5}$. Nevertheless, deteriorating conditions are accepted, but under a different rule. As a reminder, our version of the algorithm accepts all states for which $N(S*)>N(S)$, even if $CF(S*)>CF(S)$. In other words, our algorithm actually uses two cost functions: $CF(S)$ and $N(S)$. This enables the algorithm to take states that degrade the $CF(S)$ function and thus escape from the local optimum even at very low values of probability $P(X=S*)$. This can be clearly seen in the example given in Appendix A, with the relevant cases in bold. Appendixes also show the resulting S-boxes and indicate the main cryptographic indicators: nonlinearity (NL), delta uniformity (DU) and algebraic immunity (AI).

As the initial temperature rises, the proportion of accepted worsening conditions increases. This is explained by the raising of the probability of $P(X=S*)$. For instance, for $T_0=\mathrm{10,000}$ and $\Delta =CF(S*)-CF(S)\approx \mathrm{10,000}$ we obtain $P(X=S*)\approx 0.37$. Appendix B contains a fragment of the list of changes in the cost function for this initial temperature, with the case of accepting a worsening condition highlighted in bold.

It should be noted that almost identical results were obtained for the initial temperature $T_0\in \{100, 1000, \mathrm{10,000}\}$. The explanation for this is obviously the compensation of taking possible states due to the $\left[N(S*)>N(S)\right]\vee \left[CF(S*)\le CF(S)\right]\vee \left[RND(0,1)<P(X=S*)\right]$ rule. In fact, we observe a stable operation of the probabilistic search algorithm over a wide range of input data.

Appendix C shows a fragment of the list of cost function changes for $T_0=\mathrm{100,000}$. In this case, the chances of accepting deteriorating conditions are unsustainably high. For example, for $\Delta =CF(S*)-CF(S)\approx \mathrm{10,000}$ we obtain $P(X=S*)\approx 0.9$, i.e., almost every deteriorating condition is accepted and this significantly increases the complexity of the search. The relevant state acceptances are highlighted in Appendix C.

Separately, it should be commented on the high probability of forming a target S-box with $N(S)=104$. In almost every case, we observed a 100% achievement of the desired result. In the single, worst case, for the input parameters: $T_0=\mathrm{100,000}$, $\alpha =0.9$, $k_{out}=10$, $k_{\mathrm{int}}=\mathrm{10,000}$ (highlighted with the darkest background in Table 4) the probability was zero, i.e., the algorithm could not find any target results for these parameters.

To explain this observation, we conducted additional research. We plotted histograms of the number of iterations for the empirical data under the following parameters (Figure 3 and Figure 4):

• Case 1: T₀ = 100,000, α = 0.9, k_out = 1000, k_int = 100 (the mean is 81,359.10; the standard deviation is 28,406.44); and
• Case 2: T₀ = 100,000, α = 0.7, k_out = 10, k_int = 10,000 (the mean is 345,715.50; the standard deviation is 37,060.07).

The figures also show the expected frequencies for a normal distribution.

Table 5. Checking the hypothesis of a normal distribution for T₀ = 100,000, α = 0.9, k_out = 1000, k_int = 100 (the mean is 81,359.10; the standard deviation is 28,406.44).

Value Interval	$\mathbf{Expected} \mathbf{Frequencies} {\mathit{m}}_{\mathit{i}}$ (Normal Distribution)	$\mathbf{Observed} \mathbf{Frequencies} {\mathit{x}}_{\mathit{i}}$ (Empirical Data)	$\frac{{({\mathit{x}}_{\mathit{i}}-{\mathit{m}}_{\mathit{i}})}^2}{{\mathit{m}}_{\mathit{i}}}$
(29,033–45,533]	7.09	9	0.52
(45,533–62,033]	14.45	14	0.01
(62,033–78,533]	21.22	28	2.16
(78,533–95,033]	22.45	21	0.09
(95,033–111,533]	17.11	14	0.56
(111,533–128,033]	9.39	7	0.61
(128,033–144,533]	3.71	4	0.02
(144,533–161,033]	1.06	3	3.58
$\mathrm{Total} k=8$ intervals	${\chi }^2={\displaystyle \sum _{i=1}^k\frac{{(x_i-m_i)}^2}{m_i}}=$ 7.56
	$\mathrm{The} \mathrm{significance} \mathrm{level} \mathrm{is} \alpha =0.05$. The critical region is >11.070498

and

Table 6. Checking the hypothesis of a normal distribution for T₀ = 100,000, α = 0.7, k_out = 10, k_int = 10,000 (the mean is 345,715.50; the standard deviation is 37,060.07).

Value interval	$\mathbf{Expected} \mathbf{Frequencies} {\mathit{m}}_{\mathit{i}}$ (Normal Distribution)	$\mathbf{Observed} \mathbf{Frequencies} {\mathit{x}}_{\mathit{i}}$ (Empirical Data)	$\frac{{({\mathit{x}}_{\mathit{i}}-{\mathit{m}}_{\mathit{i}})}^2}{{\mathit{m}}_{\mathit{i}}}$
(256,991–280,991]	3.20	3	0.01
(280,991–304,991]	9.55	8	0.25
(304,991–328,991]	19.00	20	0.05
(328,991–352,991]	25.19	32	1.84
(352,991–376,991]	22.28	18	0.82
(376,991–400,991]	13.14	11	0.35
(400,991–424,991]	5.17	6	0.13
(424,991–448,991]	1.36	2	0.31
$\mathrm{Total} k=8$ intervals	${\chi }^2={\displaystyle \sum _{i=1}^k\frac{{(x_i-m_i)}^2}{m_i}}=$ 3.77
	$\mathrm{The} \mathrm{significance} \mathrm{level} \mathrm{is} \alpha =0.05$. The critical region is >11.070498

show the results of testing the normal distribution hypothesis. In both cases, the calculated value of the ${\chi }^2$ does not fall into the critical region, hence the hypothesis is confirmed. This means that there is a 95% probability that the number of iterations of the annealing simulation algorithm is distributed according to a normal distribution law.

Figure 5 shows graphs of the normal probability distributions for these two cases. The area under each curve is equal to 1. The total number of iterations in our research was set at 800,000 (100,000 for each of the 8 threads). Consequently, the area under the part of the curve for the values (border highlighted with a dashed line) equals the probability of the search algorithm not finding the target result. As can be seen from the graphs for both Case 1 and Case 2, this probability tends towards zero. This explains the 100% finding of the target states for these parameters. We repeated these examinations for each case in Table 1, Table 2, Table 3 and Table 4 and our theoretical calculations fully coincide with these empirical observations.

Additionally, we conducted experiments for the worst-case case (highlighted in darker colour in Table 4). To do this, we increased the number of external loops by a factor of 10 and performed several test runs of the algorithm. For the values of the mean and standard deviation obtained, Figure 5 shows a graph of the normal distribution (far right). For this graph, the area under the part of the curve for the values tends to 1, which explains the zero probability of forming the target outcome in Table 4.

Further discussion of the obtained results, particularly in the context of the comparison results presented in

Table 7. Comparison of results.

	[21,38]	[32]	[24]	[25]	Our Paper
Nonlinearity of substitution N(S)	102	92	104	104	104
Probability of generating target S-boxes	0.5%	–	–	56.4%	100%
Search cost (average number of iterations)	–	–	3,000,000	450,000	83,000

, reveals the significance of our study in advancing the field of combinatorial optimization for S-box generation. The improvements achieved by our approach are evident when contrasted with the outcomes of other known works.

Table 7 provides a comprehensive comparison of our approach with other methods for generating nonlinear S-boxes, highlighting the substantial reduction in the average number of algorithm search iterations. This reduction by more than eight times demonstrates the effectiveness of our custom implementation of the simulated annealing algorithm and the novel cost function. Moreover, it suggests that our method is more efficient and computationally less expensive than previously proposed solutions.

Another notable result in Table 7 is the enhanced frequency of reaching the target result. By employing probabilistic methods for generating substitutions, we have managed to increase the success rate by nearly two-fold. This improvement indicates that our approach is not only faster but also more reliable and robust in generating high-quality S-boxes, which is crucial for cryptographic applications.

The normal distribution of the search algorithm’s iteration count, as confirmed by the histograms, provides additional insight into the consistency and predictability of our method. The alignment between theoretical calculations and experimental results further supports the adequacy of our approach.

In conclusion, the results presented in Table 7, along with the other findings in our study, emphasize the advantages of our method in terms of efficiency, success rate, and reliability. These improvements have the potential to impact the development of more secure cryptographic algorithms and contribute to the ongoing research in the field of combinatorial optimization. Future work can further explore the potential of other probabilistic search algorithms and examine their effectiveness in conjunction with the novel cost function.

7. Conclusions

In this study, we investigated the simulated annealing algorithm for solving a combinatorial optimization problem related to the search for nonlinear substitutions (S-boxes). S-boxes play a crucial role in ensuring the efficiency of modern symmetric key cryptographic algorithms. We generated 8-bit bijective S-boxes with a nonlinearity of $N(S)=104$, utilizing our custom implementation of the simulated annealing algorithm and a novel cost function.

The experimental results demonstrate that, in most cases, the formation of target S-boxes is achieved within a stable average of 83,000 iterations. This performance ranks among the best-known results for the simulated annealing algorithm. Table 7 provides comparisons with other known works, illustrating that our approach significantly improves upon existing results. The average number of algorithm search iterations has been reduced by more than 8 times.

We also investigated the distribution of the search algorithm’s iteration count. Histograms of iteration distributions were constructed, confirming our hypothesis of a normal distribution with a 95% probability. This theoretical distribution enables the calculation of the probability of forming the target result for each considered case. The theoretical calculations align with experimental results, substantiating the adequacy of the obtained outcomes.

Another improvement in S-box generation achieved in our study is the frequency of reaching the target result. By exploring probabilistic methods of generating substitutions, we managed to nearly double the success rate.

Thus, we have significantly advanced the state-of-the-art in this research field in terms of these two crucial metrics. This progress is attributed to the utilization of the novel cost function and the optimized implementation of the simulated annealing algorithm.

A promising direction for future research involves the implementation of other probabilistic search algorithms, such as GA, GaT, and LSA, and testing their efficiency using the new cost function.