# Reevaluating Graph-Neural-Network-Based Runtime Prediction of SAT-Based Circuit Deobfuscation

^{*}

## Abstract

**:**

## 1. Introduction

- We set up a control experiment. We develop a novel synthetic circuit benchmark set of Substitution-Permutation Networks (SPN), which is the basis for the most efficient yet secure keyed-functions used today: block-ciphers. We then ask whether a GCN fully trained on traditional ISCAS benchmarks can predict the simple fact that a deep SPN is more secure than a wide SPN of the same size, to which we surprisingly discover the answer to be negative.
- Motivated by block-cipher design principles, we develop a set of engineered features extracted from the circuit graph. These include various measures of depth, which are critical in SPNs, along with width. Furthermore, we propose a partitioning scheme that allows mapping the circuit to a possible SPN analog and measures cryptanalysis metrics such as differential statistics and bit propagation width. We show how simple combinations of these metrics can beat the GCN on the SPN hardness prediction task.
- We take these metrics back to the non-SPN benchmark set and show that many positively and consistently correlate with runtime. This means that there is an important utility in extracting deeper engineered features for deobfuscation hardness. If not to supplant end-to-end learning, such metrics can certainly aid it.

## 2. Background

**Logic locking**. Formally, a logic locking scheme is an algorithm that transforms an n-input m-output original circuit ${c}_{o}\left(x\right)$ to a locked circuit ${c}_{e}(k,x)$ by adding l new key inputs, in a way that hides/corrupts the functionality of ${c}_{o}$ absent some correct key ${k}_{*}$. The locked circuit is fabricated by the untrusted foundry, and the correct key is programmed post-fabrication. Randomly inserting key-controlled XOR/XNOR gates, lookup-tables, AND-trees/comparators, and MUX networks are among common ways to implement locking [5].

**SAT attack**. An oracle-guided attack such as the SAT attack allows the attacker to query a black-box implementation of ${c}_{o}$. This corresponds to purchasing a functional/unlocked IC from the market and accessing its input/outputs/scan-chain. The attack begins by forming an SAT formula $M={c}_{e}({k}_{1},x)\ne {c}_{e}({k}_{2},x)$. This is derived by constructing the circuit for M and then using the Tseitin transform to convert it to an SAT Conjunctive-Normal-Form (CNF) formula. Satisfying M returns a Discriminating Input Pattern (DIP) $\widehat{x}$, which is queried on the oracle obtaining $\widehat{y}={c}_{o}\left(\widehat{x}\right)$. The resulting input-output relation is appended to M. The process continues until the formula is no longer satisfiable at which point solving the IO-relations will return a functionally correct key ${k}_{*}$.

**Graph Neural Networks (GNN)**. Most deep learning algorithms specialize in dealing with fixed-dimensional data. Many types of data, such as social networks, chemical structures, and circuits, are graph-like in nature. Graph Neural Networks (GNN) are a class of recent machine learning models that can directly operate on graphs and perform various learning tasks. Many of these models operate by taking in an initial set of node features as the node encoding. They then iteratively apply operations on the node features and the graph’s adjacency matrix that correspond to some variant of data being passed around and accumulated in the graph nodes along edges. This is all achieved without explicitly storing the dense representation of the graph’s adjacency matrix. GNNs have been shown to be effective for both node-level and graph-level regression and classification tasks [4]. Graph Convolutional Networks (GCNs) are a type of GNN layer that is analogous to the convolutional operation in deep neural networks.

**Substitution Permutation Networks (SPN)**. SPNs are the basic structure used in block-ciphers. Figure 1 demonstrates a general structure of an SPN. An SPN is often made up of several rounds, with each round consisting of a key mixing layer, and a group of substitution boxes, known as sboxes, which create the substitution layer and a permutation layer. The key mixing layer is typically a bit-wise XOR with the key vector, which means that to have a non-linear SPN, the sbox is the main source of non-linearity. The sbox is typically a look-up table, mapping its input bits to an equal or smaller number of output bits. The permutation in its simplest form performs a reversible bit shuffle. In a block-cipher, the keys for each round of the SPN are typically different and derived from a master key via key-expansion. It is not precisely known formally what kinds of sbox operations produce the most secure SPNs, but given the right substitution+permutation operation, the security of SPNs against practical attacks increases somewhat exponentially as the number of rounds increases.

## 3. Putting GCN-Based Deobfuscation Hardness Prediction to the Test

## 4. SPN-Motivated Features

**Non-key Part Structural Statistics**: this includes structural information about the non-key parts. This is primarily, the average number of inputs, outputs, and gates.**Number of Active Non-key Parts**: the average number of “active” non-key parts when passing a signal randomly from one of circuits’ input to an output. This is derived from the important notion of**active-sboxes**in block-cipher design. Given a change in some input bit, the change will propagate through the SPN touching many sboxes. The sboxes touched during this propagation are called active sboxes. The difficulty of differential cryptanalysis is directly tied to the number of active sboxes. This metric shows a surprising consistently high correlation with SAT attack time.**Non-Key Part Differential Statistics**: the average output flip rate when randomly flipping a bit in non-key parts. This is derived also from differential cryptanalysis, in which changes in the input of some sbox will produce changes at its output. The closer the change at the output is to half of all bits, the more propagation there is in the circuit. Furthermore, changes in the input that produce highly-likely/impossible changes at the output can facilitate differential cryptanalysis.

Algorithm 1: Circuit Partitioning |

## 5. Experiments

#### 5.1. Training Datasets

**SPN Generation**. We generate a novel training and evaluation dataset consisting of sequential and parallel SPN circuits. The SPN circuits are constructed using a Python script. The SPN sboxes are generated as random truth tables that are synthesized to gates using ABC. The truth-table generation is repeated several times and disconnected or highly skewed SPNs are discarded. Each time an SPN is generated the sboxes are each assigned new random truth tables.

- $seq/par$: type of SPN structure (sequential/parallel).
- p: number of parallel rounds.
- n: number of sboxes in each round.
- l: length for an sbox. i.e., number of inputs/outputs to each sbox.
- r: SPN round number (number of sequential rounds).

**Dataset_1**. We first select a set of benchmarks from ISCAS-85, ISCAS-89, and ITC-99 benchmarks, respectively, with diverse depth/width statistics to form a benchmark set (c432, c499, c880, c1355, c1980, c2670, c3540, c5315, s832, s953, s1196, s1238, s1488, s1494, s3271, s5378, b04, b07, b11, b12, b13, b14). Since some of these benchmark circuits are sequential, we convert them to combinational by removing flip-flops and replacing them with primary input/output pairs. Then, each circuit in the dataset is locked using XOR/XNOR locking with 64 key bits. The locking is repeated 100 times with different random seeds that results in the XOR/XNOR key-gates being inserted in different locations. These circuits have the same key size as the evaluated SPN datasets that will be introduced later. This dataset contains $22\times 100=2200$ circuits.**Dataset_2**. This dataset is comprised of a subset of the above benchmark circuits that are difficult to break for the SAT attack. The dataset is built by adjusting key sizes from 1 to 700 for the $c3540$, $s5378$, and 1 to 500 for the $b14$ benchmark circuits ($b14$ with more than 500 key bits was not deobfuscatable with our SAT attack). The locking is repeated 10 times for each key size. This leads to a total of $\left(\right(2\times 700)+(1\times 500\left)\right)\times 10=$ 19,000 locked circuits.**Dataset_3**. This dataset is derived by taking each circuit in the benchmark set in Dataset_1 (minus the $b14$ benchmark due to its size and deobfuscation time) and locking it with XOR/XNOR with the key size ranging from 1 to 200. The locking is repeated 5 times with different key gate locations. This leads to $21\times 200\times 5=$ 21,000 locked circuits.**Dataset_4**. This is a unique and novel dataset that solely comprises SPN circuits. This is built by constructing SPNs with the following parameters. The number of sboxes in each round is for different sbox number ($n\in \{4,8,16\}$), sbox length ($l\in \{4,6,8\}$), and number of rounds ($r\in \{1,2,3,4,5\}$). Each SPN is regenerated 20 times. This leads to a total of $20\times 5\times 3\times 3=900$ SPN circuits.

#### 5.2. Evaluation Datasets

#### 5.3. GCN Evaluation Results

#### 5.4. SPN-Motivated Metrics

## 6. Conclusions

## Author Contributions

## Funding

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Subramanyan, P.; Ray, S.; Malik, S. Evaluating the security of logic encryption algorithms. In Proceedings of the 2015 IEEE International Symposium on Hardware Oriented Security and Trust (HOST), Washington, DC, USA, 5–7 May 2015; IEEE: Manhattan, NY, USA, 2015; pp. 137–143. [Google Scholar]
- El Massad, M.; Garg, S.; Tripunitara, M.V. Integrated Circuit (IC) Decamouflaging: Reverse Engineering Camouflaged ICs within Minutes. In Proceedings of the Network and Distributed System Security Symposium (NDSS), San Diego, CA, USA, 8–11 February 2015. [Google Scholar]
- Chen, Z.; Kolhe, G.; Rafatirad, S.; Lu, C.T.; Manoj, S.; Homayoun, H.; Zhao, L. Estimating the circuit De-obfuscation runtime based on graph deep learning. In Proceedings of the 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE), Grenoble, France, 9–13 March 2020; IEEE: Manhattan, NY, USA, 2020; pp. 358–363. [Google Scholar]
- Kipf, T.N.; Welling, M. Semi-supervised classification with graph convolutional networks. arXiv
**2016**, arXiv:1609.02907. [Google Scholar] - Yasin, M.; Sinanoglu, O. Evolution of logic locking. In Proceedings of the 2017 IFIP/IEEE International Conference on Very Large Scale Integration (VLSI-SoC), Dhabi, United Arab Emirates, 23–25 October 2017; IEEE: Manhattan, NY, USA, 2017; pp. 1–6. [Google Scholar]
- Hutter, F.; Xu, L.; Hoos, H.H.; Leyton-Brown, K. Algorithm runtime prediction: Methods & evaluation. Artif. Intell.
**2014**, 206, 79–111. [Google Scholar] - Rajendran, J.; Pino, Y.; Sinanoglu, O.; Karri, R. Security analysis of logic obfuscation. In Proceedings of the 49th Annual Design Automation Conference, San Francisco, CA, USA, 3–7 June 2012; pp. 83–89. [Google Scholar]
- Shamsi, K.; Pan, D.Z.; Jin, Y. On the Impossibility of Approximation-Resilient Circuit Locking. In Proceedings of the 2019 IEEE International Symposium on Hardware Oriented Security and Trust (HOST), McLean, VA, USA, 5–10 May 2019; IEEE: Manhattan, NY, USA, 2019; pp. 161–170. [Google Scholar]
- Grattarola, D.; Alippi, C. Graph Neural Networks in TensorFlow and Keras with Spektral. arXiv
**2020**, arXiv:2006.12138. [Google Scholar]

**Figure 3.**A non-key part in a circuit. Non-key parts (which serve as sboxes when viewing the circuit as an SPN), are parts of the circuit that lie between key-gate logic.

**Figure 6.**Evaluation results comparison. The GCN consistently overestimates the deobfuscation difficulty of parallel SPNs.

Dataset | Avg #Gates | Dataset | Avg #Gates |
---|---|---|---|

seq_n8_l8_r1 | 7189.60 | par_p2_n4_l8_r1 | 7156.48 |

seq_n8_l8_r2 | 14,345.92 | par_p2_n8_l8_r1 | 14,368.32 |

seq_n8_l8_r3 | 21,621.12 | par_p3_n8_l8_r1 | 21,582.72 |

seq_n8_l8_r4 | 28,844.16 | par_p4_n8_l8_r1 | 28,794.24 |

seq_n16_l4_r1 | 512.00 | par_p2_n8_l4_r1 | 520.96 |

seq_n16_l4_r2 | 1056.00 | par_p2_n16_l4_r1 | 1039.68 |

seq_n16_l4_r3 | 1567.04 | par_p3_n16_l4_r1 | 1538.56 |

seq_n16_l4_r4 | 2065.92 | par_p4_n16_l4_r1 | 2089.28 |

GCN Prediction Values (in s) | ||||
---|---|---|---|---|

Training Dataset | seq_n8_l8_r1 | seq_n8_l8_r2 | seq_n8_l8_r3 | seq_n8_l8_r4 |

Real value | 1.78 | 30.84 | 1800 | 1800 |

Dataset_1 | 32.27 | 41.58 | 42.47 | 43.76 |

Dataset_2 | 711.07 | 724.06 | 685.49 | 662.13 |

Dataset_3 | 108.42 | 113.92 | 113.92 | 108.97 |

Dataset_4 | 291.16 | 673.13 | 807.61 | 1727.29 |

Training Dataset | seq_n16_l4_r1 | seq_n16_l4_r2 | seq_n16_l4_r3 | seq_n16_l4_r4 |

Real value | 0.13 | 0.51 | 5.51 | 1110.11 |

Dataset_1 | −10.98 | −7.54 | −5.88 | −5.33 |

Dataset_2 | 1309.36 | 1547.26 | 1641.45 | 1662.68 |

Dataset_3 | 406.98 | 336.88 | 286.55 | 275.77 |

Dataset_4 | −13.64 | 82.06 | 139.67 | 352.17 |

Training Dataset | par_p2_n4_l8_r1 | par_p2_n8_l8_r1 | par_p3_n8_l8_r1 | par_p4_n8_l8_r1 |

Real value | 1.42 | 3.75 | 6.03 | 8.87 |

Dataset_1 | 39.52 | 39.06 | 42.20 | 43.09 |

Dataset_2 | 696.53 | 677.57 | 702.76 | 668.33 |

Dataset_3 | 110.38 | 115.10 | 110.59 | 112.51 |

Dataset_4 | 712.06 | 715.10 | 844.00 | 919.86 |

Training Dataset | par_p2_n8_l4_r1 | par_p2_n16_l4_r1 | par_p3_n16_l4_r1 | par_p4_n16_l4_r1 |

Real value | 0.03 | 0.11 | 0.19 | 0.29 |

Dataset_1 | −11.18 | −11.21 | −10.83 | −10.16 |

Dataset_2 | 1465.58 | 1496.57 | 1544.90 | 1584.23 |

Dataset_3 | 431.60 | 436.15 | 431.13 | 430.95 |

Dataset_4 | 69.94 | 74.06 | 90.76 | 107.14 |

**Table 3.**SPN-motivated metric correlations with SAT attack time on the 4 different datasets.

**Learnability**is the ability of an sklearn MLP regressor to learn the output of the locked circuit, as measured by the average of the probability of correctly predicting output bits from the input vector. “#Keys with ≤X keys in fanin” is the number of key gates in each circuit that have fewer than X other key gates within their transitive fanin. The signal probability biases are captured as the distance of the non-key part output signal probability to 0.5.

Metrics | D1 | D2 | D3 | D4 | Average |
---|---|---|---|---|---|

#Gates | 0.89 | 0.63 | 0.04 | 0.58 | 0.54 |

Key Depth | 0.08 | −0.22 | 0.25 | 0.70 | 0.20 |

#Keys with <2 keys in fanin | 0.22 | 0.29 | 0.11 | −0.35 | 0.07 |

#Keys with <4 keys in fanin | −0.24 | −0.14 | 0.20 | −0.19 | −0.09 |

#Keys with <8 keys in fanin | −0.14 | 0.06 | 0.15 | −0.26 | −0.05 |

#Keys with <16 keys in fanin | −0.10 | 0.33 | 0.05 | −0.06 | 0.06 |

#Keys with <32 keys in fanin | −0.07 | 0.34 | 0.17 | −0.03 | 0.10 |

#Keys with <64 keys in fanin | −0.02 | 0.48 | 0.07 | 0.57 | 0.28 |

#Keys with <128 keys in fanin | - | 0.53 | 0.03 | 0.45 | 0.34 |

#Keys with <256 keys in fanin | - | 0.35 | - | 0.27 | 0.31 |

Avg signal walk depth | −0.06 | −0.11 | 0.19 | 0.81 | 0.21 |

Avg #inputs in nkeys | 0.77 | 0.10 | −0.04 | 0.61 | 0.36 |

Avg #gates in nkeys | 0.74 | 0.02 | −0.03 | 0.60 | 0.33 |

Avg #outputs in nkeys | - | - | - | 0.43 | 0.43 |

Min active nkeys | - | - | - | 0.39 | 0.39 |

Max active nkeys | 0.74 | 0.65 | 0.19 | 0.59 | 0.54 |

Avg active nkeys | 0.07 | 0.26 | 0.11 | 0.56 | 0.25 |

Avg #nkeys | - | 0.31 | 0.19 | 0.74 | 0.41 |

Min active nkeys(in percentage) | - | - | - | 0.31 | 0.31 |

Max active nkeys(in percentage) | 0.74 | 0.09 | −0.02 | 0.62 | 0.36 |

Avg active nkeys(in percentage) | 0.07 | −0.07 | −0.02 | 0.59 | 0.14 |

Avg nkeys dout/din | −0.12 | −0.13 | −0.02 | 0.01 | −0.07 |

Min nkey signal probability bias | −0.01 | −0.02 | −0.03 | −0.24 | −0.08 |

Max nkey signal probability bias | 0.12 | 0.10 | 0.06 | 0.42 | 0.18 |

Avg nkey signal probability bias | −0.05 | 0.35 | −0.13 | 0.36 | 0.13 |

Hamming distance | −0.24 | −0.29 | −0.05 | 0.65 | 0.02 |

Learnability | 0.39 | 0.28 | −0.05 | −0.05 | 0.14 |

Circuit key diameter | 0.56 | −0.09 | −0.07 | 0.60 | 0.25 |

Signal probability | −0.14 | −0.51 | −0.11 | −0.21 | −0.24 |

Circuit maximum depth | 0.08 | −0.41 | 0.24 | 0.72 | 0.16 |

Circuit maximum width | 0.93 | 0.60 | −0.08 | 0.40 | 0.46 |

Regression Models Prediction Values (in s) | ||||
---|---|---|---|---|

Regression Models | seq_n8_l8_r1 | seq_n8_l8_r2 | seq_n8_l8_r3 | seq_n8_l8_r4 |

Real value | 1.78 | 30.84 | 1800 | 1800 |

LR | 61.56 | 473.14 | 1654.00 | 1677.23 |

LASSO | 70.71 | 489.33 | 1654.54 | 1678.10 |

SVM | 91.53 | 193.47 | 320.25 | 460.60 |

MLP | 7.05 | 607.30 | 1613.23 | 1708.41 |

RR | 71.31 | 489.38 | 1654.50 | 1667.04 |

EN | 49.64 | 530.97 | 1686.88 | 1696.33 |

TS | 37.69 | 256.79 | 1835.84 | 1834.37 |

RF | 10.18 | 223.65 | 1799.11 | 1800.00 |

Regression Models | seq_n16_l4_r1 | seq_n16_l4_r2 | seq_n16_l4_r3 | seq_n16_l4_r4 |

Real value | 0.13 | 0.51 | 5.51 | 1110.11 |

LR | −35.94 | 79.59 | 416.89 | 925.24 |

LASSO | −34.47 | 72.17 | 396.85 | 927.61 |

SVM | 12.18 | 12.78 | 13.46 | 14.17 |

MLP | −24.75 | −49.10 | 288.59 | 1014.93 |

RR | −29.94 | 65.36 | 361.61 | 921.57 |

EN | −85.26 | 22.98 | 427.86 | 980.49 |

TS | −28.71 | 259.12 | 236.89 | 929.86 |

RF | 0.13 | 1.03 | 49.04 | 1055.95 |

Regression Models | par_p2_n4_l8_r1 | par_p2_n8_l8_r1 | par_p3_n8_l8_r1 | par_p4_n8_l8_r1 |

Real value | 1.42 | 3.75 | 6.03 | 8.87 |

LR | −281.38 | −270.51 | −362.75 | −395.15 |

LASSO | −300.70 | −309.73 | −439.31 | −504.49 |

SVM | 90.98 | 329.35 | 733.55 | 1295.49 |

MLP | −2.14 | −13.89 | −42.66 | −61.09 |

RR | −316.35 | −331.93 | −474.88 | −550.25 |

EN | −271.31 | −190.10 | −214.69 | −184.25 |

TS | −173.28 | −164.54 | −228.56 | −250.09 |

RF | 9.1463 | 59.40 | 284.55 | 683.83 |

Regression Models | par_p2_n8_l4_r1 | par_p2_n16_l4_r1 | par_p3_n16_l4_r1 | par_p4_n16_l4_r1 |

Real value | 0.03 | 0.11 | 0.19 | 0.29 |

LR | −197.69 | −209.09 | −276.35 | −318.07 |

LASSO | −206.98 | −229.49 | −315.49 | −376.14 |

SVM | 12.19 | 13.46 | 15.50 | 18.59 |

MLP | −27.73 | −74.42 | −123.07 | −169.48 |

RR | −213.29 | −228.19 | −310.11 | −368.13 |

EN | −231.59 | −16.78 | 150.09 | 332.30 |

TS | −127.84 | −105.34 | −116.13 | −111.35 |

RF | 0.14 | 7.38 | 7.65 | 8.49 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Zhao, G.; Shamsi, K.
Reevaluating Graph-Neural-Network-Based Runtime Prediction of SAT-Based Circuit Deobfuscation. *Cryptography* **2022**, *6*, 60.
https://doi.org/10.3390/cryptography6040060

**AMA Style**

Zhao G, Shamsi K.
Reevaluating Graph-Neural-Network-Based Runtime Prediction of SAT-Based Circuit Deobfuscation. *Cryptography*. 2022; 6(4):60.
https://doi.org/10.3390/cryptography6040060

**Chicago/Turabian Style**

Zhao, Guangwei, and Kaveh Shamsi.
2022. "Reevaluating Graph-Neural-Network-Based Runtime Prediction of SAT-Based Circuit Deobfuscation" *Cryptography* 6, no. 4: 60.
https://doi.org/10.3390/cryptography6040060