Codeformer: A GNN-Nested Transformer Model for Binary Code Similarity Detection

Round 1

Reviewer 1 Report

• The paper  does not provide much detail on the specific calculation and judgment methods used in the Codeformer model, which could make it difficult for readers to fully understand and replicate the results.

• The paper acknowledges that the method may have limitations in terms of memory allocation when processing large basic blocks, which could impact the ability to analyze large datasets.

• While the proposed method appears to improve upon traditional graph matching algorithms, it is not clear how it compares to other existing methods for binary code similarity detection that also utilize deep learning techniques.

Author Response

Thank you very much for your suggestions. We have revised the manuscript in the light of all the suggestions. Please see the attachment for detail information.

Author Response File: Author Response.pdf

Reviewer 2 Report

The paper is written well. Eligible to publish.

Author Response

Thank you very much for your suggestions. We have revised the manuscript in the light of all the suggestions. Please see the attachment for detail information.

Author Response File: Author Response.pdf

Reviewer 3 Report

This paper proposes an improvement over existing methods of analysing the binary similarity. The existing methods and their weaknesses - namely the obfuscation of certain important properties - are described and a new solution is proposed comprehending the extraction of the basic blocks features as well as the structural features of the CFG in an iterative way.

The authors propose a new framework for binary code similarity detection called Codeformer, carefully explain their fundaments and present it´s rational and structure. The validation is done by starting to set three research questions and do some benchmarking. The tests were made against  similar existing models, considering two assessment methods: AUC(ROC) and ACC. 

Then the paper performs an analysis of the results, answering to the research questions. Every indication in the answering to the research questions is adequately supported on the results. It follows a discussion and conclusions are drawn.

It is a well structured paper, relevant to the field of binary similarity detection. All discussion and conclusions seems to be valid and consistent with the presented results.

The reference list is extensive (41 references) with enough recent relevant references, as well as, some with more than five years but relevant to the paper.

The tables and figures are appropriate and informative. 

Small details: In line 8, when you first address the GNN you should describe its meaning like you do in line 10 for CFG. Also on line 46, should it be SPP instead of SSP?

Author Response

Thank you very much for your suggestions. We have revised the manuscript in the light of all the suggestions. Please see the attachment for detail information.

Author Response File: Author Response.pdf