Detecting Rug-Pull: Analyzing Smart Contract Backdoor Codes in Ethereum

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The article is well written. The topic is extremely useful and relevant.

Some recommendations for improving the manuscript "Detecting Rug-pull: Analyzing Smart Contract Backdoor Codes in Ethereum":

Please, check if you have followed all the formatting requirements for the manuscript.

In my opinion, the literature sources (references) are not described well – for example, for some references, it is not clear whether it is a book or a journal article (for example, 1, 2 and not only them) – if it is in a journal, the journal name and pages are missing. Please review the bibliographic description of the literature sources again and correct. 

Lines 20, 21, 22: According to the authors: "This approach allows for the detection of balance changes even when backdoor codes are concealed. Experimental results verifying the effectiveness of this model demonstrate 100% accuracy, 1.0 recall, and 1.0 precision.” – in my opinion, these statements are too far-fetched. There are no ideal things in life, and therefore such excellent results are hardly possible – this is rather due to the small number of objects for study, for example, whether all factors are taken into account, whether the sample is sufficiently representative, etc.?

According to the authors: lines 50-51: “However, among the 68,268,300 smart contracts deployed on Ethereum, only 705,010 smart contracts disclose their source code, which is approximately 1% [10].” Later, the authors say that they studied only 200 numbers (lines 66-67: “The study aims to analyze 200 smart contracts for each attack type to measure accuracy, recall, and precision, thereby verifying the effectiveness of this model.” Isn’t this small number (barely 200) a prerequisite for obtaining these excellent results? Maybe if more numbers were taken or another 200 were taken, the results would not be so excellent!?

I recommend that the title line of Algorithm 2 be moved to the next page, not transferred in this way!

The authors provide algorithms for some types of Backdoor Attacks, but why not for all? Could you add such algorithms for the other types of attacks?

The subfigure text should be below the figure, not above it – as is the case for Figures 4…9.

It would be good if the final version of the article did not have a break in tables on two pages (if possible), as and only the section title should be on the previous page!

 

In Table 3 – there are two identical rows: Fee. Why? According to Fig. 3 – there is another model – Transaction limitation – in the case of Table 3 it is missing – maybe one of "Fee" should be the missing one!

In my opinion, the article needs some revision according to the comments - for example, results matching the ideal (100%) could not be obtained in life! Maybe the number of studied objects is small, maybe not all factors were taken into account.

Comments on the Quality of English Language

The English could be improved to more clearly express the research.

Author Response

Comments 1: The article is well written. The topic is extremely useful and relevant. Some recommendations for improving the manuscript "Detecting Rug-pull: Analyzing Smart Contract Backdoor Codes in Ethereum": Please, check if you have followed all the formatting requirements for the manuscript. In my opinion, the literature sources (references) are not described well – for example, for some references, it is not clear whether it is a book or a journal article (for example, 1, 2 and not only them) – if it is in a journal, the journal name and pages are missing. Please review the bibliographic description of the literature sources again and correct. 

Response 1: Firstly, thank you for your review. We have modified the issues you pointed out and highlighted the references in blue.

 

Comments 2: Lines 20, 21, 22: According to the authors: "This approach allows for the detection of balance changes even when backdoor codes are concealed. Experimental results verifying the effectiveness of this model demonstrate 100% accuracy, 1.0 recall, and 1.0 precision.” – in my opinion, these statements are too far-fetched. There are no ideal things in life, and therefore such excellent results are hardly possible – this is rather due to the small number of objects for study, for example, whether all factors are taken into account, whether the sample is sufficiently representative, etc.?According to the authors: lines 50-51: “However, among the 68,268,300 smart contracts deployed on Ethereum, only 705,010 smart contracts disclose their source code, which is approximately 1% [10].” Later, the authors say that they studied only 200 numbers (lines 66-67: “The study aims to analyze 200 smart contracts for each attack type to measure accuracy, recall, and precision, thereby verifying the effectiveness of this model.” Isn’t this small number (barely 200) a prerequisite for obtaining these excellent results? Maybe if more numbers were taken or another 200 were taken, the results would not be so excellent!? In my opinion, the article needs some revision according to the comments - for example, results matching the ideal (100%) could not be obtained in life! Maybe the number of studied objects is small, maybe not all factors were taken into account.

Response 2: We have modified the issues you pointed out. To alleviate concerns about the representativeness of the sample and the reliability of the results, we have additionally collected 189 EVM codes from smart contracts deployed on the actual Ethereum network, bringing the total number of experiments to 989. This dataset includes real instances of malicious backdoor codes, which helped us evaluate the practical applicability of our model. After re-running the experiments with the added data, the model's accuracy was found to be 98%. We have highlighted these changes and the additional experimental results in blue in Sections 4.1 and 4.3 of the manuscript.

 

Comments 3: I recommend that the title line of Algorithm 2 be moved to the next page, not transferred in this way! It would be good if the final version of the article did not have a break in tables on two pages (if possible), as and only the section title should be on the previous page!

Response 3: We have modified the issues you pointed out and adjusted Algorithm 5 (pp. 5–6) so that it is not split across two pages.

 

Comments 4: The authors provide algorithms for some types of Backdoor Attacks, but why not for all? Could you add such algorithms for the other types of attacks?

Response 4: We have modified the issues you pointed out and added algorithms and descriptions for Destroy Token and Funds Manipulation in Section 2.2 (pp. 4–5). These changes have been highlighted in blue.

 

Comments 5: The subfigure text should be below the figure, not above it – as is the case for Figures 4…9.

Response 5: We have modified the issues you pointed out and modified the placement of the subfigure text to be below the figures, and these modifications are highlighted in blue in Figures 4 …9.

 

Comments 6: In Table 3 – there are two identical rows: Fee. Why? According to Fig. 3 – there is another model – Transaction limitation – in the case of Table 3 it is missing – maybe one of "Fee" should be the missing one!

Response 6: We have modified the issue you pointed out and the changes are highlighted in blue in Table 3.

 

Reviewer 2 Report

Comments and Suggestions for Authors

Here a balance-tracking- based backdoor code detection model to identify backdoor codes in smart contracts is proposed for  detecting backdoor codes by extracting functions from Ethereum bytecodes and inspecting the  extracted functions to track balance changes. Related works are inclusive. With Figure 3  the model is described in a technical way in section 3. With Python 3.9.16 and Panoramix Decompiler 0.6.1  and the Ethereum Signature Database as of October 20, 2024, the results of the proposed model is unique like it has achieved 100% accuracy for all back- door code types. Also it has accurately detected  all backdoor codes in smart contracts where multiple backdoor codes existed simultaneously, this is unique too.

 

Now matter is this model is tested with Ethereum only so what about other Blockchain platforms?

Describe practical acceptance of this model in real-time. As only one platform testing is okey but very very limited.

So in other words this model is very limited. So address this matter in details how it can be incorporated in other platforms.

 

Author Response

Comments: Here a balance-tracking- based backdoor code detection model to identify backdoor codes in smart contracts is proposed for detecting backdoor codes by extracting functions from Ethereum bytecodes and inspecting the extracted functions to track balance changes. Related works are inclusive. With Figure 3 the model is described in a technical way in section 3. With Python 3.9.16 and Panoramix Decompiler 0.6.1 and the Ethereum Signature Database as of October 20, 2024, the results of the proposed model is unique like it has achieved 100% accuracy for all back- door code types. Also it has accurately detected all backdoor codes in smart contracts where multiple backdoor codes existed simultaneously, this is unique too.

Now matter is this model is tested with Ethereum only so what about other Blockchain platforms? Describe practical acceptance of this model in real-time. As only one platform testing is okey but very very limited. So in other words this model is very limited. So address this matter in details how it can be incorporated in other platforms.

Response : First, thank you for your review. In response to your concern that the current model is limited to the Ethereum platform and regarding the potential for expansion to other blockchain platforms, we have added Section 3.4 to the manuscript. We have provided a detailed interface for integrating the model with various platforms and the changes are highlighted in blue.

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

Dear Authors, 

Thank you very much for your responses!

I have reviewed the authors’ responses to my comments in detail! I am satisfied with the responses, as well as with the corrections made by the authors, marked in yellow (which also seem to satisfy the comments of the other reviewers). I believe that the quality of the manuscript has improved after reflecting on the reviewers’ comments and that it has acquired a more complete and complete appearance (for example, after inserting the other algorithms, as well as the edits to points 3.3 and 3.4).

In my opinion, the manuscript has improved significantly and could be published in the Journal!

Reviewer 2 Report

Comments and Suggestions for Authors

Now all the necessary changes are done, its a good contribution.