Bitcoin’s APIs in Open-Source Projects: Security Usability Evaluation†
Round 1
Reviewer 1 Report
The authors presented professional and in-depth knowledge in this area. The motivation of this work is good; writing is engaging and the flow is clear and logical. Minor grammatical or typing errors observed (see comment below). Some suggestions below:
- In Section 2 Related Work, the authors cited the guidelines in Mosqueira-Rey et al. (2018). The naming convention does not appear to have direct linkage with the guidelines in Table 4. While the guidelines in Table 4 were each explained moderately clearly in Section 4, some text should be present to explain the missing link. For instance, are those guidelines in Table 4 related to Mosqueira-Rey et al. (2018) or proposed by the authors? What is the relation?
- In Section 2 Related Work, line 167. The naming of categories can be improved. Specifically, it is unclear what "security learning" means even with the description. Further, what is the difference between "Development Processes" and "Tools"? The naming of "Software Design" is also too generic and unclear, it seems to be belonging to Behavioural rather than Programming.
- Table 1 is difficult to understand as well, and not standalone because of point #2.
- In Section 4, line 576, it is strange to make a literature review in this section.
- With regards to grammatical / typing errors, they are mainly found in Sections 0, 1, 2, and 3. The authors are recommended to proof-read these sections and the entire paper. Minor readability issue in Table 6 and in line 928, to check if "lib-Bitcoin" is intentional.
Other than the above, the reviewer would also like to share some views as below:
- In line 88, it is recommended to find out if Merkle refers to construction of the Merkle Hash Tree (MHT) and merkle_root refers to calculation of the MHT's root hash after the construction. These are two different processes and it would be good to find out before suggesting to change the naming convention.
- Bitcoin experienced an accidental hard fork in 2013 (https://bitcoinmagazine.com/articles/bitcoin-network-shaken-by-blockchain-fork-1363144448), would this work be relevant to prevent that?
Author Response
In Section 2 Related Work, the authors cited the guidelines in Mosqueira-Rey et al. (2018). The naming convention does not appear to have direct linkage with the guidelines in Table 4. While the guidelines in Table 4 were each explained moderately clearly in Section 4, some text should be present to explain the missing link. For instance, are those guidelines in Table 4 related to Mosqueira-Rey et al. (2018) or proposed by the authors? What is the relation?
We want to clearly state that the guidelines referred to in our paper are not proposed by us. The guideline naming convention used in our paper is, in fact, the very same used by Mosqueira-Rey et al., i.e. our guidelines are the ones proposed by Mosequira-Rey et al. referred to by the same names. Back to the technical points in this comment, the link is is made by KC-1 and KC-2 which are covered by our work as table 4 states. We have also shown that on page 11 when listed the checks check-cpp-api tool performs (i.e. KC-1-1). On page 11, we made it clear that those guidelines listed hroughout the paper are Mosqueira-Rey’s work. We are referring to line 254 that reads “check-cpp-api currently provides checks for the following Mosqueira-Rey et al. (2018) guidelines [21]:”. It was good to re-emphasise that on page 14 adding to the first statement of the last paragraph “…… by Mosqueira-Rey et al. (2018)” to clarify the issue further.
In Section 2 Related Work, line 167. The naming of categories can be improved. Specifically, it is unclear what "security learning" means even with the description. Further, what is the difference between "Development Processes" and "Tools"? The naming of "Software Design" is also too generic and unclear, it seems to be belonging to Behavioural rather than Programming.
a. For “security learning” name, it is a quite well-known name in the area of usable security and has been used by some figures such as Weir et al. (2016), Boopathi et al. (2015)
b. Development process refers to the existing behavioural techniques those related to the software development life cycle such as SSDLC, Dialect, etc. The tools are more into what the software developer technically use which covers mainly the tools used such as those security alarm generators etc.
c. We agree that the term “software design” is too generic and we changed it to “Implementation Choices” that reflects the nature of the concepts included in that category. Actually, both terms have been mentioned by Fabio et al (2014). The term “software quality” that has been used by Lavallée and Robillard (2015) would be a generic one as well but we are fine to adopt this should you think it is a better naming.
Table 1 is difficult to understand as well, and not standalone because of point #2.
In systematic literature survey, researchers used to list all related workpieces into a table that summarises the domain and shows the categories/concepts used in the domain. We have used this technique as discussed by Barbara and Stuart (2007). Yli-Huumo and Ko (2016) in their paper “Where Is Current Research on Blockchain Technology?” has used that, Merino et al (2018) in their paper “A systematic literature review of software visualization evaluation”, and Groenewald et al (208) in their paper “Understanding 3D Mid-Air Hand Gestures with Interactive Surfaces and Displays: A Systematic Literature Review”.
In Section 4, line 576, it is strange to make a literature review in this section.
Such part of the section is moved to the appropriate location in the literature survey section.
With regards to grammatical / typing errors, they are mainly found in Sections 0, 1, 2, and 3. The authors are recommended to proof-read these sections and the entire paper. Minor readability issue in Table 6 and in line 928, to check if "lib-Bitcoin" is intentional.
- Line 928 is corrected
- Table 6 is corrected
- The entire paper has been run against Grammarly and manual proofreading.
In line 88, it is recommended to find out if Merkle refers to construction of the Merkle Hash Tree (MHT) and merkle_root refers to calculation of the MHT's root hash after the construction. These are two different processes and it would be good to find out before suggesting to change the naming convention.
We are not sure where it is as the first mention of Merkle is on line 490. On 490, we explicitly said it is Merkle_root.
Bitcoin experienced an accidental hard fork in 2013 (https://bitcoinmagazine.com/articles/bitcoin-network-shaken-by-blockchain-fork-1363144448), would this work be relevant to prevent that?
Well, thanks for drawing our attention to this hiccup that seemed related to the protocol itself and the implementation of the protocol. Thus, our answer is no, our work would not have been able to prevent the hard fork since we looked at the API and its usability and not at the correctness of the protocol or implementation. The verification of Bitcoin’s security or the implementation of the protocols is outside the scope of this research. It is, however, appealing to investigate in the future. Some directions such as formal verification could be a possible option here.
Reviewer 2 Report
The investigated topic in the manuscript is very interesting and highly relevant to the new age information driven software world.
It is true that, we are nowhere near to the vulnerability-free software applications. As a researcher working in the field of blockchain we understood the need for ensuring the security. In that context, the topic of "API usability and usable security" presented in this manuscript is is highly interesting.
The manuscript is a well written piece and a lengthy one with 35 pages. Few sections in the manuscript has substantial repetitions which can be avoided. This could help in shortening the manuscript or in addition the some of the information for example, few listings can be moved to supplementary information.
The survey research methodology is clearly presented with the information sources. However, it would much appreciated if author(s) could provide a visual representation of the data selection based on the keywords. I strongly advice the author(s) to include a figure for methodology.
Minor comments:
The section numbers seems to be wrong, for example the introduction section have zero number, which should start with one. e.g.: 1. Introduction; 2. Research methodology; ..... and so on.
The introduction section can be further improved. The novelty of this survey is not clear. In addition, the research gap based on the literature study is not clear.
The conclusion section is more or less resemble as abstract. Its needs revisions and should highlight on the information drawn based on the survey outcomes.
Author Response
The manuscript is a well written piece and a lengthy one with 35 pages. Few sections in the manuscript has substantial repetitions which can be avoided. This could help in shortening the manuscript or in addition the some of the information for example, few listings can be moved to supplementary information.
We agree on the lengthy nature of the paper but it is mainly because it contains a Systematic literature survey which is normally that long as we have shown in our examples in the next review comment. We can still move those tables to an appendix but the Latex template does not seem to have allowed for that appendices.
The survey research methodology is clearly presented with the information sources. However, it would much appreciated if author(s) could provide a visual representation of the data selection based on the keywords. I strongly advice the author(s) to include a figure for methodology.
In a systematic literature survey, researchers used to list all related workpieces into a table that summarises the domain and shows the categories/concepts used in the domain. We have used this technique as discussed by Barbara and Stuart (2007). Yli-Huumo and Ko (2016) in their paper “Where Is Current Research on Blockchain Technology?” has used that, Merino et al (2018) in their paper “A systematic literature review of software visualization evaluation”, and Groenewald et al (208) in their paper “Understanding 3D Mid-Air Hand Gestures with Interactive Surfaces and Displays: A Systematic Literature Review”.
Minor comments:
The section numbers seems to be wrong, for example the introduction section have zero number, which should start with one. e.g.: 1. Introduction; 2. Research methodology; ..... and so on.
This is actually the Electronics journal’s Latex template created section numbers which we can’t change. We have run decent proofreading of the paper against Grammarly though.
The introduction section can be further improved. The novelty of this survey is not clear. In addition, the research gap based on the literature study is not clear.
We have added a paragraph at the end of that section to discuss that. We also clearly mentioned, “While some research has been done on the usability of Bitcoin's applications from an end user's point of view, as far as the authors of this paper know there is no research yet that addresses usability aspects from a developer's point of view” to highlight the gap in the literature.
The conclusion section is more or less resemble as abstract. Its needs revisions and should highlight on the information drawn based on the survey outcomes.
We modified that section framing the questions we answered and the research limitations as well.
