Blockchain Technologies and Digitalization in Function of Student Work Evaluation
, Ana Savic 3,*, Hana Stefanovic 4, Jelena Doganjic 5 and Zivanka Miladinovic Bogavac 6
Round 1
Reviewer 1 Report
The main objective and outline of the paper:
The paper presents a collaborative learning and student work elaboration model that applies the blockchain technology in higher education. The paper argues that blockchain technology can be used not exclusively for securing examinations but can be integrated in project management and peer evaluation. A CLSEW model was developed by the researchers and tested to see whether it can be effectively used in collaborative learning environments. Quantitative analysis was also conducted to explore the potential applications of blockchain technology in education. The topic is interesting, relevant and up-to-date. The establishment of more secure learning and exam circumstances is a must for educational institutions raging from elementary to higher educational institutions.
General comments:
The paper is structured well and needs some modifications and improvement.
The abstract summarized the essence of the paper, however, it should include the main focus of the paper, should describe more precisely the algorithms and methods used for the paper. The sentence in line 26 stands out from the abstract, it needs to be linked more precisely to the research. Furthermore, the concept of circular economy is only mentioned in the abstract and in the Discussion (line 704), there is no definition, it is not linked to the research. Furthermore, the abstract should mention the research method and the main findings.
The paper has 6 keywords, they fit the topic but need to be in alphabetical order.
The paper gives deep insight into the background of the research and discusses in detail the two main pillars of the research. The Methodology section is also elaborated well and describes in detail how the CLSEW model is built. (Please check the acronym for Public Key Infrastructure (PKI) In 3.3 Figure 5 calls for more explanation – the colours on the figure can also be used for clarification and can explain the two sides on the Figure. The sentence in Line 484-486 needs to be moved below Figure 5.
The evaluation section focuses on the results; however, it needs major improvements. It is very difficult to follow, authors seem to have tried to push all the results very densely in the section. The evaluation section starts with the results of a questionnaire; however, it was not mentioned in the methods section, no quantitative research was described. Line 592 mentioned qualitative analysis, but nothing is added to the research results on qualitative basis.
The authors failed to explain how the questionnaire was developed, what the response options were, if the Likert scale was used, how many response options the participants had. Validity and reliability need to be added to the results section. Since section 4 is difficult to follow, a table with the demographic profile of the respondents as well as aa table with the common descriptive results of the responses would be beneficial. Percentage distributions could be more easily followed if backed by a table.
Regarding Q9: Was the concept “blockchain technology” familiar to the respondents? Did the participants understand the response options? High bias can be detected in case of this question. Lines 681-682 need reference.
The Discussion is very short, it is rather a summary than a discussion. The in-depth discussion is missing, as the limitations of the research. Authors are asked to include a discussion section where findings are related to literature.
The paper needs some language editing, there are some misspelled words, and wrong conjunctions. It is advisable to use the impersonal “their” instead of him/her. In case of questionnaire, we talk about “responses” and not “answers”. Please correct these mistakes.
The paper has 8 figures, which are placed well in the paper and could support the research. However, the quality and size of the figures need improving. Some has tiny letters, other are too dark making the figure illegible. It would be beneficial to resize the figures. In case of figure 5 and 6 a work flow would help to support both the understanding of the process and the content of the figures.
The references are formatted well, they include a wide range reference, a good proportion of references are from the latest years (59.3%), they are well selected. In case of some references DOI is missing, please insert it. However, the in-text referencing must be corrected, please check the template and follow the requirements throughout the paper. Please check that the references are numbered in order of their appearance in the paper. Number 46 is missing. Some extra references are needed on page 4 in the paragraphs about collaborative learning (paragraph 2,3,4,9). The collaborative models are not introduced and referenced, while some of the principals are in italics while others are not. Putting each of them in italics would highlight them throughout the section. References are also needed in the section 3.3 – for the encoding algorithms, in line 495-529. The algorithm might be familiar to the readers, but it needs to be referenced.
The paper is highly relevant and interesting explores new modes of the application of block chain technology in education.
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
First of all, I would like to thank the authors for their contribution.
The introduction presents a good background for the study, but I propose "split" in different sub-sections because it is too long.
A more critical synthesis of the literature and potential construction of a research agenda will also improve the significance and value of the paper
Discussion of results needs to be clearer and there are no Conclusions.
Author(s) need to mention ethical issues for their study. I propose to add the following reference:
Petousi, V., & Sifaki, E. (2020). Contextualizing harm in the framework of research misconduct. Findings from a discourse analysis of scientific publications, International Journal of Sustainable Development, 23(3/4), 149-174, https://doi.org/10.1504/IJSD.2020.10037655
I wish you the best of luck with the revisions of your manuscript.
Comments for author File: Comments.pdf
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report
Blockchain technology, a system of Distributed Ledger Technology, has been gaining huge attention in areas beyond its cryptocurrency roots since more or less: blockchain and education, blockchain and cybersecurity, blockchain and finance, blockchain and logistics, etc.
This paper presents blockchain-technology-based collaborative learning and evaluation of the student work model.
Strengths:
The presentation was clear and very comprehensible.
In general, the paper presents appropriate organization and the points advanced logically.
This paper provides an overview of the development of the area, namely on the theoretical basis of collaborative learning and student work evaluation.
The Documentation of sources and references are appropriate and up-to-date
The figures and tables are appropriate.
The model proposed was validated.
Weaknesses
The limitations of the study could be presented.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 4 Report
- This study proposes a methodology using blockchain technology in collaborative learning and tries to explore the possibility in the education field, which is to the scope of the journal
- Since the evaluation is required for all students to evaluate her peers in the proposed model, it will be a daunting task if the student size is large (n(n-1) for a course with n students) for the lecturer. Do the authors have any recommendation of suitable student size for a course?
- For fig 3, it seems the arrow out from "Evaluation Block i+3" points to nowhere, it's advised to redraw the figure (using ... may be better than an arrow)
- About the authentication, it's suggested that the authors stay neutral with the applied technologies; RSA and SHA may evolve or be replaced with other more advanced algorithms. Therefore, a rewrite in section 3.3 to show the tech-neutrality is preferred. However, for the ongoing experiments mentioned by the authors, RSA and SHA are quite adequate.
- Some of the symbols used are not defined (Lines 506-517), please check.
- line 611, track-->tracked
- In section 4, it's advised to add more demographic information of the respondents. Educators' response should weigh in more importance.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 5 Report
The paper without any doubt presents some interesting thoughts and ideas. I doubt , however, whether the journal Sustainbailty is the right outlet for presenting these ideas. The name of the journal is Sustainabilty and I cannot see any direct link between the arguments offered and sustainabilty issues. One of the minor adjustments that have been made to the manuscript was using the word sustanaible once, but without any consequences for the overall presentation of the material. The „model“ can be used for all types of purposes and learning objectives. Sustainability might be one, but the specifics of applying the model to sustainabilty issues are not discussed. In the discussion section the term „circular economy“ is used, but again without any specific relationship to the overal argument.
Other than that I have problems understanding the status of the „model“. Is it a thought product or has it been implemented (as it is suggested then and there). If it has been implemented it would be nice to learn whether the far reaching expectations and assumptions linked to the model have in fact been observable. Generally it would be advisable to make a clearer distinction between the model and the far reaching, non substantiated expecations linked to it. Experiences with the implementation of learning technoogies in the past show that most far reaching expectations are hard to fulfil in the real time practice of a university or any other learning institution.
Some of the problems discussed (e.g. in the section detailing the results of a questionnaire) seem to be very region specific. The paper should make clear what the specific regional problems and opportunities are that the model intends to address. In my country the sharing of student data with non university actors is simply not viable and a technoogical solution for this non-problem is not needed. Furthermore, the learning managment system that we are using (without blockchain technology) does not allow students to make any changes on the data that have been stored in the system and even professors cannt make changes without permission once the original data have been registered. And diploma etc. are only granted once all necessary information on student accomplishments have been completely stored. The problem of diploma granted illegitimately does insofar not exist.
The model might be appropriate for a specific institutional environment but does not present a generic solution. This is fine, as long as the authors deal with these limitations (which also might be opportunities).
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
The research paper has been revised according to the requirements and following the recommendations of the reviewer. It has been significantly improved. Thank you for correcting the paper. The authors added the relevant parts, the abstract, the discussion as well as specific explanations were improved, and figures and tables were corrected and added. The methodology part was also improved.
However, there are still some improvements to be done.
The abstract is two long, the template asks for an abstract no longer than 200 words. Please check the requirements for the abstract in the template, and please shorten the abstract but keep all the necessary parts.
The acronym for Collaborative Learning and Student Work Elaboration (CLSWE) is written as the Collaborative Learning and Evaluation of Student Work (CLSWE) in line 135. Please check.
“that students are active in reasoning and are functionally active” – please check
1.1. Introduction of the Proposed Model – please check the wording as well as formatting. It needs to be formatted according to the template.
The intext references need correction, please carefully check the template. Where there are more than two references in a row the following needs to be used: [31-39]. When two or more works are referenced then please use [3,24,46]. Do not put each reference in individual brackets. Some references are superscript, please correct. Reference at the end of the sentence must be followed by the full stop. When there is a direct quote, please add the page numbers to avoid plagiarism (e.g. line 793, 806). Please be careful with the font size, they are not the same throughout the paper (abstract, line 521-522 for instance). There are two commas between keywords. Line 640 starts with a dot. Line 654: two full stops at the end of a sentence.
Line 483-485, be careful with the bulleted list – it must be indented.
English language proofreading and editing is still an issue in the paper, as for example line 540-548 needs revising, there are several language errors in the sentences – the use of definite and indefinite article, lack of verbs etc., ie needs to be written i.e. (e.g. line 820).
Table 1 does not display the unit of measurement and a word is misspelled. Please check the template for the format of the table. The same applies to Table 2, it is not formatted according to the template.
There needs to be a table for the % distribution of the responses for the questions. A table with % distribution would enable the reader to make comparisons and see the general view of the respondents’ opinion.
“On that way, quantification of result shows "average’" lecturers’ perception and standard deviation to understand the variability of the answers.” – Not the average lecturers’ perception but the lecturers’ average perception and standard deviation. Please correct. These are responses not answers, please correct. Please add the Mode and the Median, since a Likert scale was used.
I would recommend a thorough proofreading both for English editing and language checking, and formatting as well. There are numerous mistakes in the paper. After these revisions, the paper can be accepted for publication, since it is highly relevant and up-to-date.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 5 Report
Some changes have been made to the manuscript, but there has not been any significant re-orientation. Insofar my previous comments are still applicable.
