Advanced Network and System Security Teaching

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The proposed work for review covers a contemporary topic, although it is not in the thematic plan of the journal - presentation of a new, modern curriculum, which aims to prepare future specialists in the field of network security. The authors of the curriculum have approached it correctly, analyzing what the future trends are - more and more devices connected to the Internet and the need for their protection, by training people to provide network protection. The topic of the presented curriculum is contemporary, covering the basics for training future specialists in the field of network security.

Strengths of the work:
1. A very thorough review of similar developments in various databases has been made. Only those developments that are related to the topic of the presented work have been analyzed and presented;
2. A thorough analysis of the success rate of the introduced course has been made for the last four years - for the time since the course was launched

Notes and recommendations:
1. In addition to the literature review of various works on the topic in global scientific databases (Scopus, IeeeXplore, Elsevier, Google scholar and etc), did the authors also carried out a survey among various higher education institutions in Serbia and in other countries, to see if a similar course like theirs is offered there. Such a study is also important;
2. It would be good the authors to provide, if such a survey has been carried out, how many of the graduates have been working in the same field. It would be also good to provide results of an other survey that shows how many of the graduates have the knowledge and skills gained from the course helped them during their work or in some other situations. The results of such surveys will give the best attestation of the presented course and it will be possible to state how successful it is;
3. Did the authors have conducted a survey among employers or companies that deal with network security for their opinion on whether the proposed curriculum - in terms of content - meets the current needs of the labor market. The results of such a study will additionally contribute to the better attestation of the presented course;
4. Some of the tables, such as tables 5, 6, 7, 8, should be moved. They should be located immediately after the paragraph in which they are mentioned for the first time. Their current location makes it difficult to reading and understanding the presented results;
5. The figures should be made larger so that they are better visible and placed immediately after the paragraph where they are mentioned for the first time;

Author Response

Comments 1: In addition to the literature review of various works on the topic in global scientific databases (Scopus, IeeeXplore, Elsevier, Google scholar and etc), did the authors also carried out a survey among various higher education institutions in Serbia and in other countries, to see if a similar course like theirs is offered there. Such a study is also important;

Response 1: Thank you for pointing this out. Such research was conducted in 2017 as part of the ISSES project, and there was no similar course available in Serbia. The results of the project, including the course, were made available to other universities in Serbia. However, unfortunately, the course was either not implemented elsewhere, or the number of interested students has been very small so far.

Comments 2: It would be good the authors to provide, if such a survey has been carried out, how many of the graduates have been working in the same field. It would be also good to provide results of an other survey that shows how many of the graduates have the knowledge and skills gained from the course helped them during their work or in some other situations. The results of such surveys will give the best attestation of the presented course and it will be possible to state how successful it is;

Response 2: Agreed, there are surveys conducted with students who have graduated, but they are carried out periodically every seven years as part of the accreditation process. We will include the proposed question in the survey during the next opportunity, if possible.

Comments 3: Did the authors have conducted a survey among employers or companies that deal with network security for their opinion on whether the proposed curriculum - in terms of content - meets the current needs of the labor market. The results of such a study will additionally contribute to the better attestation of the presented course;

Response 3: Same response as for comment 2.

Comments 4: Some of the tables, such as tables 5, 6, 7, 8, should be moved. They should be located immediately after the paragraph in which they are mentioned for the first time. Their current location makes it difficult to reading and understanding the presented results;

Response 4: Agreed, this was changed in the manuscript.

Comments 5: The figures should be made larger so that they are better visible and placed immediately after the paragraph where they are mentioned for the first time;

Response 5: Agreed, this was changed in the manuscript.

Reviewer 2 Report

Comments and Suggestions for Authors

I have the following comments:

1. Related works. While the paper mentions a comparison with 44 other related courses, the methodology for selecting and evaluating these courses is not explained. It would be helpful to include a clear description of the criteria used for comparison and how the data was collected and analyzed. It would also be good to provide a more detailed discussion of the related work's limitations and gaps and how the ANS course uniquely addresses them.

2. The flexibility and scalability of the laboratory environment are highlighted, but the technical details of the implementation could be expanded. For instance: what specific tools, platforms, or technologies were used? How were practical exercises designed to simulate real-world scenarios?

3. The paper mentions both qualitative and quantitative analyses, but the methodology and metrics for these evaluations are not clearly defined. For instance: What specific metrics were used to measure the improvement in practical skills? How were students’ qualitative feedback and creativity evaluated?

Author Response

Comments 1: Related works. While the paper mentions a comparison with 44 other related courses, the methodology for selecting and evaluating these courses is not explained. It would be helpful to include a clear description of the criteria used for comparison and how the data was collected and analyzed. It would also be good to provide a more detailed discussion of the related work's limitations and gaps and how the ANS course uniquely addresses them.

Response 1: Thank you for these comments. We have, accordingly, modified the Related Work section to further emphasize the search process methodology by adding the following (added in line 87, section 2.1., page 2 and line 117, section 2.1., page 3):

Each found paper was analyzed through three eliminatory steps:

1. Analysis and Comparison of the Paper Title with Selection Criteria
2. Analysis and Comparison of the Paper Abstract with Selection Criteria
3. Analysis of the Paper itself for extraction of characteristics

The key characteristics of the papers and laboratory environments described in them were extracted by gathering information directly from the papers (and the corresponding appendices and accompanying information if there are any). As will be observed in the following chapters, some characteristics of the environments remain undefined, which is due to the inability to derive these features solely from the content of the papers or the accompanying information or appendices.

We have also, accordingly, further elaborated upon the related work's limitations and gaps and how the ANS course uniquely addresses them by adding the following (added in line 151, section 2.2., page 6, and, line 194, section 2.3., page 10):

Due to this fact, students who passed this course gained comprehensive knowledge of this industry field, enabling them to select aspects they find interesting for potential further specialization, which is not facilitated by other presented courses.

From the table it can be seen that almost 50% (18 laboratory environments) do not have the ability for remote access, while the ANS laboratory does. In addition, 10 laboratory environments clearly present the need for scheduling, while the ANS laboratory is flexible and allows students to access it whenever they desire.

It can be observed also that many of the laboratory environments do not present a way of high configuration (therefore many of the laboratory environments have a low configuration level). This is mostly due to the fact that the description presented in the paper either does not delve into the details of configuration of the laboratory environment, or the presented environments are described in such a way that they only fit into the mold of the already mentioned set of exercises and therefore can't be changed for any other. The ANS laboratory environment, due to its approach to design, where virtual machines are used to build it, allows the expansion by installing any new scenario either on the existing virtual machines, or adding a new one to fit the purposes of the new exercise (the only criteria here is that there is storage for the new virtual machines). The details of design and configurability of the laboratory environment will be expanded on in the following section.

Comments 2: The flexibility and scalability of the laboratory environment are highlighted, but the technical details of the implementation could be expanded. For instance: what specific tools, platforms, or technologies were used? How were practical exercises designed to simulate real-world scenarios?

Response 2: Thank you for pointing this out.  The following was added in accordance with the comments, in section 3 (changes were made in section 3.1., page 12, line 273 and section 3.2., page 13, line 313):

The laboratory consists of four ThinkSystem SR550 servers with Intel(R) Xeon(R) Gold 5120 processors with 56 cores, 256 GB RAM and 3.5TB disks each. These servers form a cluster powered by VMware ESXi, 8.0.3. One laboratory environment requires approximately 11GB of RAM in basic configuration, allowing 84 concurrent and fully isolated laboratories.

To increase the practicality and applicability of knowledge from the practical test, scenarios were designed to include modern technologies that students would encounter in practice, for example Spring Boot for the web server application.

Comments 3: The paper mentions both qualitative and quantitative analyses, but the methodology and metrics for these evaluations are not clearly defined. For instance: What specific metrics were used to measure the improvement in practical skills? How were students’ qualitative feedback and creativity evaluated?

Response 3: Agreed. Section 5 was further elaborated upon to include descriptions of the requested measurements. The following was added (section 5., page 18, line 491):

…shown through the correlation between the practical test results and their final grade and general results in the course; and positive feedback for the practical part of the course.

Reviewer 3 Report

Comments and Suggestions for Authors

This paper mainly discusses the "Advanced Network and System Security (ANS)" course offered at the School of Electrical Engineering, University of Belgrade, Serbia. To address the shortage of cybersecurity experts, the School of Electrical Engineering at the University of Belgrade has been offering the ANS course since the 2019/2020 academic year. The course covers topics such as computer system and network security, intrusion detection and prevention, as well as ethical hacking methods. The paper provides a detailed introduction to the laboratory environment and exercises related to the course, emphasizing the importance of a multidisciplinary laboratory environment for successful learning outcomes. However, there are still some issues:

1. Although the paper mentions that the course covers a wide range of cybersecurity fields, it may need to further discuss how these areas align with current cybersecurity trends and needs.

2. The paper mentions the modular design of the laboratory environment, but it may need to further explore how to easily expand to adapt to the evolving field of cybersecurity.

3. The paper compares the ANS course with other related courses, but it may need to analyze in more detail the specific differences between these courses and the unique advantages of the ANS course.

4. The paper may need to provide a more detailed explanation of the methods used for data collection and analysis to ensure the validity and reliability of the research results.

Author Response

Comments 1: Although the paper mentions that the course covers a wide range of cybersecurity fields, it may need to further discuss how these areas align with current cybersecurity trends and needs.

Response 1: Thank you for pointing this out. This was further elaborated upon, by adding the following in section 2.2., line 130,  page 4:

…which is developed by world experts, and therefore aligns with the with current cybersecurity trends and needs.

Comments 2: The paper mentions the modular design of the laboratory environment, but it may need to further explore how to easily expand to adapt to the evolving field of cybersecurity.

Response 2: Agreed. We have, accordingly, further elaborated upon this in section 2.3. (added in line 198, section 2.3., page 10):

It can be observed also that many of the laboratory environments do not present a way of high configuration (therefore many of the laboratory environments have a low configuration level). This is mostly due to the fact that the description presented in the paper either does not delve into the details of configuration of the laboratory environment, or the presented environments are described in such a way that they only fit into the mold of the already mentioned set of exercises and therefore can't be changed for any other. The ANS laboratory environment, due to its approach to design, where virtual machines are used to build it, allows the expansion by installing any new scenario either on the existing virtual machines, or adding a new one to fit the purposes of the new exercise (the only criteria here is that there is storage for the new virtual machines). The details of design and configurability of the laboratory environment will be expanded on in the following section.

Comments 3: The paper compares the ANS course with other related courses, but it may need to analyze in more detail the specific differences between these courses and the unique advantages of the ANS course.

Response 3: Agreed. We have, accordingly, further elaborated upon the related work's limitations and gaps and how the ANS course uniquely addresses them by adding the following (added in line 151, section 2.2., page 6, and, line 194, section 2.3., page 10):

Due to this fact, students who passed this course gained comprehensive knowledge of this industry field, enabling them to select aspects they find interesting for potential further specialization, which is not facilitated by other presented courses.

From the table it can be seen that almost 50% (18 laboratory environments) do not have the ability for remote access, while the ANS laboratory does. In addition, 10 laboratory environments clearly present the need for scheduling, while the ANS laboratory is flexible and allows students to access it whenever they desire.

It can be observed also that many of the laboratory environments do not present a way of high configuration (therefore many of the laboratory environments have a low configuration level). This is mostly due to the fact that the description presented in the paper either does not delve into the details of configuration of the laboratory environment, or the presented environments are described in such a way that they only fit into the mold of the already mentioned set of exercises and therefore can't be changed for any other. The ANS laboratory environment, due to its approach to design, where virtual machines are used to build it, allows the expansion by installing any new scenario either on the existing virtual machines, or adding a new one to fit the purposes of the new exercise (the only criteria here is that there is storage for the new virtual machines). The details of design and configurability of the laboratory environment will be expanded on in the following section.

Comments 4: The paper may need to provide a more detailed explanation of the methods used for data collection and analysis to ensure the validity and reliability of the research results.

Response 4: Thank you for pointing this out. The following changes were made in accordance with your feedback, with the following additions being made (added in section 4.1., page 13, line 323, and section 4.2., page 15, line 415):

Quantitative analysis was performed based on data about student achievement, which was collected in the analyzed period.

They were anonymous and voluntary, with Google Forms being used to implement the surveys.

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

It is a pity that you cannot provide in the work surveys with graduates, showing their professional realization. The results of these surveys would best evaluate the presented academic discipline.

I do not have any other remarks or recommendations.

Best regards.

Reviewer 2 Report

Comments and Suggestions for Authors

The revised manuscript looks good to me. I don't have much concerns.

Reviewer 3 Report

Comments and Suggestions for Authors

No further comments.