2.1. Modern Cyber-Security Training Platforms
Nowadays, a high variety of research and commercial platforms is available for cyber-security training for both individuals and organizations. A comparison of them with our method is presented in Table 1
and is detailed in [24
Usually, most of the general-purpose e-learning platforms (e.g., Coursera (Mountain View, CA, USA, 2012–2020), Udacity (Mountain View, CA, USA, 2011–2020), edX (MA, USA, 2012–2020), etc.) offer introductory and main educational courses on cyber-security. On the other hand, specialized solutions, such as the SANS (Bethesda, MD, USA, 2000–2020) [11
], CyberInternAcademy (MO, USA, 2017–2020) [12
], StationX (London, UK, 1996–2020) [13
], Cybrary (College Park, MD, USA, 2016–2020) [14
], and AwareGO (Reykjavík, Iceland, 2011–2020) [15
], support more advance and focused training. In most cases, all these approaches target individuals whose goal is to develop/sharpen new skills. However, they fail to provide hands-on experience on real systems or even cyber-ranges. Modern cyber-ranges platforms, such as BeOne (Hilversum, The Netherlands, 2013–2020) [16
], ISACA’s CyberSecurity Nexus (CSX) (Rolling Meadows, IL, USA, 1967–2020) [17
], Kaspersky (Moscow, Russia, 1997–2020) [18
], and CyberBit (Raanana, Israel, 2019–2020) [19
], offer more advance features.
THREAT-ARREST combines all modern training aspects of serious gaming [25
], emulation and simulation in a concrete manner [27
], and offers continuous security assurance and programme adaptation based on the trainee’s performance and skills (Table 1
). The platform [24
] offers training on known and/or new advanced cyber-attack scenarios, taking different types of action against them, including: preparedness, detection and analysis, incident response, and post incident response actions. The THREAT-ARREST platform supports the use of security testing, monitoring and assessment tools at different layers in the implementation stack, including:
Network layer tools (e.g., intrusion detection systems, firewalls, honeypots/honeynet);
Infrastructure layer tools (e.g., security monitors, passive and active penetration testing tools (e.g., configuration testing, SSL/TLS testing);
Application layer tools (e.g., security monitors, code analysis, as well as passive and active penetration testing tools such as authentication testing, database testing, session management testing, data validation and injection testing).
The procedure begins by analyzing the organization’s system. The Assurance Tool [28
] evaluates the current security level and reports the most significant security issues that must drive the following training process. Then, hybrid training programmes are produced, and tailored to the organizational needs and the trainee types. This includes the main training material along with serious games, as well as the simulation and emulation of the cyber range system. THREAT-ARREST also provides continuous evaluation of: (a) the performance of individual trainees in specific training programmes; and (b) the effectiveness of training programmes across sub-groups of trainees or the entire organization. These evaluations are used to tailor programmes to the needs of individual trainees or alter them at a more macroscopic level.
The whole operation is defined under a methodology called “Cyber Threat and Training Preparation (CTTP) modelling” [24
], which determines the learning goals of a training programme, the learning path of the trainee, as well as how to drive the on-demand instantiation of the virtual labs with the advance cyber-ranges features for these programmes and assess the trainee’s actions automatically.
This article documents this latest characteristic of the THREAT-ARREST platform and the CTTP modelling concept (see Section 2.3
and Section 3
). Moreover, the scope of a CTTP programme can be aligned with cyber-security professional specialization programmes, e.g., from ISACA or ISC2
. Therefore, the dynamic adaptation of the training process and the continuous improvement and building of skills constitutes a novel and competitive feature of the THREAT-ARREST solution.
2.2. Teaching Cyber-Security
Surveys concerning cyber-security exercises are reported in [29
]. ISO-22398 [32
] is the international standard that defines several exercise methodologies, such as seminars, simulations, workshops, tabletops and serious games, capture the flag (CTF), red/blue team, etc. These techniques provide hands-on experience to trainees and can assist the development of technical skills. The educational process may involve serious games, simulation with virtual labs, and/or collaboration learning. Although the importance of pedagogical aspects in exercises is recognized in the literature [33
], it has not been adequately studied and covered by researchers and practitioners, respectively [33
To support effectual training, one has to understand how expertise is built and which educational approaches can improve the trainee’s performance [29
]. Ordinarily, skills’ development and behavioral learning start with lecture-oriented teaching. As the trainee’s knowledgeable capacity increases, his/her “cognitive learning” is enhanced. Then, deeper knowledge on the subject can be built, by moving to “constructivist learning” approaches that mostly utilize exploratory learning [34
] (react to learning as a researcher) and problem-based learning [36
] (begin by resolving an actual problem and examining the relevant background information). Studies on university students [37
] reveal that reaching a high-order of thinking and understanding becomes critical and of great importance in the cyber-security field. Although students successfully complete a relevant course and know (cognitive learning) the main concepts, they usually incorrectly reason about the application of core notions (constructivist learning), such as the differences between confidentiality/integrity or authentication/authorization.
Ericsson defined a well-established Deliberate Practice (DP) theory [38
] for the continuous skills’ improvement. Thereupon, students require well-specialized goals that improve a specific area of expertise in their field, while on the other hand, they are “not benefitting by tasks which can be completed in an automated fashion”. The full achievements of the DP approach can be accomplished when the trainee reaches the highest layers on the Miller’s pyramid [39
]—an educational method for assessing the trainee’s competence based on four levels of: “Knows”, “Knows how”, “Shows how”, and “Does”. Cyber Security Exercises (CSEs) [40
] is a novel educational methodology for cyber-security that combines the aforementioned pedagogical approaches. An exercise is defined in three phases of: (i) planning the scope and objectives, (ii) implementation, and (iii) evaluation/feedback. This also complies with the relevant phases defined by the MITRE corporation [41
] (exercise planning, exercise execution, and post exercise). At the planning stage, the trainer identifies the scope of the exercise, the involved security aspects, and the pedagogical methods, as well as which elements will be simulated during the exercise and the scenario steps. During the implementation stage, the trainer monitors the students and tries to handle events and incidents, driving the students to pass through all learning goals. The process is based on the Boyd’s Observe-Orient-Decide-Act (OODA) loop [42
]. In the feedback stage, the students and the trainer go through all the main exercise elements. This is the most valuable phase for the individuals as they can ask questions on the underlying concepts, which will hopefully lead to the achievement of the defined learning objectives.
The study in [43
] indicates that students can reach competence in cyber-security only via hands-on learning with virtual labs led by an instructor. Therefore, a proper training programme must incorporate a series of good content and tutor interaction, pedagogical framework, and essential virtualized exercises for hands-on interplay. In [44
], researchers propose a technology-enhanced pedagogical framework for training with virtual labs. The process starts by applying the Constructive Alignment [22
] (map intended learning outcomes with deployed teaching activities) for the design of the curriculum. The learning follows the Kolb’s experiential learning cycle [23
] (disassembled in four subsequent phases of learning for “Concrete Experience”, “Reflective Observation”, “Abstract Conceptualization”, and “Active Experimentation”) and the educational elements are categorized based on the Bloom’s Taxonomy [45
] (method for the classification of learning objectives into levels of complexity and specificity). Collaborative learning may also be supported for team work. The students are evaluated via on-line quizzes and discussion boards.
Several studies also examine the inclusion of modern gamification techniques in the learning process [46
]. The implication of serious games is generally considered positive, as the trainee can become familiarized with the involved topics in a more relaxed manner, even in his/her free time.
Another aspect that is usually neglected in cyber-security training programmes is “psychology”. This affects both the attacker and the threat model—motivation to devote effort and launch an attack; and the legitimate user-communication/team-working skills, tendency to ignore warnings or defined procedures, etc. These issues are examined in [48
]. The “age, sex, or cultural background may make a person more subjectable to some malicious behavior”. Thus, despite their familiarization with technology, young people may be at greater risk of being tricked by phishing emails than older ones. Moreover, “different type of trainees has diverse expectations” from a cyber-security course. For instance, computer science students are mostly interested on how an attack can be performed, while psychology students focus more on why someone would exploit a vulnerability and harm a system or a person, and general public may be concerned about the side-effects of a successful hit.
Other challenging issues [49
] include: (i) the “dynamicity” of the Computer Science, (ii) the “workforce needs” and the requirement for industry standards, and (iii) a “common taxonomy” for threats and the underlying security properties. A modern curriculum design methodology must be able to easily align in the continuous evolving Computer Science and cyber-security fields [49
]. Moreover, training programmes should cover the current threat landscape and potentially lead to a professional certification [50
]. A common vocabulary across all these aspects must be followed by a well-established programme or body of programmes [50
The THREAT-ARREST platform supports a model-driven operation based on a methodology called CTTP modelling, which administrates the whole training process. At first, experts examine a piloting system (i.e., for smart shipping, healthcare, and smart energy) and record its main components, user types, etc. The core CTTP sub-model defines how a digital twin of this system can be instantiated on the developed Emulation and Simulation tools. Thereupon, the experts also apply the STRIDE threat model [21
] in order to capture the current security status of the piloting system, including the potential threats, vulnerabilities, and the proper deployment of the required defense mechanisms. This information is also part of the core sub-model (a well-structured XML or JSON format [28
]) and offers a common and widely-used vocabulary across the whole training experience.
Based on the analysis outcomes, we identify the most critical security aspects for the examined organization and tailor a training programme to its needs. The training perspectives are recorded in the training sub-model. This includes the learning objectives for each trainee type and the organization as a whole, as well as the dynamic adaptation and skill development features that are presented in this article (Section 3
and Section 4
The trainer defines complete training programmes with ordinary training material (e.g., lectures, tutorials, etc.), serious games, and virtual labs (emulated and simulated scenarios). The learning path for a programme is consisted by a series of CTTP models. Each model defines which of these modules will be activated and their correlation with the learning objectives (Constructive Alignment). The model-driven approach enables us to provide a high variety of CTTP models where different scenarios of escalated difficulty are activated based on the trainee’s type, expectations, and performance. The variations of a model are mapped in the Bloom’s taxonomy. The trainee begins the training by building the basis of the cognitive learning and then proceeding to constructivist learning and high-order thinking. Multi-user CTTP models are also supported (i.e., red/blue team and advance CTF scenarios), offering also collaborative learning opportunities. Thus, the successful learning of a security (or other) topic is performed in several iterations based on the Kolb’s learning cycle. Moreover, the programmes curriculum can correlated with professional specification bodies, such as those from ISACA and ISC2, and learning outcomes of the models and the programme as a whole are mapped based on the Constructive Alignment methodology.