PocketCTF: A Fully Featured Approach for Hosting Portable Attack and Defense Cybersecurity Exercises

: Capture the ﬂag (CTF) challenges are broadly used for engaging trainees in the technical aspects of cybersecurity, maintaining hands-on lab exercises, and integrating gamiﬁcation elements. However, deploying the appropriate digital environment for conducting cybersecurity exercises can be challenging and typically requires a lot of effort and system resources by educators. In this paper, we present PocketCTF, an extensible and fully independent CTF platform, open to educators to run realistic virtual labs to host cybersecurity exercises in their classrooms. PocketCTF is based on containerization technologies to minimize the deployment effort and to utilize less system resources. A proof-of-concept implementation demonstrates the feasibility of deploying CTF challenges that allows the trainees to engage not only in offensive security but also in defensive tasks that have to be conducted during cybersecurity incidents. When using PocketCTF, educators can deploy hands-on labs, spending less time on the deployment and without necessarily having the advanced technical background to deploy complex labs and scenarios. Contributions: Conceptualization, S.K. C.N.; methodology, S.K. C.N.; software, S.K.; validation, S.K., C.N. and L.L.R.; analysis, S.K.; investigation, S.K. and L.L.R.; resources, L.C.; data curation, S.K., C.N.; writing—original preparation, S.K.; writing—review and editing, C.N. and E.M.; visualization, S.K.; supervision, E.M.; project administration, L.C.


Introduction
Capture the flag (CTF) challenges and virtual labs for organizing hands-on cybersecurity exercises have recently become very popular worldwide [1,2]. Presented mostly during hacking conferences such as DEF CON [1] or Black Hat, CTF challenges are frequently being used by professors and other educators as well [3][4][5][6][7][8]. A significant benefit of CTF and virtual labs is the capability to replicate realistic cybersecurity scenarios that can engage students and practitioners in offensive tactics [6]. Moreover, CTF challenges can be used inside and outside of the classroom by assigning exercises to the students since self-learning attributes can be integrated [9]. However, in hosting a CTF, there seems to be a tradeoff between realistic, real-world exercises and infrastructure resources [2]. In particular, for the purpose of deploying realistic virtual labs and CTF challenges, a complex infrastructure has to be set up that must include all of the important services, from vulnerable services to defensive tools and CTF platforms to host the flags.
In the past, creating authentic computer security scenarios has been identified as a very demanding and challenging task that requires a great deal of effort from educators and lab personnel [10]. In addition, the learning outcomes from using hands-on practices need to follow curriculum guidelines or frameworks that address the required collaborative activities in cybersecurity within the industry, government, and academic institutions [2,10,11]. What is more, educators are usually in need of a way to provide customized challenges according to the courses and to privately maintain the overall learning process. In this direction, open virtual labs can better support the educator requisites, such as Vulnhub [12], which is an open repository that provides hands-on lab cybersecurity exercises as virtual images [3,11,13]. Similarly, the SEED labs [14,15] maintain virtual labs, and ENISA CSIRT [16]

Related Work
The related work in the area of CTF platforms is vast and includes many interesting works that propose various approaches to design, implement, and evaluate its role in cybersecurity education. Here, we present a subset of previous works that are the most relevant to the PocketCTF approach. The idea of using LXC containers or Dockers instead of virtual machines for CTF labs has been under research over the last few years [11,30]. More specifically, Irvine et al. introduced a framework for parameterizing cybersecurity labs by using Docker containers instead of virtualization technologies [24]. In another research study [33], a cyber range for hosting web attacks was deployed on a Raspberry Pi. The researchers demonstrated that virtual labs can be deployed by using only containerization technologies. The key benefits of using containers instead of virtual machines include the usage of fewer resources, allowing the educators to deploy a higher number of systems and services that are easier compared to virtual machines. Similarly, AlSalamah et al. [34] Information 2021, 12, 318 3 of 13 analyzed how containerization techniques open new possibilities for cybersecurity educational labs, highlighting the difference between virtual machines and containers. Likewise, research has been conducted in terms of the overall architecture and the toolsets that can be used for providing educational cyberspaces and for creating hands-on lab exercises for cybersecurity [14].
Other approaches include various reinforced learning perspectives utilizing simulation and emulation processes derived from complex CTF deployments and versatile cyber ranges [35][36][37][38][39]. Complex CTF deployments could include workstations, firewalls, switches, web services, and tools that are frequently used by red or blue teams to create high-fidelity training environments [40][41][42]. Such approaches typically appear in cyber range platforms and not in CTF deployments [43,44]. What is more, network emulation has been studied along with virtualization and attack emulation for replicating realistic scenarios and for security evaluation purposes [45]. Moreover, the benefits of virtualization and Dockers have also been considered by analyzing their capabilities for network virtualization [46,47]. The networking functionalities of containerization technology in the content of lightweight virtualization instances have been under research [30,48]. Finally, the capabilities of cyber ranges to use virtualization technologies to easily recover or to reproduce laboratory scenarios have been also studied [49].
Regarding performance evaluation, research endeavors have tried to characterize the CPU and disk I/O overhead introduced by Docker [50]. Furthermore, extensive performance evaluation of the Docker containers has been also conducted, and the benefits of containerization in comparison to virtual machines have been highlighted [51,52]. Another work conducted a performance comparison between LXC and Docker containers [53]. Moreover, in our previous work [54], we performed an initial performance comparison between several virtualization and containerization technologies. Based on the results of [54], here, we select the most appropriate technology to design and implement the Pock-etCTF platform. In particular, in this paper, we propose the deployment of CTF scenarios using LXC containers that can run isolated instances of Linux environments, providing an individual cyberspace for each participant/trainee. What is more, our focal point relies on blue teaming and defensive actions (e.g., attack detection) in contrast to all of the previous works, which only deploy vulnerable services for offensive actions without mentioning defensive tasks.

Virtualization and Containerization Technologies
The main goal of PocketCTF is to deploy cybersecurity scenarios and multiple services that are required for offensive and defensive tasks without significantly increasing the total performance overhead. In this section, we discover the benefits of containerization technology and how it can serve our purposes.

Overview
Virtualization and containerization are the two most frequently used mechanisms to host applications in a computer system. Virtualization uses the notion of a virtual machine to simulate hardware devices for running a separate operating system (i.e., guest OS). There are various virtualization technologies, including the Kernel-based Virtual Machine (KVM), to run Virtual Machines.
Contrary to virtual machines, containers can be used to host the OS to run applications and services. Containers use the resource isolation features of the Linux kernel (such as cgroups and kernel namespaces), which allows them to run within a single Linux instance, avoiding the overhead of starting and maintaining virtual machines. It is also possible to use containers as a lightweight virtualization option and even initiate graphical user interfaces by installing the related software packages. A significant benefit of containers compared to virtual machines is that the former is easier to manage than the latter and requires fewer system resources as well. Therefore, using containers, we can create complex network topologies requiring less deployment and integration effort. On the other hand, the main downside of containers is related to the fact that they share access to the kernel of the host, introducing security issues. Consequently, malware can gain access to the main host if it manages to escape from the container. The two basic containerization technologies are Dockers and LXC. Dockers (i.e., the most popular container technology) are lightweight, standalone, executable packages that include everything required to run applications. Docker is a single-purpose virtualization application, while LXC is multi-purpose operating system virtualization. On the contrary, LXC focuses on OS containerization being capable of deploying systems that are closer to an operating system and maintaining separate Linux kernels. In both technologies, a base host image is used to deploy the containers by including the components and software packages in the base host image. Therefore, each container is considered to be a running instance of the base host image. The main differences between Dockers and LXC are elaborated on in [55]. A summary of the differences between virtual machines and containers is presented in Table 1. Table 1. Summary matrix for benefits and drawbacks for virtual machines and containers.

Virtual Machines Containers
Guest OS Each virtual system runs on virtual hardware, and the kernel is loaded into its virtual memory.
All the guests share the same kernel loaded in the physical memory.

Isolation
Libraries and files are completely isolated.
Directories can be mounted and can be shared between the containers and the physical machine.

Performance
All instructions need to be translated between virtual systems and the physical machine, which incurs a performance decrease.
Near-native host performance and is especially better in terms of I/O interruptions.

Communication
Full virtualization of network devices. A special driver is assigned for connecting the containers to network interfaces.

Storage
Need a large amount of disk space as each virtual system needs to store the whole OS and associated applications.
Containers require less amount of storage as the base host image is shared among the containers.

Memory requirements
Virtual machines create a unique kernel that requires a significant amount of memory

Containers do not have significant memory requirements
There is also the ability to run a virtual machine inside another virtual machine. This is called nested virtualization and offers great flexibility and cost savings. Nested virtualization is important for cloud computing, as deploying VMs on top of Infrastructure-as-a-Service cloud providers is becoming more commonplace and requires nested virtualization support [56,57]. However, poor nested virtualization performance remains a key issue. In this paper, we define the notion of nested containerization, which is the execution of a container inside another container. Combinations can be also considered, such as running a container inside a virtual machine. As we mention below, nested containerization is a key technological component in PocketCTF architecture. In Table 2, the capability of supporting nested KVM, Dockers, and LXC is presented. We observe that a KVM virtual machine can initiate all technologies. Thus, it is possible to initiate a KVM-based virtual machine inside another KVM virtual machine (nested virtualization). We also observe that it is also possible to initiate a Docker container inside another Docker instance. More importantly, we can deploy Docker containers inside an LXC. On the contrary, it is not possible to deploy an LXC container inside a Docker. Even if the deployment of Docker containers inside a Docker is possible, it requires specific privileges from the main Docker daemon (namely privileged mode). On the contrary, LXC can bypass the requirement for the privileged mode and uses nesting features to enable the deployment of Docker containers internally without acquiring privileges from the main host. To this end, the design principles of LXC are more focused on the creation of environments that are as close as possible to a standard Linux installation. Therefore, as a technology, LXC is much closer to a complete OS environment with attached networking and storage interfaces, while the filesystem is just an abstraction for Docker.
As mentioned above, LXC containers are not executed in a privileged mode, a feature that is required to deploy a Docker inside a Docker. This is an important concept from a security point of view, as a CTF platform is a multitenant environment, and security should be taken into consideration to avoid any malicious actions that may threaten the integrity of the whole platform. On the other hand, running LXC containers in non-privileged mode thwarts privilege escalation attacks that are possible in nested Dockers running in Dockers since the latter runs in privileged mode. Furthermore, LXC uses AppArmor [58] to ship with a default security profile intended to protect the host from misuses inside the container.

Performance Evaluation of Virtualization and Containerization Technologies
It is important to conduct a performance evaluation and to deploy test cases for investigating the benefits and drawbacks of each virtualization technology. In particular, the purpose of the experiment was to discover the performance capabilities and to measure the total overhead of each virtualization and containerization technology (with or without nested virtualization or containerization). To this end, we have considered the following scenarios: (i) a host system running Linux without virtualization; (ii) initiating a Docker container with XRDP and GUI; (iii) initiating an LXC container that hosts an operating system (also supporting XRDP and GUI); (iv) initiating a KVM based virtual machine; (v) initiating a Docker container inside an LXC container; (vi) initiating Docker services that are executed on a KVM based virtual machine; and (vii) initiating a Docker container inside another Docker.
To perform the evaluation tests, a native Linux system was used, and the specifications included an Intel Core i7-9750H CPU, 12GB DDR4 RAM and 1TB NVME-SSD. In all of our tests, we ensured that all of the other applications were turned off. In the conducted experiments, Passmark [59] was used to retrieve the CPU and memory scores, and Flexible I/O Tester (FIO) [60] was used for the disk performance tests. We have considered the following metrics in detail:

1.
CPU Score: This is an aggregate mark that provides the overall CPU performance. In Table 3, the results from the performance evaluation and benchmarks are presented. Considering the first results, the benefits of containerization included the low-performance overhead in terms of the I/O-disk cache writing and reading speeds (see Table 3). As a result, the containers can execute applications and services faster than virtualization technologies such as KVM. Moreover, executing a Docker using KVM reduces the total performance (CPU Score of 7933, 3000 lower than the other approaches). Therefore, using KVM to run containers is considered resource-intensive and eventually significantly reduces performance. Running Docker containers in LXC maintains a very good performance and is very close to the performance of the host machine. Finally, on each of the containerization deployments, the disk performance outweighs virtual machines due to the fact that the containers isolate and use a smaller amount of disk space, which is stored as a partition to the base host. Based on the above analysis, we considered the option of using LXC and Dockers running inside LXC as the most appropriate technologies for deploying the trainees' virtual hosts in PocketCTF. The deployment of LXC containers requires less deployment effort and nearly zero configuration. Moreover, LXC containers can run Dockers to install and run applications and services. In this way, the disk space requirements are reduced, resulting in better availability, extensibility, and scalability overall [48].

PocketCTF: A Portable Capture the Flag
The outcomes of our evaluation were important for discovering the benefits and challenges of each virtualization technology and to decide which technology would fit our needs. In this section, we present our approach based on the findings of the previous section.

Software Architecture
The architectural software components of PocketCTF are presented in Figure 1. The management service is the interface used by the educator to manage the cybersecurity exercise scenarios. Educators deploy and initiate LXC instances separately for each trainee. Note that the educator must manually inform (e.g., send an email) each trainee of their assigned IP address of their LXC instance to perform an RDP connection and to gain access to their cyberspace environment to play the CTF scenario.
According to the proposed architecture (Figure 1), the LXC instances (i.e., the Virtual Network 01, 02, etc.) hosts an Ubuntu Linux as the main environment for each trainee, featuring a GUI (gnome-desktop) including important security tools such as Suricata, Wireshark, and Nmap, among others. Each LXC container maintains the virtual Ubuntu system in a different namespace, and the vulnerable services along with the attack hosts are deployed internally to the LXC by using Docker containers. In other words, each LXC container includes several Docker containers that have a unique IP address to the virtual network. The running Dockers inside the LXC either perform attacks that the trainee should detect or include the vulnerable hosts/services. Note that these LXC containers can be converted to an LXC image to easily replicate the infrastructure. As presented in Figure 1, for the management of the various LXC instances, an LXD server is required. The LXD server enables the easy and quick deployment of different LXC containers, each one with its own virtual network. The LXD server is deployed using Proxmox [61], which is an open-source server virtualization platform for KVM virtual machines and LXC containers with a web-based interface. According to the proposed architecture (Figure 1), the LXC instances (i.e., the Virtual Network 01, 02, etc.) hosts an Ubuntu Linux as the main environment for each trainee, featuring a GUI (gnome-desktop) including important security tools such as Suricata, Wireshark, and Nmap, among others. Each LXC container maintains the virtual Ubuntu system in a different namespace, and the vulnerable services along with the attack hosts are deployed internally to the LXC by using Docker containers. In other words, each LXC container includes several Docker containers that have a unique IP address to the virtual network. The running Dockers inside the LXC either perform attacks that the trainee should detect or include the vulnerable hosts/services. Note that these LXC containers can be converted to an LXC image to easily replicate the infrastructure. As presented in Figure  1, for the management of the various LXC instances, an LXD server is required. The LXD server enables the easy and quick deployment of different LXC containers, each one with its own virtual network. The LXD server is deployed using Proxmox [61], which is an open-source server virtualization platform for KVM virtual machines and LXC containers with a web-based interface.
Moreover, the educator can access the frontend interface of PocketCTF to manage the exercises. This interface is based on CTFd [62] (an open-source platform that can hosts CTF challenges), which is installed on a dedicated LXC container. The PocketCTF reserves a separate LXC container that is only for running a SIEM, which is of a significantly heavyweight to be installed in separate LXC containers for each trainee. The downside of this approach is that the same instance of the running SIEM is accessed by every trainee. In PocketCTF, we have opted for the deployment of an open-source SIEM named Wazuh [63]. The proposed approach can be extended by using internal or external virtual machines that could host additional services that might require more resources or that can run Microsoft Windows virtual machines.

Implementation of a Proof-of-Concept Scenario
To implement a proof-of-concept scenario, we have deployed a CTF scenario using PocketCTF. The main learning goal of the scenario was to engage students to learn how to configure a SIEM and to detect attacks such as DoS and port scanning. This scenario requires one LXC container running the Ubuntu OS and three Docker containers. In particular, the first Docker container (Docker 01) represents the victim host, while the two Moreover, the educator can access the frontend interface of PocketCTF to manage the exercises. This interface is based on CTFd [62] (an open-source platform that can hosts CTF challenges), which is installed on a dedicated LXC container. The PocketCTF reserves a separate LXC container that is only for running a SIEM, which is of a significantly heavyweight to be installed in separate LXC containers for each trainee. The downside of this approach is that the same instance of the running SIEM is accessed by every trainee. In PocketCTF, we have opted for the deployment of an open-source SIEM named Wazuh [63]. The proposed approach can be extended by using internal or external virtual machines that could host additional services that might require more resources or that can run Microsoft Windows virtual machines.

Implementation of a Proof-of-Concept Scenario
To implement a proof-of-concept scenario, we have deployed a CTF scenario using PocketCTF. The main learning goal of the scenario was to engage students to learn how to configure a SIEM and to detect attacks such as DoS and port scanning. This scenario requires one LXC container running the Ubuntu OS and three Docker containers. In particular, the first Docker container (Docker 01) represents the victim host, while the two other Docker containers (Docker 02-03) initiate a DoS attack and port scanning (using hping [64] and Nmap [65]). The attacks are automatically repeated every 2 min using the created shell scripts, and the detection relies on the network behavior and on the events that are extracted by Suricata. The events are parsed and sent to the SIEM, where the trainees can access and analyze the events overall. In Figure 2, the proof-of-concept deployment of the scenario is presented. hping [64] and Nmap [65]). The attacks are automatically repeated every 2 min using the created shell scripts, and the detection relies on the network behavior and on the events that are extracted by Suricata. The events are parsed and sent to the SIEM, where the trainees can access and analyze the events overall. In Figure 2, the proof-of-concept deployment of the scenario is presented. Using RDP, the trainees are connected to the IP address of the LXC container that hosts the different Docker services and the required software packages. As mentioned before, the internal virtual network is handled by the LXC container, and therefore, each deployment is isolated. An example of a successful RDP connection to an LXC container that initiates a virtualized operating system is presented in Figure 3.  Using RDP, the trainees are connected to the IP address of the LXC container that hosts the different Docker services and the required software packages. As mentioned before, the internal virtual network is handled by the LXC container, and therefore, each deployment is isolated. An example of a successful RDP connection to an LXC container that initiates a virtualized operating system is presented in Figure 3.
hping [64] and Nmap [65]). The attacks are automatically repeated every 2 min using the created shell scripts, and the detection relies on the network behavior and on the events that are extracted by Suricata. The events are parsed and sent to the SIEM, where the trainees can access and analyze the events overall. In Figure 2, the proof-of-concept deployment of the scenario is presented. Using RDP, the trainees are connected to the IP address of the LXC container that hosts the different Docker services and the required software packages. As mentioned before, the internal virtual network is handled by the LXC container, and therefore, each deployment is isolated. An example of a successful RDP connection to an LXC container that initiates a virtualized operating system is presented in Figure 3.  The trainees must execute the following steps to complete the scenario:

1.
Trainees need to access the LXC dedicated to them by using RDP: The trainees are invited to access the system via RDP. The educator may explain that this will be their main system for the exercises and the total approach.

2.
Access the CTF platform to access the instructions: The trainees are invited to access the web API of the platform where the main features of the platform are explained. Configure IDS and read alerts: The trainees should read and evaluate the alerts from the IDS by checking the log file generated by Suricata (i.e., fast.log). Afterwards, this file will be forwarded to the SIEM. 5.
Forwarding log files to SIEM for parsing: The trainees should configure the SIEM agents to ingest the fast.log (generated by Suricata) and the generic log files (generated by the host OS) into Wazuh. 6.
Create scheduled queries which will trigger the corresponding alerts: The trainees must create scheduled queries that will fetch data to detect the DoS attack and generate an alert.
By following and completing the proof-of-concept scenario, the trainees are able to understand the fundamental concepts of data shippers by deploying the SIEM agents and to technically investigate the logs from the intrusion detection. The trainees are able to follow step-by-step instructions from the CTF platform in order to detect and respond to the attack in real-time.

Discussion
The main benefit of the proposed PocketCTF is the high performance and fewer requirements, due to the kernelless deployment of the LXC containers. Another important benefit is the small disk space required to run the containers in contrast to virtual machines. The above is possible since the LXC images (also called LXC templates) can be used to deploy multiple LXC containers that minimize the required disk space. Educators are able to create customized exercises and to store the LXC images or distribute them online. Therefore, every LXC image can maintain a different cybersecurity scenario, including multiple attack paths, and have the software components for the defensive actions included.
The benefit for the trainees is that they maintain a single point of interaction by entering the appointed LXC that maintains everything that is required. Once converted to an LXC image, the LXC container will be used as the core for deploying the LXC containers. Educators can use existing exercises that are hosted as virtual machine images (e.g., from Vulnhub), as it is possible to convert them to LXC images. It is also important to note also that we tested the possibility of deploying PocketCTF in a Raspberry Pi 4 board. The benefits and challenges for deploying in a Raspberry Pi compared to a typical PC machine are summarized in Table 4. In general, it was feasible to host the LXC containers on a Raspberry Pi 4 but with some restrictions, as discussed in Table 4. Deploying the approach on a Raspberry Pi, for example, restricted the deployment of the Dockers that are only designed for ARM technology. However, it is possible to rebuild existing Docker images for ARM technology using Docker buildx [66]. Using this tool, we successfully deployed more than 10 containers by using an Ubuntu system as the LXD host operating system on the Raspberry Pi.
In Table 5, a summary of the benefits and challenges of PocketCTF are presented with regard to portability, scalability, compatibility, and usability. The numerous benefits of our approach underscore the practical added value of PocketCTF. The compatibility issues and, more specifically, the inability to deploy Windows hosts should be taken into serious consideration. A possible solution is to utilize external networks to regularly initiate Windows hosts by using KVM or other virtualization technology. Such an approach may solve this issue, but it is the opposite direction to having a portable and performance-wise solution.

Conclusions and Future Work
This paper presented PocketCTF, a flexible and portable platform for cybersecurity educators to create and manage virtual labs, both for offensive and defensive security training. PocketCTF can host and manage a large variety of security tools that are ready to deploy, decreasing the total effort needed to deploy the exercises. PocketCTF is focused on portability and easy-deployment and introduces a way to include all of the important software packages for hands-on labs inside a single virtual machine, while vulnerable services are deployed as Docker containers along with the victim and attack hosts. We also discussed the potential benefits of using containerization instead of virtualization technologies to deploy virtual labs. We investigated the various features that containerization and virtualization provide by analyzing the benefits of each approach. The findings indicate that deploying the required services in LXC and Docker containers will significantly decrease the total overhead. The deployment of PocketCTF confirms that the total overhead is decreased by using containers instead of virtual machines and that the total management is easier for creating and deploying cybersecurity hands-on labs. The required services necessary for the intrusion detection, the SIEM, and the CTF platform are automatically deployed or replicated and require less effort and system resources than other approaches.
Future work includes the extensive validation of PocketCTF by conducting stress tests and by including other security scenarios as well. In terms of the evaluation, we intend to further test the approach by deploying an LXD cluster to distribute the required resources and scale the approach on multiple systems. Furthermore, future work includes the conversion of existing challenges to containers and providing specific steps that have to be followed as documentation. Finally, we consider the alignment of the exercises to cybersecurity education frameworks as an important aspect. Towards this direction, we intend to further investigate the existing taxonomies regarding the learning impact by utilizing scenarios that will be executed and hosted accordingly to PocketCTF.
Funding: This work is performed as part of the SPHINX project that has received funding from the European Union's Horizon 2020 research and innovation program under grant agreement No. 826183 on Digital Society, Trust & Cyber Security E-Health, Well-being, and Ageing. The funding body has not participated in the elaboration of this research paper. This work was also supported by the European Commission under the Horizon 2020 Programme (H2020) through the Project CyberSec4Europe (Grant Agreement no. 830929).
Institutional Review Board Statement: Not applicable.