6.2. Knowledge Units for HE and Employers
Table 7 shows the KUs’ importance in descending order, along with their scores for HE (i.e.,
SUM points) and employers (i.e.,
AVG rating). The
SUM points value represents how many points each individual KU has received during the data collection on HE courses (see
Figure 3). All the
SUM points totaled 896, which is the total number of ECTS credits for all the evaluated courses. The
AVG rating is the average rating survey respondents gave to the importance of each KU, on a scale from 1 (“Not important at all”) to 5 (“Very important”).
From
Table 7, we notice that
Human SecurityF KUs are much higher in the employers’ list than is the case for HE. There are four KUs from
Human SecurityF in the employers’ top 12 most important competencies, while in HE, the highest-scoring
Human SecurityF KU is only in the 22nd spot. Clearly,
Human SecurityF is very important for employers, especially
Identity ManagementF and
Personal Data Privacy and SecurityF (first and second spots on the list, respectively), while it is not a significant priority in HE.
The KUs from
Data SecurityA, appear to be very high in the list of both HE and employers; however, closer inspection revealed significant differences (which were already implied in
Table 6, where
Data SecurityA was the most important area in HE, but only 4th for employers). We notice that
CryptographyA, which was, by a large margin, the most important unit in HE, was very low on the list for employers (only #47). The same was true for
CryptanalysisA, which went from 6th place to dead last. Employers appreciate some of the KUs from
Data SecurityA, but, overall, the area is not nearly as important as it is in HE, where all the
Data SecurityA KUs were in the top half of the list.
We can notice a very similar thing happening with
Organizational SecurityG, only this time, it is the employers that hold the associated KUs in much higher regard. While at the very top of the list, the KUs from
Organizational SecurityG were placed similarly for HE and employers (even though they were not the same KUs), they consistently ranked lower on the HE side of the table afterwards. This explains the disparity in
Table 6, where
Organizational SecurityG is placed as the 2nd most important KA for employers, but only 6th for HE. The most interesting KU from
Organizational SecurityG is probably
Business Continuity, Disaster Recovery, and Incident ManagementG, which is the most important KU from this KA for employers (#9); however, it is low on the HE list (only #33). Similarly,
Personnel SecurityG was relatively important for employers (#16), but very close to the bottom for HE (#51).
Employers also highly valued competencies in Personal Data Privacy and SecurityF (#2) and PrivacyH (#14), which are listed much lower in HE (#42 and #41, respectively). We wonder if the evident importance of this knowledge among employers has been caused by the increased regulation (e.g., General Data Protection Regulation (GDPR)). With the exception of PrivacyH, the employers did not seem to have much interest in Societal SecurityH, whereas PrivacyH was the least covered KU from Societal SecurityH in HE.
Table 7 also shows a relatively small difference between the employers’ ratings. The difference between the highest-rated
Identity ManagementF (
AVG rating of 4.82) and the lowest-rated
CryptanalysisA (
AVG rating of 3.63) was less than 30%. The lowest-rated KU was still well above the mean value of 3 (i.e., “Neither important nor unimportant”). A potential explanation is that the importance of all skills is quite uniform across the labor market, but this is not a realistic assumption in our view, and we see the lack of differences as mainly due to participants’ reluctance to label any skill as less important. This view is also supported by the modus (i.e., the most frequently chosen rating), which was, for 83.6% of the KUs, the best possible rating of 5 (i.e., very important). For example, the
Cyber LawH, although not the primary domain of technical staff and relevant to only a relatively small number of cybersecurity profiles, was not rated lower than 3 (i.e., “Neither important, nor unimportant”) by any of the respondents. While this is an interesting observation on its own, it raises the issue of confidence in the resulting differences in the importance of KUs. Luckily, here we are primarily looking at rankings, which should not be affected (since the inflation of individuals should still be consistent), while the relative distance between KUs (i.e., the AVG rating) is very likely not proportional because of the inflated importance of the large majority of KUs. Ultimately, we can still confidently claim that one KU is more important for employers than another, but we cannot be overly sure by how much.
On the other hand, the problem of everything being important for employers cannot be seen in HE, where the KUs that are not considered important or are too niche were hardly covered, or not covered at all (as can be seen in the tail-end of
Table 7). This brings us to what we think is the fundamental reason why it is normal for there to be significant differences between what HE is focused on (i.e., on the education of their students) and what competencies employers find important. Of course, educational institutions (HE or otherwise) should provide their students with knowledge that is relevant and in demand by the industry; however, it is not feasible for these institutions to teach their students everything they might potentially need to know one day, and they must therefore prioritize, while employers value everything, at least on some level. Additionally, by teaching cybersecurity competencies to their students (often alongside computer and/or information technology knowledge), HE institutions must start with the basics and build advanced cybersecurity knowledge on top of that over time. Meanwhile, employers are usually interested in the more advanced and/or applicable knowledge (while refusing to rate any competencies as unimportant). For this reason, we believe that KUs like
CryptographyA,
Network DefenseD,
Network ArchitectureD, etc., which represent foundational knowledge that many other KUs rely on, receive significant attention in HE, while they are considerably less relevant to industry. It also appears that HE, at least on some level, prioritizes more technical skills and maybe places less emphasis on social areas (e.g.,
Human SecurityF).
6.3. Knowledge Units Between Different Employers
In addition to comparing the competencies focused on by HE institutions and employers, we were also interested in whether (and how) the results differed for employers based on the type of staff surveyed (management staff and ICT technical staff) and the employer’s size.
Some differences are evident when comparing managerial and ICT technical staff. The managerial staff puts more weight on skills in Organizational SecurityG, while the ICT technical staff has Data SecurityA at the top of the list. This difference seems to be, at least to some extent, linked to their job tasks—the managerial staff is more concerned and focused on organization and management, while the technical staff is more focused on technology and practical security. In both cases, Human SecurityF KUs were also very high on the list. In addition, managers ranked Societal SecurityH higher than ICT technical staff, who, in turn, ranked selected Connection SecurityD and Software SecurityB KUs higher.
A very similar distribution of competence importance is observed between smaller (40% of included organizations) and larger (60%) organizations. Organizations with 100 or fewer employees were considered small. Responses from smaller companies placed greater emphasis on technical skills, similar to those of ICT technical staff. Thus, skills in
Data SecurityA,
Software SecurityB, and selected units in
Connection SecurityD were valued highly. Even though
Organizational SecurityG was, overall, the second most important area for employers, smaller organizations did not value
Organizational SecurityG nearly as much. To a lesser extent, the same was also true for KUs in
Human SecurityF (which was the most important KA for employers overall). For companies with more than 100 employees, the trend reversed, resembling that of managers. Larger organizations valued the skills in
Human SecurityF and
Organizational SecurityG, while
Data SecurityA,
Software SecurityB, and
Connection SecurityD ranked lower than average. It is worth noting that the similarities shown between staff and organization size are not the result of a poor distribution of respondents (i.e., the possibility that the survey would have included predominately managers from larger organizations and ICT technicians from smaller companies), as the distribution of both types of staff in the survey was the same for both sizes of organizations, as we have already shown in
Table 4.
6.4. Final Composite Importance of Knowledge Units
Table 8 lists the most important KUs, combining the results from HE and employers. Because the importance of KUs in HE and among employers was measured in ways that cannot be aggregated or normalized in an impartial way, we chose to combine the KUs based on their ranks (column # in
Table 7). This means the construction process does not account for whether there are large or small differences between KUs (it just considers their ranks), but it should still provide a broad overview of the most important KUs.
The final list in
Table 8 includes the 20 KUs with the lowest combined rank. To avoid a longer table, we limited ourselves to the top 20, but all the rankings can be calculated from the data in
Table 7.
Most of the KAs are represented in the final list. The exceptions are Component SecurityC and Societal SecurityH. System SecurityE is also represented poorly, with only one KU (i.e., System ControlE). The most significantly represented and high on the list are the KUs from Data SecurityA, which is understandable, because they include base knowledge for many other KAs/KUs. In our opinion, the KUs that are, most surprisingly, missing from the top 20, are CryptographyA, PrivacyH, Social EngineeringF, Systems AdministrationG, and System AccessE.
6.5. Comparison with Related Research
In the related research section, we have noted research that has previously examined similar approaches to assessing the importance and relevance of cybersecurity competencies. Here, we compare two of those most similar to our research. Both are similar in the framework used to classify competencies and sectors they analyzed (i.e., HE and industry).
Dragoni et al. [
15] analyzed cybersecurity-related M.Sc. study programs, by contacting program representatives, and used a very slightly modified CSEC2017 framework (that is still wholly comparable to the “stock” framework used in our research). Meanwhile, in our research, we analyzed course descriptions to identify the knowledge units covered by courses and, consequently, by education programs. This should be a better method because it removes the bias introduced by study program representatives rating their own programs. Dragoni et al.’s research also differentiated between mandatory and non-mandatory courses when discussing coverage of knowledge units, whereas we chose to treat both types of courses equally because, in our opinion, the main thing is the option for a student to obtain the knowledge. Unfortunately, Dragoni et al. did not state which study programs they analyzed, but their supplement sources list the universities from which they received feedback. Based on that data, their and our studies most likely included the same study program from Masaryk University and JAMK University of Applied Science, but the remaining programs analyzed should be unique to each of the studies.
In general, the results of Dragoni et al. [
15] matched very well with the HE results presented in this paper. Dragoni et al. found
Data SecurityA and
Connection securityD to be the two most covered KAs. Our results agreed, but put
Societal SecurityH between the two, whereas Dragoni et al.’s results did not consider
Societal SecurityH important. We also agreed that
System RetirementE and
Component ProcurementC are two of the least-covered KUs in HE. Dragoni et al. also produced a list of the 10 most covered KUs according to their survey (also presented in
Table 9). The KUs are listed here as they appear in their list (from the most covered to the 10th), with the spot they took in our HE research (from
Table 7) in brackets:
CryptographyA (also #1 in our study),
Data Integrity and AuthenticationA (#4),
Secure Communication ProtocolsA (#16),
Access ControlA (#15),
Network DefenseD (#2),
System AccessE (#38),
System ControlE (#3),
Network ArchitectureD (#11),
Data PrivacyA (#21), and
Risk ManagementG (#9). The results were complementary, with significant overlap; however, there were some differences, with the most significant outlier being
System AccessE, which was much less common in the study programs we analyzed.
Budde et al. [
16] used a survey distributed among industry (comparable to our employer survey) and academia (comparable to our HE analysis) to evaluate the cybersecurity skills required to perform six different cybersecurity-related jobs. They collected responses from 60 participants (50 from academia and 10 from industry). The competence collection was done for six specific roles, from which the results were extracted. This is different from our research, where we focused on the most important cybersecurity competencies for organizations rather than collecting data for specific roles, which could make the data less generalizable. Additionally, specifically for the industry/employers’ side of the research, this paper uses a larger sample size, providing additional assurance.
Budde et al. [
16] identified seven KUs they named “transversal”, which were relevant to most job profiles included in their analysis. They were
Network DefenseD (#3 in our
Table 8),
Fundamental PrinciplesB (#19),
Secure Communication ProtocolsA (#4),
Business Continuity, Disaster Recovery, and Incident ManagementG (#17),
Network ArchitectureD (#20),
System ControlE (#2), and
System AccessE (#34).
Budde et al. [
16] also compiled a list of the top 10 most important cybersecurity skills according to the feedback they received. In their list, they combined the responses from academia and industry, similarly to what we have done in
Table 8, where we combined the results from HE and employers. The top 10 skills, according to Budde et al., are presented in
Table 9, together with the results from Dragoni et al. and this research (from
Table 8). Note that Dragoni et al.’s list is based on data from HE only, while Budde et al. and this research combine the results from HE and industry. When looking at
Table 9 overall, there are a lot of KUs from Data Security
A. There are five KUs that are present in all three top 10 lists. They are
Data Integrity and AuthenticationA,
Secure Communication ProtocolsA,
Access ControlA,
Network DefenseD, and
Risk ManagementG. One KU from
System SecurityE, is in each of the three top 10 lists, but it is a different one each time.
Software SecurityB and
Human SecurityF each contain only one KU across all three lists, while
Societal SecurityH and
Component SecurityC have none.
When comparing the results from Budde et al. [
16] and the results from this paper (in
Table 9), the overlap of skills is significant (six out of the ten KUs are in both lists); however, there are some exceptions. They are
Business Continuity, Disaster Recovery, and Incident ManagementG (which was #17 in our overall list of KU importance),
Fundamental PrinciplesB (#19),
Network ArchitectureD (#20), and
Common System ArchitecturesE (#45). Most of them are still within our top 20, although the KU from
System SecurityE, was not considered important in our research at all.
Budde et al. [
16] also compared their results from academia and industry.
Table 10 lists the KAs by importance, based on our and Budde et al.’s research results. The results show that the two studies’ results matched well, but some meaningful differences exist. The biggest difference between HE and Academia is
Societal SecurityH, which are almost on the completely opposite sides of the lists, with it being ranked as the second most important in our research and second least important by Budde et al. The second major difference is the
System SecurityE, which is much less relevant according to this research than theirs.
Data SecurityA has been the most focused KA in academia/HE across all three studies we have compared (Dragoni et al. [
15], Budde et al. [
16], and our research). In the result comparison for employers/industry, the lower part of the lists matches very well; however, there are significant differences in the results for
Human SecurityF, which was the most important in this research but considerably lower (#4) on theirs, and vice versa for
Societal SecurityH. The only consistent data point across all studies (including Dragoni et al.) was
Components Security, which was considered the least important in all of them.
Finally, Budde et al. [
16] also created the top 10 most relevant skills for academia and industry. Their results from academia matched very well with their top 10 overall most required skills (only one KU was missing from the academia list that was on the overall top 10 list, and many of them were in the same order of importance). Considering these results and the difference between the number of respondents from academia and industry, Budde et al.’s top 10 most important skills (listed in
Table 9) appear to be skewed heavily towards academic results. Meanwhile, the industry results included only four KUs in the top 10 overall. They included
Fundamental PrinciplesB, which was the only different KU between the academia’s top 10 and overall top 10. The second most important KU, according to the industry, was
DocumentationB. Here, it is important to remember that both these KUs are a part of
Software SecurityB KA (i.e., they relate to fundamental principles in designing and implementing software and software documentation), which would be required knowledge mainly by developers (which was not a profile that was included in Budde et al.’s research). We suspect that the survey participants might have been misguided by these very broadly named KUs, misunderstood their purpose, and, ultimately, applied them to profiles that did not necessarily need them. This would have skewed the results, and it explains the very high placement of
Software SecurityB among other KAs. While
Software SecurityB is important, the majority of the industries that need cybersecurity are not developing their own solutions; therefore, it does not make sense for
Software SecurityB to be at the top of the list for the general importance of cybersecurity skills. It should be noted that a similar problem caused by the general naming of KUs (e.g.,
DocumentationB) could also have skewed the results of this research. However, we had foreseen this problem and tried to reduce this risk as much as possible by describing each of the KUs in our survey. It is hard to be sure that it was enough to eliminate the problem entirely, but, at the very least, the effect does not appear to have been as significant as it appeared in Budde et al., as the highest-ranked KU from
Software SecurityB was
EthicsB in 21st place (for employers). The results from HE should not be affected, as they were marked by paper authors who understood the KUs’ context.