Systematic Literature Review of Security Pattern Research

Abstract

Security patterns encompass security-related issues in secure software system development and operations that often appear in certain contexts. Since the late 1990s, about 500 security patterns have been proposed. Although the technical components are well investigated, the direction, overall picture, and barriers to implementation are not. Here, a systematic literature review of 240 papers is used to devise a taxonomy for security pattern research. Our taxonomy and the survey results should improve communications among practitioners and researchers, standardize the terminology, and increase the effectiveness of security patterns.

1. Introduction

A pattern is a solution to a problem that arises within a specific context [1]. Security patterns are patterns specific to security problems and solutions. Security patterns describe security-related problems and corresponding solutions as best practices that recur under specific contexts in secure developments and operations [2]. There are concrete security patterns and abstract ones that capture successful secure analysis, designs, and implementations. Security patterns provide guidelines to improve security characteristics such as confidentiality, integrity, and availability since security patterns incorporate the knowledge of security experts [3]. To reveal the overall picture and directions of security pattern research, we employ a systematic literature review of 240 papers [4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,239,240,241,242,243] and devise a taxonomy in this paper.

The number of security patterns has recently grown; however, they are still challenging to apply appropriately. Although there are ongoing studies and practices about various topics such as the discovery, documentation, formalization application of security patterns [2], the current trends and prospects of security patterns research are uncertain due to the diversity in the research results themselves. Most studies have focused on technical aspects and implementation, but few have examined the overall picture and significant technical challenges. To elucidate the current trends and prospects, it is necessary to have a scheme to categorize and analyze research papers on security patterns.

Several surveys such as [244] have been conducted on specific security patterns and related patterns to classify and analyze them. Moreover, there are some existing works such as [3,245] on studying different research techniques and approaches for security patterns. However, the survey and analysis are limited to specific security patterns or a small number of research papers. None of these related works is a rigorous survey targeting many research papers on security patterns in general.

It is clear that only considering limited security characteristics triad (i.e., CIA standing for confidentiality, integrity, and availability) is not enough to accomplish the complexity of secure software systems properly. Thus, it is crucial to have a resource for characterizing available security pattern research concerning various characteristics, not only CIA but also others such as suitable development methodologies and phases, to support practitioners select existing security pattern methods and tools, and to help systems and software security community communicate and conduct further research in security pattern methods and tools.

This paper proposes a taxonomy for characterizing and classifying security pattern research through a systematic literature review (SLR) [246]. Based on the taxonomy, we categorize and analyze 240 papers to clarify state-of-the-art and future directions of security pattern research in terms of 13 facets including topics and security characteristics. Our taxonomy and the survey results should improve communications among practitioners and researchers, standardize the terminology, and increase the effectiveness of security patterns. We summarize our contributions as follows:

• Using an SLR to identify necessary facets, we created a comprehensive taxonomy. Our taxonomy characterizes security pattern research to help practitioners choose existing security pattern methods as well as tools. Besides, our taxonomy serves as a resource for the software security community to support communication and research in security pattern methods and tools.
• We surveyed and classified existing security pattern research and clarified state-of-the-art and future directions of security pattern research based on our taxonomy. These findings should stimulate and improve security pattern research, resulting in the improvement of the effectiveness of security patterns.

The rest of this paper is organized as follows. Section 2 summarizes related work. Section 3 overviews our SLR and taxonomy construction process. Section 4 outlines our taxonomy. Section 5 shows the survey results of the facets in the taxonomy. Section 6 shows the validation and use case of the taxonomy. Section 7 describes limitations of the taxonomy and the SLR. Finally, Section 8 provides the conclusion and future work.

2. Related Work

Several surveys have been conducted on specific security patterns and related patterns to classify and analyze them. Nobukazu et al. [242] conducted a survey of security patterns in general; however, the number of patterns studied is limited since it was an early survey in 2008. Uzunov et al. [30] conducted a comprehensive survey of security solutions including patterns for distributed publish/subscribe systems. Washizaki et al. [244] conducted a survey of IoT patterns including security patterns for IoT systems. Laverdiere et al. [155] conducted another comprehensive survey of security patterns useful at the design phase. These surveys are not intended to reveal characteristics of security patterns in general at all phases in the software system lifecycle. Besides, none of them is a comprehensive survey of security pattern research.

Apart from the surveys of specific or general security patterns, there are some existing works on studying different research techniques and approaches for security patterns. Alvi et al. [5] conducted a study to compare various classification schemes of security patterns; in the study, other research approaches such as application and formalization are out of scope. Rajmohan et al. [245] systematically analyzed around 20 research papers that have been published around patterns and architectures for IoT security and privacy; the analysis is limited to IoT security and privacy patterns only. Ito et al. [3] conducted a systematic mapping study targeting security pattern research; however, the number of research papers surveyed is limited to only 30, and it is a brief summary of pattern research without in-depth analysis.

None of those mentioned above related works is a rigorous survey targeting many research papers on security patterns in general. In our preliminary conference paper, we reported a result of an SLR targeting more than 200 papers [247]. Since the set of papers to be analyzed was limited to those published from 1992 to early August 2016, we expanded the SLR target to include more papers published from August 2016 to 2017 in this paper. In addition, we added an in-depth analysis based on the survey (such as security measurements in detail) and related works. Besides, we revised the taxonomy by removing two subfeatures “User” and “Type of pattern”. We removed the former since it is quite similar to “Phase” under the same feature “Purpose”. We removed the latter since it is redundant to hold in addition to “Security pattern” and “Attack pattern” under the same feature “Pattern”. In addition, we added a new subfeature “Pattern modeling” to the feature “Method” since it was additionally identified as an essential and independent feature in addition to “Methodology” and “Relationship between pattern”.

3. Taxonomy Construction

The development of a taxonomy can be approached in two different ways: top-down and bottom-up [248,249]. In the top-down approach, the taxonomy is built upon existing knowledge structures, allowing established definitions and categorizations to be reused, increasing the probability of achieving an objective classification procedure [248]. Existing works have classified and analyzed security pattern research, but none have provided a comprehensive guide that takes major characteristics into account. Therefore, we adopt a top-down approach to design our taxonomy.

Figure 1 outlines how various characteristics are identified to distinguish existing security pattern studies to realize a comprehensive taxonomy, which classifies security pattern research as feature diagrams. A top-down approach is used by having four steps: determining the scope, conducting an SLR, analyzing the results, and validating the results.

• To determine the scope, we first defined our purpose and goals. The purpose is to support the classification, comparison, reuse, and extension of security pattern research. Our goals are to improve communications about security software stakeholders such as researchers, developers, and users and improve the research achievements’ availability. Thus, we aimed to develop a taxonomy to classify security patterns and standard terminology.
• Next, we conducted an SLR aiming to aggregate existing evidence to achieve the research goal and support the construction of evidence-based guidelines for practitioners and researchers [250]. The SLR used Scopus (https://www.scopus.com/) which is citation and abstract database provided by Elsevier, to search for papers about security pattern research. The search query was the following.
  TITLE-ABS-KEY‘‘security patter’’) AND
  ( LIMIT-TO(SUBJAREA,‘‘COMP’’) OR LIMIT-TO(SUBJAREA,‘‘ENG’’) )
  Scopus was chosen because its effectiveness as a software engineering SLR has been demonstrated [244,251,252,253]. In addition, the results can be easily exported. On 23 October 2018, our query returned 484 papers published between 1992 and 2017. The following inclusion and exclusion criteria were subsequently used to compile research on security patterns:
  Inclusion criteria:
  • Studies published in conference proceedings or journals in the form of papers employing security patterns for systems and software systems engineering.
  Exclusion criteria:
  • Studies that do not employ any security pattern.
  • Studies that introduce or propose security patterns without any further engineering activities such as application of patterns.
  Each paper was initially read by one author to determine if it was within the scope of this study. If it fitted within the scope, the author analyzed it against known features used in [3] and identified additional characteristics. Then another author confirmed the assessment. If these classifications conflicted, all authors discussed to reach a consensus. This procedure returned 240 papers and their initial classification results (The list of 240 papers and their analysis details are available at http://www.washi.cs.waseda.ac.jp/security/.).
• Afterward, the identified characteristics were merged using existing methods such as CWE (Common Weakness Enumeration) [254] and CVSS (Common Vulnerability Scoring System) [255] as well as critical concepts clarified in the Security and Privacy Metamodel [256] to form a feature diagram [257]. A feature diagram is a tree to visualize four types of relationships between a parent feature and its child features (subfeatures): The first is “Mandatory”, which indicates a required subfeature. The second is “Optional”, which denotes a voluntary feature. The third is “Or”, which requires at least one of the subfeatures. The fourth is “Alternative”, which means only one subfeature is selected among all possible subfeatures. Since a feature diagram essentially defines a taxonomy, feature diagrams have been used for defining taxonomies to classify papers and documents in literature reviews [258,259].
• Finally, the taxonomy was validated by classifying existing security pattern research identified in the SLR. Each subfeature was assigned to one of the authors to check initial classification results in terms of the assigned subfeature. If the author identified classification conflicts in terms of the assigned subfeature, all authors discussed to reach a consensus modify the taxonomy and/or classification results.

4. Constructed Taxonomy

Figure 2 shows our taxonomy, which includes five features as facets for characterizing and classifying security pattern research.

The first feature is “Purpose”, which includes topics addressed by security pattern research and phases of the systems and software lifecycle. These are particularly important to help practitioners choose appropriate methods and tools against their needs, such as necessary supports (e.g., application of security patterns) and phases necessary to be secured (e.g., secure design) by utilizing security pattern research. These are also helpful for researchers to identify each topic’s and phase’s maturity and envision necessary future efforts.

The second is “Research Implementation”, which consists of the platform to realize the pattern research results, whether the results are encapsulated or (semi-)automated as a tool and whether experiments or case studies are performed to evaluate the results relevant to the original research purpose. These are important to help practitioners choose easy-to-use or empirically validated methods on targeted platforms. These are also helpful for researchers to identify each research area’s maturity and envision necessary future efforts.

The third is “Quality”, which consists of items related to quality characteristics: threats and vulnerabilities toward a specific security problem; security characteristics in detail such as privacy, integrity, and availability; and whether a security measurement system is incorporated in order to detect changes in security by introducing or applying the research results. These are useful for choosing and carefully using specific methods and tools by understanding their impact on quality characteristics. These are also helpful for researchers to envision necessary future efforts concerning quality aspects, including security and privacy.

The fourth is “Pattern”, which includes the types of patterns employed or addressed in the pattern research. Patterns that address security concerns can be classified into two types: security patterns and attack patterns. The former addresses both recurring security problems and corresponding solutions from the viewpoint of defenders to security risks, while the latter addresses only security problems from the viewpoint of malicious attackers by detailing security risks. These are useful for choosing specific methods and tools against intended patterns, especially when practitioners examine specific patterns for use. These are also helpful for researchers to envision necessary future efforts in terms of each specific pattern.

The fifth is “Method”, which includes the methodology, pattern modeling notations, and pattern relationships. These are useful for considering the adaptability of specific methods and tools to ongoing or intended development contexts, including development methodologies, modeling notations, and patterns considered to be used. These are also helpful for researchers to envision necessary future efforts concerning development contexts and security pattern combinations.

5. Survey Results

The 240 papers identified in the SLR are classified by the 13 facets defined in the taxonomy to clarify state-of-the-art approaches and future research directions. Below, we summarize how the taxonomy helps characterization and classification of papers on security pattern research.

5.1. Purpose

5.1.1. Topic

Figure 3 divides the 240 papers by research topic. Most papers propose or report applications of patterns during systems and software development, certain development methodologies, and pattern classification. Empirical and case study reports are limited, indicating that future research should consider case studies, methodologies, and applied experiments.

Although security patterns have been documented and reported at conferences such as PLoP (https://www.hillside.net/plop/) since the late 1990s, patterns are still manually identified and extracted. Pattern extraction is rarely reported (i.e., 1%) [11,13] in research papers. Mechanisms for identifying and extracting security patterns are highly anticipated, but in reality, research is not being conducted on this topic. Similarly, automatically identifying critical attack and security patterns is desired to determine coding requirements and design, but these topics are not extensively researched as only 8% of papers report pattern specifications and verification. Hence, more research on these topics should be conducted in the future.

5.1.2. Phase of Lifecycle

As shown in

Table 1. Phases targeted by security pattern research.

Phase	Number of Papers
Design	123
Analysis	68
Implementation	37
Any	18
Development	14
Test	7
Operation	6
Maintenance	5
Modeling	5
Run	4
Deployment	3
Evolution	3
Verification	2
Evaluation	1
Integration	1
Disposal	1

, we categorized the papers into 16 phases. Each paper is categorized into zero or more phases. Numerous phases from “Analysis” to “Evolution” can be research targets. Besides, if target phases are not clearly specified in a paper, the paper is categorized as “Any”; each paper should be classified into more concrete phases as possible.

The most commonly investigated phases are “Design” followed by “Analysis” and “Implementation”. Hence, research targets are skewed towards the earlier phases. Few report postimplementation phases including “Evolution” and “Maintenance”, suggesting that security pattern research in later phases may be a frontier field. Cutting-edge topics include multidimensional classification of patterns [16], detection of patterns from the program source code [41], improvement of existing legacy software systems using patterns [10], and security patterns for operational dynamics [181]. Classifying security patterns for the system lifecycle, defining and formalizing patterns that address dynamic behaviors, and utilizing defined patterns in existing software systems are topics that should be further examined.

5.2. Research Implementation

5.2.1. Platform

Among the 240 papers, 25% (61) are platform specific (Figure 4), including Ambient Intelligence Environments, Business Process Management (BPM), and Multiagent Systems (MAS). Most reports use general platforms like the web, cloud, and distributed system.

A few papers address Cyber Physical Systems (CPS) and the Internet of Things (IoT) [227,233], and 75% do not refer to a specific computing platform. Since various IoT security patterns are emerging, active research on platforms involving IoT, cloud, and their applications is desirable.

5.2.2. Tool and Automation

As shown in Figure 5, about 34% (82 papers) mention tools or automation. Many use tools and approaches that involve modeling. A few also include aspect-oriented approaches, formal verification, and code generation. Because the majority of reports create a unique tool, there are many tools for modeling, analysis, design, and implementation. However, few studies propose testing tools (such as model-based testing [179,209]) and operating tools (such a runtime framework [40,171,173,199]).

Tools should span the entire lifecycle because security issues appear in all phases. Hence, future studies should develop tools that directly incorporate security patterns in the testing and operation phases.

5.2.3. Evaluation Method

About half (51.6%, 124 papers) incorporate an evaluation by implementing a case study (19.5%), referencing examples (15%), and conducting experiments (4.1%). Additionally, 12.9% report using an evaluation without specifying the method.

The findings indicate that evaluations of security pattern usage are an immature research area. Even if an evaluation is conducted, it is often limited to a case study or referencing an example. Stricter evaluation methods (e.g., a control experiment) are almost nonexistent. More rigorous evaluation methods are expected in the future to improve the maturity, usefulness, and validity of security pattern research.

5.3. Quality

5.3.1. Vulnerability and Threat

Vulnerabilities or threats are mentioned in 29.6% (71) of the papers. Only 1.2% (3 [50,189,219]) (Although more papers refer to STRIDE, most do not incorporate STRIDE into their proposed techniques or achievements. For example, in [45], STRIDE is not handled in the proposed security testing technique; STRIDE is mentioned just in its case study in terms of threat identification without detailed explanations [45]. By excluding such papers, we finally identified that three papers [50,189,219] incorporate STRIDE into their research techniques or achievements in terms of threat identification modeling and classification.) refer to STRIDE [260], which is advocated by Microsoft, while another 2.5% (6) refer to other publicly available information in terms of vulnerabilities and threats. Among these six papers, one [10] references CVSS [255], which summarizes risk information. Four papers reference more tangible vulnerability information such as CWE [254] and Common Vulnerability and Exposures (CVE) [261]. Furthermore, one paper [214] refers to Common Attack Pattern Enumeration and Classification (CAPEC) [262], which categorizes known attacks employed by adversaries.

Security patterns should clearly explain how to deal with security measures that involve addressing system vulnerabilities and threats. Thus, the fact that more than 70% of the papers do not mention vulnerabilities or threats is troublesome. Future research should collect both the theoretical and actual relationships on vulnerabilities and threats to achieve practical uses of security patterns. Currently, few papers refer to publicly available information on known vulnerabilities and threats. Consequently, future research should investigate how to utilize such publicly available information and increase the awareness of security patterns.

5.3.2. Security Characteristic

Over half (58.8%, 141) mention security characteristics. Of the 141 papers, 91.5% (129) refer to CIA characteristics. In these papers, there are 109, 84, and 71 references to confidentiality, integrity, and availability, respectively (Figure 6).

Another 37 papers reference non-CIA security characteristics such as accountability, authenticity, authentication, authorization, and nonrepudiation. Twenty-five of these mention both CIA and non-CIA characteristics, while 12 mention non-CIA characteristics only.

We confirmed that many studies examine security characteristics, especially those based on CIA. Confidentiality, which allows only individuals with granted permission to access information, is essential and the most mentioned characteristic. One example involving privacy and confidentiality is Role-Based Access Control (RBAC).

5.3.3. Security Measurement

Only a few papers (10.8%, 26) adopted security measurements to evaluate patterns. Two used STRIDE [260]. Ref. [57] evaluated the handling of potential threats via a graph and indicated STRIDE’s attack categories against secure and nonsecure systems. Ref. [141] evaluated the system against security attacks by using STRIDE. Their evaluation based on fuzzy logic defined five levels for the five main events, where the levels correspond to a STRIDE’s category.

The following list summarizes other measurements in the literature. The numbers of levels and categories are different, and each level and category are defined differently. The majority of them employ an approach to evaluate three to five discrete levels for the likelihood of exposing vulnerabilities and their effects on the system associated with security patterns.

• In [5], security patterns found in 23 papers are grouped into 14 categories. Then the categories are evaluated using nine levels of quality standard classifications.
• In [26], forces and Solution are used to evaluate attribute, risk reduction frequency, risk reduction consequence, annual number of attacks, cost per attack, and cost solution. Furthermore, XSS (Cross Site Scripting) is evaluated as a case study.
• In [29], seven levels of security criteria are used to compare and evaluate nine security patterns. In addition, performance gain and loss is compared. The implementation cost and degree of security are also evaluated in three levels.
• In [37], the following three categories are used for evaluating security pattern description elements (problem and forces, structure description, structure image, behavior description, behavior image, consequences, and example): not provided, minimal, and satisfactory.
• In [39], measures against possible threats are evaluated using a graph.
• In [59], resource access restrictions granted to different roles are evaluated in terms of four operations: C (create), R (retrieve), U (update), and D (delete).
• Ref. [76] supports an aspect-oriented approach and proposes an evaluation using Object Constraint Language (OCL) for Account Lockout with Selective Logging (ALSEL) and IMAP system.
• In [116], nine levels of quality are used to evaluate nine concerns such as threats and attacks to be avoided, an attack pattern to be applied, threats to be passed, and security requirements.
• In [125], security patterns of eight categories such as accountability, confidentiality, and integrity are evaluated.
• In [152], security patterns of a distributed system are categorized and five quality indicators are evaluated.
• In [155], using the 6$\sigma$ approach, 12 security patterns are evaluated by 6 categories of undesirable properties.
• In [200], using its own unique evaluation formula, the applicability of patterns is calculated as rate.
• In [202], three indices (completeness, isolation, and verifiability) are used as the engineering principles of security kernel.
• Ref. [203] is related to security patterns of a grid system. Password and digital signature are expressed as graphic extension of Backus normal form (a.k.a. Backus–Naur form) in the authentication pattern.
• In [204], using an example of an ATM terminal, security objects, and patterns are described and evaluated in eight matrices.
• Ref. [205] categorizes patterns into three layers and evaluates them.

Because each research paper used its own evaluation categories, assessing the evaluation results’ applicability is challenging. In the future, a standard index such as STRIDE should be used to evaluate results to have comparable security pattern research.

5.4. Security Related Patterns

5.4.1. Security Pattern

Most papers (77.9%, 187) mention a specific security pattern by name. On average, each paper mentions 4.9 patterns. Although there are 1179 references to a pattern name, only 558 are unique patterns. Of these, 31.5% (176 patterns) are mentioned in at least two papers. By the definition of the word “pattern”, a software pattern should be used by many practitioners. However, this study reveals that the majority of patterns ( 70%) are not actually shared. As shown in

Table 2. Major security patterns mentioned in at least ten papers (adopted from [247] with updates of numbers and descriptions).

Security Pattern	Number of Appearances
Role-Based Access Control (RBAC)	49
Authorization	34
Authentication	23
Access control	21
Authenticator	21
Secure logger	19
Check point	17
Reference monitor	15
Secure pipe	14
Single access point	13
Authentication enforcer	11
Attribute-Based Access Control (ABAC)	10
Encrypted Storage	10
Firewall	10
Replicated system	10

, only 16 patterns are mentioned in 10+ papers. These patterns are related to authentication, authorization, and access control.

Ironically, over 22% of the papers on security patterns do not mention a certain pattern by name. Without a pattern name, it is difficult to explain a new idea or method. Our results reveal about one-third of the patterns are common; using easy-to-use directed graph representations such as UML class diagrams would contribute to their high reusability. Although many research papers express patterns without specific names, this will become more challenging in the future as research expands to include concepts that are often hard to describe by a structural description such as availability.

5.4.2. Attack Pattern

Attack patterns are much less prevalent than security patterns. Only 17.0% (41) papers mention attack patterns. Many patterns are mentioned in only one paper.

Table 3. Appearances of attack patterns in at least two papers (adopted from [247] with updates of numbers and descriptions).

Attack Pattern	Number of Appearances
Spoofing	6
Denial-of-service (DoS)	5
Misuse	5
Information disclosure	4
Injection	4
Tampering	4
Malicious Virtual Machine Migration	3
Elevation of privilege	2
Integrity	2
Message secrecy violation	2
Repudiation	2
Resource usage monitoring	2
Session state poisoning	2
Theft of services	2

summarizes patterns mentioned in multiple papers.

Moreover, the abstraction varies widely. Some refer to abstract attack patterns in STRIDE, which is a categorization of attack patterns. Others discuss CIA security characteristics. One is specific to illegal money transfers in a certain application. Although attack patterns and security characteristics are common, specific examples are rare.

5.5. Method

5.5.1. Methodology

Ninety-seven papers (40.4%) describe a development methodology (Figure 7). Some discuss a methodology related to a model-driven development approach (14.2%) or an aspect-oriented development approach (2.1%). Although many development methodologies are reported, few examine security-focused methodologies. As IoT becomes ubiquitous, studies on the methodology should intentionally focus on security by design.

5.5.2. Pattern Modeling Notation

The types of modeling notations used in security-pattern research are examined. About two-thirds of the papers represent the notations of security patterns, which can be categorized into six groups.

Table 4. Pattern modeling notation.

Group	Example	Number of Papers
UML	Class/Activity diagram	104 (43.3%)
Goal oriented	i* (i-star), KAOS, threat tree	13 (5.4%)
Formal	Z notation, formula	12 (5.0%)
Natural language	Text, structured document	12 (5.0%)
Original	Original notations	6 (2.5%)
Others	Process model, XML, OWL	65 (27.1%)
Not specified		77 (32.1%)

shows the groupings, where multiple groups indicate papers using multiple notations. The “UML” group includes UML diagrams and UML based notations. The “Goal-oriented”, “Formal”, and “Natural language” groups include models used in goal-oriented methods, formal notations, and natural language notations, respectively.

Security patterns are mostly UML, which is reasonable since UML is generally accepted for modeling software and systems. In papers that address specific development methods or tools, formal, goal-oriented, and natural language notations are used in 13, 12, and 12 papers, respectively. Moreover, about one-third of papers do not describe the notations of security patterns. In the future, the notation should be described to clarify security patterns.

5.5.3. Relationship between Patterns

Because security patterns are often used and applied as combinations, their relationships must be clarified. Relationships between patterns can be classified into two types: between security patterns (relationship A) to enhance described security methods by combination and between an attack pattern and a corresponding security pattern to reduce security risks (relationship B).

Of the 240 papers, 98 papers (40.8%) focus on relationship A. Only 5.8% (14) mention relationship B. These results demonstrate that security pattern combinations are often not considered. In the future, more research needs to be conducted with an emphasis on relationship B. Such research is expected to reveal how security patterns reduce risks imposed by attack patterns in specific development processes.

6. Validation and Use Case of Taxonomy

There are multiple methods to validate a taxonomy. Examples include demonstrating the orthogonality of its classification features, benchmarking against existing classification schemes, or confirming its utility to classify existing knowledge [263]. Herein orthogonality means that a paper can be classified as only one category of possible combinations of concrete features. We validated our taxonomy by classifying the research papers identified in the SLR. Because fitting of each characteristic gave only one classification category shown in the survey results in Section 5, the classification features are orthogonal.

Besides, we validated our taxonomy by classifying the four latest popular papers (On 30 December 2020, we applied the same search query for papers published in or after 2018 at Scopus. Our query returned 129 papers. By applying the inclusion and exclusion criteria to them, we confirmed that 66 papers fit within our study’s scope. Among 66, we selected the top four mostly-cited papers that meet the inclusion criteria. The list of these latest 129 papers is available at http://www.washi.cs.waseda.ac.jp/security/.) that are not included in the SLR.

Table 5. Classification of latest papers: Purpose and Research implementation.

Paper	Topic	Phase	Platform	Tool	Evaluation
[264]	Application	Design	Cloud	Model-driven development	Example
[265]	Survey	Design	Cloud	–	–
[266]	Application	Design	IoT	–	–
[267]	Application	Analysis	–	Goal-oriented modeling	Case study

,

Table 6. Classification of latest papers: Quality.

Paper	Vulnerability	Characteristic	Measurement
[264]	–	Integrity, Availability	Security level
[265]	Vulnerabilities, threats	Confidentiality, Integrity, Availability	–
[266]	–	Confidentiality	–
[267]	STRIDE, threats	Confidentiality, Integrity, Availability	–

,

Table 7. Classification of latest papers: Pattern.

Paper	Security Pattern	Attack Pattern
[264]	Protection Reverse Proxy	–
[265]	Authorization, Authentication, Logging/Auditing	–
[266]	Reference Monitor, Role Based Access Control (RBAC), Remote Authenticator/Authorizer, Matrix Authentication, File Authentication	–
[267]	Alternative service, Client Checking, Separation of Duty, Certification authority, Supervision Relation, Access Control, Auditing, Input Guard, Server sandbox, Firewall, Replicated System, Load Balancer, Limited View, Full View with Errors, Secure Access Layer, Secure Pipe, Storage Encryption, Equipment siting and protection, Supporting Utility, Physical Entry Control, Cabling security	–

and

Table 8. Classification of latest papers: Method.

Paper	Methodology	Modeling	Relationship
[264]	Model-driven	UML, Operational Flow Language	–
[265]	–	UML	–
[266]	–	–	–
[267]	Goal-oriented requirements engineering	Goal-oriented model	–

show the classification results of these papers using our taxonomy.

We summarized the results aligned with classification features as follows.

• Purpose: Three-quarters of the papers report security pattern applications targeting the earlier phases, including analysis and design.
• Research Implementation: Half of the papers use cloud as their application platform. Half mention tools. Evaluations of security patterns are limited to an example or a case study.
• Quality: Half of the papers mention vulnerabilities or threats. Only one paper refers to publicly available information regarding vulnerabilities and threats (i.e., STRIDE). All studies examine security characteristics based on the CIA. Only one paper adopts security measurements.
• Pattern: All papers focus on security patterns only.
• Method: Half of the papers describe some development methodologies. Three-quarters represent the pattern modeling notations, including UML as the most major one. None of the papers explicitly handle security pattern combinations.

Since these trends are quite similar to those of our SLR for 240 papers published in 1992–2017, we believe our taxonomy and SLR results are still applicable to the latest situation and useful. Moreover, we confirmed that we successfully classified the latest popular papers according to the characteristics defined in our taxonomy and show how it can help classify security pattern research papers.

Based on the classification capability, our taxonomy should guide practitioners and researchers in the two use cases (UCs).

• UC1 is to help practitioners select appropriate security pattern methods and tools. When practitioners want to reuse and eventually extend existing methods and tools, these must be compared prior to selecting the most appropriate one for the scenario. Selection should be based on how the methods and tools meet the intended objectives. The taxonomy helps compare criteria to assess methods and tools according to their characteristics.
• UC2 is to communicate research methods and tools to researchers. By incorporating the characteristics of security pattern research into a single structure, the taxonomy can serve as a framework to guide future communications and research on security pattern methods and the corresponding tools. For example, the taxonomy can serve as the basis to build an open repository of information of existing methods and tools. Moreover, the taxonomy should stimulate and improve security pattern research, resulting in improvement of the effectiveness of security patterns.

7. Limitation

Since papers (such as [6,11,45]) dealing with attack patterns often mention them together with related security patterns, we used the term “security pattern” only for the search query. Nevertheless, we may miss research papers that deal with attack papers only without mentioning security patterns. To address this issue, we plan to extend the search query to include the term “attack pattern”.

Research papers are often published in conference proceedings first and refereed journals second, and later (and rarely) in books. To narrow down the survey scope to a specific range of targets, we limit the target publication to a paper in a journal or conference proceeding. Nevertheless, we may miss some books or book chapters describing the latest security pattern research achievements that are not published in conference proceedings or journals. We plan to extend the survey’s target scope to include books and book chapters to address this issue.

We chose Scopus as the search engine since it is effectively used in SLRs of software engineering, and the search results can be exported. The database covers many major publishers, including IEEE, ACM, Springer Nature, Wiley Blackwell, Taylor and Francis, and Elsevier. Furthermore, the database provides a mechanism to perform keyword searches. Although many other SLRs have adopted it, relevant papers may have been missed. To mitigate this issue, we plan to use other databases, extend our SLR, and elicit a public review of the results.

Our SLR’s targets are papers published in 1992–2017. Furthermore, we confirmed that the results are still applicable to the latest situation by examining the top four mostly-cited papers out of 66 latest papers published in 2018–2021. Since these four papers are the most cited ones, we believe that these can represent the latest 66 papers’ trend to some extent. Nevertheless, we still need to continue validating the trends by enhancing our SLR to include the latest publications since other less-cited 62 papers may indicate different directions.

8. Conclusions and Future Work

It is crucial to have a resource for characterizing available security pattern research concerning various characteristics, not only CIA but also others such as suitable development methodologies and phases, to help practitioners select appropriate security pattern methods and tools and to help systems and software security community to communicate and research in methods and tools.

To respond to the necessity, we devised a new comprehensive taxonomy for security pattern research via an SLR. Herein 13 facets are used to define the taxonomy. To clarify the state-of-the-art and future directions of security pattern research from various facets, including topics and security characteristics, this taxonomy analyzed the contents of 240 security pattern research papers identified through an SLR, demonstrating its usefulness. This taxonomy should also support communications among researchers, practitioners, and stakeholders. Hence, it should improve not only the quality of security pattern research but also the effectiveness of security patterns.

The analysis results are summarized along with five features as follows.

• Purpose: Most papers report applications of security patterns, development methodologies, and pattern classification. Research targets are skewed towards the earlier phases, including analysis and design.
• Research Implementation: Most papers use general platforms like the web, cloud, and distributed systems. Only around one third mention tools or automation. Evaluations of security pattern usage are an immature research area since it is often limited to a case study or referencing an example even if an evaluation is conducted.
• Quality: Vulnerabilities or threats are mentioned in only less than one-third of the papers. Many studies examine security characteristics, especially those based on the CIA. Only a few papers adopted security measurements to evaluate patterns.
• Pattern: There are attack patterns and security patterns, but most focus on security patterns and not attack patterns. Most papers mention a specific security pattern by name. There are more than 230 unique security patterns mentioned.
• Method: Around two-fifths describe some development methodologies, in which the model-driven approach is the most major one. About two-thirds represent the pattern modeling notations, including UML as the most major one. Security pattern combinations are often not considered.

Future efforts include experimentally verifying our taxonomy using the two use cases (UC1 and UC2) in Section 6. We will implement a collaborative Wiki so that the community can refine and modify the taxonomy online. Besides, we intend to enhance our SLR to include the latest publications that have been published in or after 2018 to confirm the identified research trends and gaps in this paper still exist. In addition, we will extend our SLR using additional databases and additional categories. Our findings will be shared with the public so that our taxonomy can be validated and revised by the community, and standard terminology can be defined.