1. Introduction
In the era of increasing digitalization, information security is becoming an increasingly important issue to be tackled, and every company, both large and small, must be able to protect their information systems from external attacks that can have negative impacts in terms of financial losses and reputational damage. Non-bank financial companies are one type that have a high risk of information security threats, considering that they obtain funding from other financial institutions to support economic growth; thus, they must have an effective strategy to protect their assets and infrastructure. According to data from the Indonesian Cyber Security Landscape in 2023 in
Figure 1, attacks on organizational information systems amounted to 403,990,813 anomalies, with the highest traffic occurring in August with a total of 78,464,385 anomalies and the lowest occurring in November with a total of 19,296,439 anomalies. Unfortunately, these kinds of activities absolutely have risks and consequences that negatively impact device and network performance, theft of sensitive and secret data, reputation damage, and trust toward the organization [
1].
Furthermore, from
Figure 1, it was revealed that the Cyber Contact Center Team at BSSN (
Badan Siber dan Sandi Negara) sent around 1762 notifications, and the results of the Classification of Top 5 Incident Indications are presented in detail in
Figure 2, namely, Traffic Anomaly with as many as 858, Data Breach with 268, Web Defacement with 172, Sensitive Data Exposure with 113, and Malicious Software with 104 [
1].
One solution that is widely used by companies to overcome information security threats to cyberattacks is to build a Security Operation Center (SOC), a typical kind of security control center that focuses on monitoring, detection, analysis, and rapid response to cybersecurity threats. An SOC also aims to protect organizations from any cyberattacks that can harm assets, reputation, and business [
2]. Thus, a well-designed SOC can help organizations maintain their security posture, mitigate risks, and respond quickly to security incidents [
3]. It is also responsible for cybersecurity incident management, cyberattack detection, continuous and protective security monitoring, log and event management, coordination, and investigation [
4]. Thus, having an SOC provides huge benefits for cybersecurity in companies, especially in providing centralized, continuous security monitoring and response [
5]. In terms of people, the formation of an SOC can resolve the challenge of the low availability of cybersecurity experts through recruiting, training, and preserving them for better growth so that their skills and experience can match those of the parent company [
5]. In terms of process, it also can form an efficient and effective framework, both in terms of manpower and finances [
5]. Technologically, it can balance the availability of existing technological infrastructure within a company so that it can be adjusted according to the company’s cybersecurity needs [
5]. Although SOCs have become an important part of companies’ information security strategies, there has not been much research evaluating the effectiveness of SOCs in addressing information security threats as there is no unified theoretical model or even agreed metric for whether it correlates with business resilience as a tool of centric evaluation or even improves security outcomes in outcome-centric studies. Previous research on SOCs has been conducted in several countries, but little work has been performed in the Indonesia context, especially in non-bank financial companies. Information technology in Indonesia is undergoing rapid development, so the risk of threats is increasing, and research therefore needs to be conducted to evaluate the effectiveness of SOCs in strategic management in overcoming threats holistically.
The company used as the object of the research is PT NBFC (Non-Bank Financial Company), which functions as an agency whose activities are related to collecting funds from the public by issuing securities then distributing them to finance investment companies that need loans [
6]. Interestingly, it has implemented an SOC for its system security but still has problems, namely, not having a process for analyzing the impact on Financial and Regulatory Requirements as well as calculating the costs and efforts required for recovery on detected events/incidents. Unfortunately, there are no Key Performance Indicators (KPIs) or Key Risk Indicators (KRIs) either, which are set by management to monitor the effectiveness of physical access controls and compliance with applicable standards. Also, it does not yet have an official program related to detecting and preventing insider threats within the organization, a watchlist process, or additional monitoring of employees who resign, especially those who have the potential to become insider threats. Essentially, in this case, there is a security and compliance review process carried out on third parties, both onsite and offsite, but there is no full integration between File Integrity Monitoring (FIM) and SIEM (Security Information and Event Management) solutions to detect unauthorized changes to audit logs.
The company currently uses a solution system to analyze the incident trends but does not yet have analytical capabilities to process those incident data as the impact analysis of incident reports does not consider financial and regulatory aspects. It also does not analyze incident data regularly and continually to identify trends, threat patterns, attack types, or key areas that need special consideration and attention. Lastly, in their simple scanning process, there is no Business Continuity Plan (BCP) testing calendar that includes fire, earthquake, and other disaster simulations for each critical process and system listed in the Business Impact Analysis (BIA) linking SOC inputs to processes and outcomes with respect to the contingency plan and dynamic capability of the organization. According to these problems, the real use case of cybersecurity that occurs at PT NBFC can be seen in
Figure 3, which explains that the attacks or insecurity in the system occur when users use work email for personal matters, and a breach occurs when they log into a pawnshop account on a device that has no endpoint security and is at risk of malware. Then, the user may open targeted phishing email links, installing apk files, which are not in the Play Store, and as a result their data may be exposed externally on the Deep Web or even the Dark Web. The next problem is that some users indicated that they share passwords with others, do not change their passwords regularly, use them repeatedly on public Wi-Fi, and fail to diversify their passwords, which increases susceptibility of MTM attacks. On certain occasions, some also leave their PCs or laptops unlocked, which may result in the worst-case scenario of illegal access to office and personal data.
1.1. Research Gap
Although the increasing role of SOCs in cyber risk management has great significance, the literature on SOC assessment has largely been focused on the implementation of different individual standards or frameworks. Popular standards like ISO/IEC 27005 [
7] and NIST SP 800-30 [
8] are primarily applied for information security risk management, and there has been very little research published that has combined these two frameworks into a single and coherent process for SOC assessment in detail. Most of the literature works have focused on either risk identification and analysis or control assessment, and there has been little effort to seamlessly tie the result of risk assessment with that of capability or maturity assessment of the SOC. Moreover, there has been little empirical research published related to the role of SOCs in risk management for non-bank financial institutions, especially for developing nations like Indonesia. Most of the studies have been focused on banking institutions or overall information security management systems (ISMSs), and there has been little understanding related to the role of SOCs, how they manage risk, and where they need to improve their processes in specific sectors. In addition, most previous research on SOC-related issues has never integrated risk assessments and maturity evaluations. In current practices, most research on maturity evaluation focuses on the definition of organizational capacity based on maturity models. Less attention has been paid to the study of concrete risk evaluation results based on the predefined ISO/IEC 27005 and NIST SP 800-30 frameworks and even less to the results of such frameworks in the improvement of maturity-related areas in an SOC. Therefore, the current research aims to provide practical applications of the integration of the ISO/IEC 27005 and NIST SP 800-30 frameworks in an operational SOC of a non-bank financial organization in Indonesia, together with the usage of maturity evaluation, focusing on the improvement of SOC-related practices.
1.2. Significance of Study
This research holds importance from both theoretical and practical perspectives as it can contribute positively to the information security and SOC literature by containing an analytical insight with a case study appraisal of an ISO/IEC 27005 and NIST SP 800-30 framework combination. It could be understood in the context that most research conducted on SOC risk management discusses its findings from a conceptual standpoint or in some cases adhering solely to one framework. This research will prove that more than one established framework or combinations thereof can be applied in an organizational SOC context, as such an approach will contain all elements of empirical appraisal. Additionally, it will provide an in-depth study of an underexplored area in social sciences, that of non-bank financial institutions in Indonesia, by providing concrete and empirical evidence on the way SOC risk management can performed in an organizational context that differs from banking institutions. From a practical standpoint, it also provides technical elements of appraisal that will allow information security experts, as well as organizational managers, to focus on more informed methods of risk management. Moreover, within this research, the technical evidence can act as a reference point in that it will contain all elements of the empirical appraisal to prevent misinterpretation and misrepresentation, enabling informed decision making that aligns with best practices and standards.
3. Research Methodology
3.1. Research Systematics
The research stages describe the methods used in the research. The research stages that took place in this study can be seen in
Figure 6.
As shown in
Figure 6 above, there are several research stages, first related to problem identification, which involves not only listing the symptom but also distinguishing its root causes and defining the gap between the expected and actual condition. Often, non-banking industry identifies problems based on performance and market signals after deviation occurs while banking through risk and compliance perspective before failure occurs. Next, a literature study was conducted, namely, the process of searching for related sources from the internet as references and benchmarks. This was useful for the next phase, related to the analysis of current conditions of the market and relevant organizations from various industries to elucidate the outlook of PT NBFC. Its purpose is not merely descriptive but also diagnostic, with the goal of investigating existing practices. The result is critical for the formation of a new framework by preventing purely normative or speculative models to avoid overgeneralization and conceptual overload. Supposedly, the framework is an enabler that shift complexity into action and clarity, and it is not a constraint that cannot be explained and accidental. Lastly, the conclusions and suggestions, as the logical syntheses of theory and evidence, were derived from alignment between the observed problems and proposed framework elements through the integration of structured consequences data into risk governance process to improve the quality of decision making.
3.2. Case Study Description
This research used a single-case study methodology to investigate the application of an integrated framework of information security risk assessment in a real-world scenario. It was carried out in a non-bank financial company in Indonesia referred to as PT NBFC to avoid revealing any compromises and to preserve its confidentiality. As a non-bank financial institution, it handles a substantial volume of sensitive, confidential, and even secret information as important assets that are essential to its operational continuity. The organization also has an SOC, which oversees and manages security incidents in accordance with information security risk management practices. Unfortunately, it is also attacked by increasing numbers of cyberattacks from various domains, which was a consideration that led to the selection of this company as the research object in this study. Thus, this study specifically examined and analyzed the management of SOC-related risks and associated security practices using an integrated framework of ISO/IEC 27005 and NIST SP 800-30 against the periodical assessment within the organization, particularly within SOC domains such as risk identification, risk analysis, risk evaluation, and their association of risk treatment and security control. The findings are presented in an aggregate and anonymized form to ensure confidentiality while offering insight that may serve as practical reference for others in Indonesia seeking to implement the integrated risk assessment.
3.3. Data Source
This research makes use of varied sources of data to aid in the analysis of the case study and adds validity in the form of data triangulation. The main sources of data in this study pertain to internal organizational data on information security as well as operations of the SOC in PT NBFC. Data sources here relate to information security policies, SOP, risk registers, internal audit reports, as well as data records on the SOC that are relevant in risk identification, risk analysis, as well as the implementation of risk control. This data, apart from being based on documentary evidence, also utilizes expert opinion to aid in the assessment. Input in this regard was gathered from people involved in ISM as well as staff in the SOC of PT NBFC, such as SOC analysts and information security experts. Other sources of data, which were secondary in nature, were also gathered to add depth in the form of varied perspectives related to the relevant literature, as well as relevant and selected international standards, which in this case are ISO/IEC 27005 and NIST SP 800-30 as well as other kinds of guidelines set out by regulatory bodies in the context of Indonesia that might influence the implementation significantly.
The KAMI index was utilized as another complementary method for evaluating the organizational maturity levels in relation to the operations of the SOC. For this research, five domains were included in the evaluation, namely, governance, risk management, information security framework, asset management, and technology, as well as security controls. These domains were evaluated using a questionnaire that was completed by key organizational members directly engaged in the management of SOC processes and information security, namely, the SOC manager, information security officer, and the senior IT security personnel. These organizational members were targeted because of their involvement in security monitoring, response, and management processes within the organization. Evaluation of the score was based on the conventional KAMI index scoring system, whereby the individual scores were weighted at the domain levels; the scores were then aggregated to provide a comprehensive organizational score then validated by a comprehensive review toward organizational documents.
3.4. Documents Used
The documents used in this research directly derive from PT NBFC as a primary source and journals to support the analysis, which is related to the assessment of information security systems presented by using the ontology approach as a formal description of a concept explicitly in an insight from each concept along with its limitations, while the relationship of concepts includes a hierarchy of classes [
34].
The documents from PT NBFC consist of the results of the assessment at PT NBFC, SOC Audit Results, NBFC SOC Guidelines, and Technical Studies—Managed Service SOC, and the domain ontology for these documents can be seen in
Figure 7.
- 2.
Journal Publication Documents from Internet
Journal documents used are related to this research as supporting material for parameter assessment analysis of the SOC information system, the domain ontology of which can be seen in
Figure 8.
3.5. SOC Information System Assessment Parameters
The parameters for conducting security assessments in organizations encompass several key principles that ensure the protection and reliability of information systems [
35,
36]. One of the fundamental parameters is confidentiality, which ensures that sensitive data is accessible only to authorized individuals, also preventing unauthorized access and data breaches. Furthermore, integrity is another critical factor focusing on maintaining the accuracy, reliability, and consistency of information by protecting it from unauthorized modifications, corruption, or cyber threats such as viruses and hackers. Availability is also essential, ensuring that information systems, websites, or platforms remain accessible to authorized users whenever needed, minimizing downtime and disruptions that could impact organizational operations. In addition to these principles, accountability plays a crucial role in security assessments by ensuring that all actions taken within the system are traceable to responsible individuals, thereby promoting transparency and compliance with security policies. Lastly, authentication strengthens security by verifying the identity of users, ensuring that only legitimate individuals can access protected resources, thereby reducing the risk of unauthorized access and potential cyber threats [
37,
38]. By applying these parameters, organizations can effectively assess and enhance their security posture, ensuring a robust and resilient cybersecurity framework that safeguards sensitive data and operational integrity [
39,
40].
4. Result and Discussion
Model development conducted with modern SOCs involves consideration of an organization’s unique cybersecurity risks and requirements, current trends in cybersecurity, and the latest technologies available [
3]. Modern SOC models that can be used for development are the Next-Generation Cognitive Computing SOC (NGC2SOC) model, which is an SOC model in a much better position to strengthen cybersecurity strategies. The goal of the NGC2SOC model consists of advanced intelligence-driven tactics to conduct real-time investigations of known and unknown vulnerabilities, direct access, evidence visualization, and additional advanced tools or practices that reduce potential risk to critical assets combined with fully automated cybersecurity problem recovery [
36]. In this research, framework evaluation, framework design, and implementation were carried out altogether and accordingly.
4.1. Object of Study
As stated, the object of analysis is the SOC of a non-bank financial company in Indonesia, which deals with security events and incidents as well as the management of responses to these incidents. In this case, the research aims to give a practical basis to the results of risk and maturity analysis. From the results of data analysis conducted, it was found that implementing a modern SOC must involve a holistic approach that considers SOC components consisting of humans, processes, and technology. This will protect the organization from the threat of cyberattacks and can reduce the risk of data breaches that are detrimental to the organization. The results of this study are recommendations for standards/frameworks for SOC information system security assessments at PT NBFC which can be applied to other organizations as well. The proposed standard is the result of an analysis of two frameworks, namely, ISO 27005:2018 and NIST SP 800-30, as the recommendation for conducting SOC guidance assessments. Putra and Soewito (2023) [
41] previously conducted risk assessments using ISO/IEC 27005:2018 as a risk management framework and NIST SP 800-30 Rev.1 as risk assessment guidelines. Thus, in this study, SOC effectiveness was reconceptualized and extended through a mediating mechanism by incorporating structured incident consequence information. Essentially, ISO/IEC 27005:2018 supports the general concepts of ISO 27001 and emphasizes the importance of aligning risk management activities with the organization’s overall strategy and objectives [
42]. Meanwhile, NIST SP 800-30 has been proven to provide more contributions such as knowledge about information security that is consistent and comprehensive for policy makers, thus supporting decision makers that do not hesitate to take risks because each risk has been well investigated [
43]. In addition, this study also applied the KAMI index as a research novelty in evaluating the level of maturity, the level of completeness of the implementation of ISO 27001, and the map of the information system security governance area in an organization. The KAMI index was once used by Sofyan et al. (2024) [
44] as well, who successfully employed it in the form of a score for the category of electronic systems used and the results of an evaluation of the level of completeness of implementation based on the ISO 27001 standard. Therefore, through SOC guidance assessment recommendations, which can be seen in
Figure 9, the study presents a perimeter of the adaptive ecosystem by reframing security by embedding it from design to deployment.
4.2. Implementation
At this stage, the steps involved in utilizing SOC guidance assessment recommendations are utilized to ensure a structured and effective approach to system security evaluation. The implementation is formulated as a combination of two widely recognized frameworks: ISO/IEC 27005:2018, which serves as a comprehensive risk management framework, and NIST SP 800-30 Rev. 1, which provides detailed risk assessment guidelines. By integrating these two standards, organizations can successfully conduct risk management processes that align with the industry’s best practices while maintaining a practical and systematic approach to identifying, analyzing, and mitigating security threats. In addition, the ISO/IEC 27005:2018 framework focuses on establishing a structured methodology for managing risks within an organization’s information security system, ensuring that potential threats are assessed and addressed effectively. Meanwhile, NIST SP 800-30 Rev. 1 complements this framework by offering structured guidelines for risk assessment, also enabling organizations to evaluate security vulnerabilities in a methodical manner. This combined approach enhances the effectiveness of risk management efforts, making it easier for organizations to implement security measures that are both comprehensive and practical. By following these guidance recommendations, organizations can improve their SOC operations, ensuring that security risks are proactively managed and mitigated to maintain a resilient cybersecurity posture, thus presenting metrics in terms of cognitive, autonomous, responsive and predictive security.
4.2.1. Cybersecurity
In cybersecurity, maintaining the security of information systems requires the application of fundamental principles that ensure data protection and reliability. One of these principles is confidentiality, which ensures that only authorized individuals have access to sensitive information, preventing unauthorized access and potential data breaches. Meanwhile, integrity is another critical principle focusing on the security and accuracy of data by preventing any unauthorized modifications, corruption, or destruction caused by threats such as viruses or hackers. This ensures that the information remains complete, reliable, and unaltered. Additionally, availability is essential in ensuring that all authorized users within an organization can access the necessary information on designated websites or platforms at any time and from any location. By guaranteeing continuous access, organizations can maintain smooth operations without disruptions caused by cyber threats or system failures. The fulfillment of these principles plays a crucial role in strengthening an organization’s data security, minimizing the risk of cyberattacks, and ensuring the reliability of its information systems. Implementing these security measures not only protects sensitive data but also enhances the overall resilience of the organization’s cybersecurity infrastructure [
45,
46]. Nonetheless, the organization should move from static compliance to continuous risk intelligence by incorporating real-time risk dashboards and automated audit evidence.
4.2.2. Risk Assessment and Scoring Procedure
At this stage, the KAMI index assessment can be carried out to help determine the condition of information security based on the SNI (
Standar Nasional Indonesia) following the ISO/IEC 27001 criteria. In the filling criteria within the list of questions provided in the KAMI index assessment, each answer has a certain value. The assessment also provides methods and formulas for processing obtained data that describe the condition of the company’s information security. The results of the KAMI index assessment are presented in the form of a dashboard that displays the electronic system category score, final evaluation results, the level of completeness of the implementation of the ISO27001 standard according to the electronic system category, and a radar chart that explains the level of information security readiness for each area [
47]. After obtaining the analysis results from the KAMI index, context determination is carried out [
41]. Then, risk criteria are used to track the level of risk as unacceptable or acceptable, which can include several limits with risk scale targets tailored to the needs of the organization.
This section also describes the methodology that was followed for the risk assessment and scoring process in the case study at PT NBFC, in which the risk assessment was performed by adopting an integrated approach of ISO/IEC 27005 and NIST SP 800-30 to identify, analyze, and evaluate security risks pertaining to SOC activities. The risk assessment process started with the identification of critical information assets linked to SOC operations, followed by the identification of relevant threat scenarios and vulnerabilities affecting such assets. Existing security controls were reviewed to understand the current risk mitigation measures already in place. This led to a set of defined risk scenarios that epitomize the possible security events related to the SOC environment. For each of the identified risk scenarios, the determination of risk levels was performed based on the assessment of likelihood and impact. The likelihood denotes the possibility of a threat exploiting vulnerability, while impact denotes potential consequences on organizational operations, information assets, and security objectives. Both likelihood and impact were assessed based on a five-level ordinal scale, from very low (1) to very high (5), in accordance with commonly adopted practices from ISO/IEC 27005 and NIST SP 800-30. Each scenario’s overall risk was calculated through the combination of scores over likelihood and impact by using the qualitative risk matrix. Consequently, the resulting risk values were categorized into low, medium, and high to support prioritization. The risk categorization was used to determine the proper treatment options for risk, including risk mitigation, risk acceptance, or risk monitoring, in accordance with organizational risk management. The risk scoring has been performed by expert judgment involving personnel in charge of information security and SOC operations. Scoring was carried out through structured review sessions to ensure consistency regarding the interpretation of criteria on likelihood and impact. In case of differences in assessment, discussion was held to obtain a common understanding of the risk level, while the outcome of the risk assessment provided the basis for the following maturity evaluation and analysis. The risk scale based on the NIST SP 800-30 framework can be seen in
Table 3.
In determining the context, the technology alone is insufficient for role-based security training, and gamified phishing simulations and behavior-based metrics become necessity for human-centered cultural transformation in prioritizing data value and ethics. Simply put, security should become a continuous process not a final gate in terms of compliance checks in secure-by-design architecture. Even an internal system must re-authenticate as well as re-authorize in every access request through continuous verification as opposed to one-time login and context-aware access for devices, location, and behavior. In this case, the impact criteria are using the level option, which is based on the level description in NIST SP 800-30 as can be seen in
Table 4. The likelihood criteria using the consideration of impacts that allow threats to occur and the possibility of threats arising or occurring can be seen in
Table 5.
4.2.3. Maturity Assessment Method
Here, the methodology used in assessing the SOC-related processes for maturity at PT NBFC is explained. In general, maturity assessment complements the risk assessment results by providing an additional dimension for evaluating the extent to which the existing practices related to the SOC are established, managed, and continuously improved within the organization. Assessment of maturity was performed according to a structured model that consists of five levels ranging from initial to optimized. With the increase in maturity level, more process definition, implementation, measurement, and continuous improvement are represented. The scope of this evaluation involves SOC-related domains relevant to information security risk management and operational effectiveness. Based on internal organizational documentation review and expert judgment of personnel involved in SOC and information security management, the scores of maturities were derived. Evidence examined included indications for documented procedures, consistency of implementation, monitoring of practices, and the improvement of mechanisms applied. Based on the predefined criteria for each maturity level considered, each domain was assessed, and a maturity level was assigned based on the appropriateness and completeness of meeting that criterion or set of criteria. For comparability, interpretation, and reporting, maturity levels were expressed as percentage values that indicated the degree of attainment of a particular domain. These percentages were utilized to facilitate visualization and comparison across domains rather than as absolute performance indicators. A target maturity level was defined by the organization internally based on its objectives and expectations of risk management, whereas the domains that attain or are near this target were highlighted for further analysis. The maturity assessment outcomes were analyzed jointly with the risk assessment results to determine the strengths, weaknesses, and areas of improvement for the SOC. This combined analysis provided further insight into how exposed risk and process maturity are interacting to influence SOC effectiveness overall.
4.2.4. Risk Assessment
At this stage, an assessment was carried out of the identified risks, and an evaluation was carried out for each risk scenario.
Risk identification is the process of finding, recognizing, or describing risk attributes as it includes identifying sources, events, and causes of risk in an organization.
- a.
Asset identification is the process of identifying assets starting with a weighted factor analysis of all ERP assets. Each information asset was scored for each important factor and given weight for each criterion. The weight values were obtained from the risk owner and IT risk officer in the organization. The weighted factor analysis criteria consist of criterion 1 (impact on revenue, 30%), criterion 2 (impact on profitability, 40%), and criterion 3 (impact on public image, 30%). Critical factors were assessed using a score ranging from 0.1 to 1.0, and criteria were given a weight ranging from 1 to 100, each weighted to indicate the importance of the criteria set for the organization. The range of values obtained refers to NIST SP 800-30 revision 1.
- b.
In threat identification, threat sources are divided into two categories, namely, adversarial and non-adversarial threat sources. Then, identification of all threats that disrupt information security aspects is carried out. The questions used to identify threats are as follows:
- i.
What threats to assets do you know or suspect?
- ii.
What threats are the most dangerous to the organization?
- iii.
What threats would be the most expensive to address if attacked?
- iv.
What threats would cost the most to prevent?
- c.
In identification of control, the security controls that the company has implemented to protect the organization’s assets from threats are identified.
- d.
Vulnerability identification explores the extent to which a company has implemented controls to protect assets from threats. Vulnerabilities that do not have associated threats may not require the implementation of controls, but they need to be identified and monitored. However, ineffective implementation of controls or controls that do not function properly can lead to vulnerabilities.
Risk analysis is the practice of mapping assets, asset values, threats, security controls, vulnerabilities, and impacts on CIA (confidentiality, integrity, and availability) aspects. In this case, it is intended to obtain impact assessment results and identify potential information security risks. In this study, the calculation of the risk using the formula can be presented as the probability of a successful attack on an organization (frequency of loss = likelihood × probability of attack success) multiplied by the expected loss from a successful attack (magnitude of loss = asset value × probability of loss) plus the uncertainty of estimating all the values mentioned. Loss frequency is a measure of the likelihood of an attack occurring combined with the probability of the attack being successful if it targets an organization. Loss magnitude is a combination of the value of an asset and the probability of its loss due to an attack.
Risk evaluation in this study aimed to compare the results of the risk analysis with the risk criteria then determine whether the risk rating is acceptable or tolerable. The stages of risk evaluation include the preparation of risk priorities based on the magnitude of the risk, with the following provisions:
- a.
The highest risk level has the highest priority.
- b.
If there is more than one risk with the same risk magnitude, then the risk priority is determined based on the order of the impact area from the highest to the lowest according to the amount of loss.
- c.
If there is still more than one risk that has the same magnitude and area of impact, then the risk priority is determined based on the order of the highest to the lowest risk category according to the frequency of loss.
- d.
If there is still more than one risk that has the same magnitude, amount of loss, and frequency of loss, then the risk priority is determined based on the assessment of the risk owner.
Risk determination is the initial step before determining risk priorities, while the priority risk matrix is classified based on NIST SP 800-30 revision 1 and is a matrix of relationships between assets and threats.
4.2.5. Risk Treatment and Risk Acceptance Strategy
At this stage, an assessment is carried out of the identified risks, and an evaluation is carried out for each risk scenario.
Risk Treatment Strategy: Risk handling aims to control the risk that is harmful by developing relevant handling to control the risk causes, measuring the effectiveness of the handling and, if the estimated risk value remains at an intolerable level, prepare alternative handling. According to ISO/IEC 27005:2018, there are four risk handling options, namely, risk modification, risk avoidance, risk sharing, and risk retention. In this study, 142 risks were found with unacceptable decisions for 32 modification risks. The total risk acceptance is 110, consisting of 90 risk retentions, two risk avoidances, and 18 risk sharing incidents. The risk handling scenarios were sorted based on risk priority from the highest to the lowest risk level. The following in
Table 6 is an example of risk priority.
- 2.
A risk acceptance strategy was carried out to explain more clearly some of the security controls that have been selected for risk management. In this study, an information security team was formed to determine the roles and responsibilities of the Person in Charge (PIC) of information security activities in each related division within the organization. In determining information security controls, the PIC is required to be responsible for accepting the risk.
4.2.6. Monitoring
At this stage, monitoring is carried out based on the risk list while documenting and monitoring risks. The process uses a risk list that provides holistic information about risks and allows stakeholders to make decisions related to these risks and their management. In information security 4.0, monitoring is no longer a passive process but it is intelligent, with multi-layer architecture within cycles and should be automated through observability by AI-assisted interpretation or clear expert operational explanation. Of course, the key idea here is that monitoring without response is obsolete, and control deviation instantly influences risk score and triggers mitigation workflow that impacts the risk heatmap. The risk owner or PIC uses the risk list to document and manage risks to the asset organization.
4.2.7. SOC Assessment
At this stage, an SOC assessment of an organization in handling information system security threats is produced. This can minimize the risk of information system security in an organization. The results of the conducted research are expected to provide benefits, while the implications that can be used are as follows:
They are useful as a reference for the strengths and weaknesses of ISO 27005 and NIST in information system security.
They can be used as a reference in the SOC information system security assessment for organizations to use the proposed information system security assessment framework to protect organizations from the threat of cyberattacks.
4.3. Evaluation Framework
At this stage, the framework assessment consists of ISO 27005:2018 and NIST SP 800-30, and the proposed framework combines ISO 27005:2018 with NIST SP 800-30 for the assessment of information system security at PT NBFC by calculating the maturity value or level to find out the extent to which the organization has implemented the clauses and annexes in the framework. Essentially, the process does not eliminate humans but augments them to validate high-impact decision. The maturity level measurement instrument used can be seen in
Table 7 [
48].
To determine the maturity level of the clauses and annexes, use the maturity level Formula (1) [
49].
The following are the evaluation results of the ISO 27005:2018 framework, NIST SP 800-30, and the proposed framework:
In the ISO 27005 framework assessment, the maturity method is carried out based on the domain context establishment, risk analysis/assessment, risk treatment, risk acceptance, risk communication, monitoring, and critical analysis. Furthermore, a spider web graph is created to find out which domain is approaching the maturity target. The desired maturity target for the organization is 80% at category level 4—Managed and Measurable. The spider web graph in ISO 27005:2018 can be seen in
Figure 10, from which it can be concluded that only the context establishment has exceeded the target, while the domains that have not reached the maturity target are related to the risk analysis/assessment, risk treatment, risk acceptance, risk communication, monitoring, and critical analysis domains.
- 2.
NIST SP 800-30 Framework Assessment
In the NIST SP 800-30 framework assessment, the capability maturity method is used based on the domain identify, protect, detect, respond, and recovery. Furthermore, a spider web graph is also created to determine which domains are approaching the maturity target. The spider web graph in NIST SP 800-30 can be seen in
Figure 11, from which it can be concluded that the detect and respond domains have exceeded the target, while the domains that have not reached the maturity target are the identify, protect, and recover domains.
- 3.
Assessment of the Proposed Framework
In the assessment of the new framework, a combination of the ISO 27005:2018 and NIST SP 800-20 frameworks is carried out using the capability maturity method based on the domain research assessment, risk assessment, risk analysis, risk evaluation, risk treatment, risk acceptance, and monitoring. Furthermore, a spider web graph is created to find out which domains are approaching the maturity target, which is expected by the organization to be 80% at the Managed and Measurable level. The spider web graph in the proposed framework can be seen in
Figure 12, from which it can be concluded that the research assessment and risk evaluation domains have exceeded the target, while the domains that have not reached the maturity target are the risk assessment, risk analysis, risk treatment, risk acceptance, and monitoring domains. The risk acceptance domain only reaches level 1 initially, so the organization needs to make improvements in the risk acceptance domain related to the risk approval of information security.
From the description of the framework assessment that has been carried out consisting of the ISO 27005:2018 framework, NIST SP 800-30, and the proposed framework, it was concluded that the proposed framework achieved good maturity because there were two domains that reached the target maturity value and one domain that was at level 4—Managed and Measurable. In addition, the domains used in the new framework include domains from ISO 27005:2018 and NIST SP 800-30 so that organizations can utilize a more comprehensive approach, involving strategic, managerial, and technical aspects of risk management.
6. Conclusions
The research on the analysis of assessment parameters on the SOC information system in the PT NBFC Case Study was successfully carried out, and the following conclusions were obtained. The evaluation shows that the ISO 27005:2018 and NIST SP 800-30 frameworks each have advantages and disadvantages in assessing information system security within the organization. ISO 27005:2018 focuses more on information security risk management as a whole system, while NIST SP 800-30 provides more detailed guidance on the risk assessment process. The proposed developed framework integrates important parameters from both frameworks with adjustments based on the review results. It uses maturity levels to adjust to the specific needs of PT NBFC, resulting in a context-appropriate evaluation. Meanwhile, the results of the maturity level assessment on ISO 27005:2018 show that only the context establishment has exceeded the target, while the other domain that has not reached the maturity target (80% level 4). On the other hand, in NIST SP 800-30, the detect and respond domains have exceeded the target compared with other domains. Meanwhile, the proposed framework that combines ISO 27005:2018 and NIST SP 800-30 has achieved good maturity and has been successfully implemented because there are two domains that have achieved the target maturity value and one domain that is already at level 4—Managed and Measurable. In addition, the domains used in the new framework include domains from ISO 27005:2018 and NIST SP 800-30 so that organizations can take advantage of a more comprehensive approach, involving strategic, managerial, and technical aspects of risk management. The proposed framework includes several parameters/domains, namely, research assessment, risk assessment, risk treatment, risk acceptance strategy, and monitoring. These parameters are designed to be applicable in various types of organizations by adjusting to the specific needs of each organization. Also, implementation of the proposed framework in the form of SOC assessment guidance involves the development of clear guidance on the assessment process, including practical steps to identify, analyze, and respond to threats. This guidance is designed to be used by SOC teams to improve their ability to assess and manage information security risks, which includes Input, Actions, and Implementation Guidance.