Benchmarking ISO Risk Management Systems to Assess Efficacy and Help Identify Hidden Organizational Risk
Abstract
:1. Introduction
- To develop a benchmarking model for risk management based on scientific literature and ISO standards in order to assess the efficacy of real risk management systems and see whether hidden risk can still be identified through ISO standard risk management systems and the risk assessment process used by operating organizations.
- To test the benchmarking model on six real-life and ISO-certified risk management systems.
2. Context for the Study
2.1. Recent Developments Influencing the Development of Benchmarking Models
- A benchmarking system can be applied to stimulate a genuine endeavor for perfection, rather than to judge or criticize [21].
2.2. Risk Management in ISO Standards
- The scope of the risk management system must be defined.
- The risk management process must be documented.
- Policies regarding risk management must exist and be documented.
- Internal audits must be conducted.
- Management review and formal review and approval for suitability and adequacy, for example, review of operational planning and control, assessments of risk, nonconformity, and the efficacy of any corrective action taken.
- Knowledge of all legal requirements must exist.
- Risk and root cause analysis must be conducted.
- Risk assessment/evaluation must be conducted.
- Criteria must be set for the management system process and risk/quality acceptance.
2.3. Scientific Literature on Risk Issues in Risk Management Systems
- Treatment of residual risk [39].
3. Development of a Benchmarking Model for an ISO Risk Management System
- Step 1: Validation and evaluation of the foundational elements of a generic risk management system that is based on ISO standards. Assessment template with a simple scoring system.
- Step 2: Validation and evaluation of some of the most critical elements of the risk management process, according to ISO and scientific literature on risk management issues.
3.1. Step 1
- Scope, context, and boundaries of the risk management system.
- Compliance with regulative requirements concerning the business.
- Certifications.
- Policies regarding risk are documented.
- Risk management system is documented.
- Risk analysis is conducted in a formal way.
- Risk assessment is conducted in a formal way.
- Risk (acceptance) criteria are set.
- Residual risk is addressed (identified and assessed).
3.2. Step 2
- Scope and outer boundaries of the risk management system.
- Internal boundaries and interfaces, complexity of the organizational structure, and distribution of accountability.
- Hierarchical structure with regard to risk, both safety and security risk.
- Resources, knowledge, and experience needed to support the risk management system.
- 5.
- Risk analysis ability to capture complexity of the business operation and systems (foundation, method, technique).
- 6.
- Risk assessment ability to capture risk evaluation (ability to capture risk knowledge).
- 7.
- Risk criteria setting in risk assessment.
- 8.
- Identification and treatment of residual risk, risk that is left after formal risk mitigation/treatment.
4. Research Methodology and Hypotheses
4.1. Research Methodology
4.1.1. Setting Selection Criteria for Participants in the Study
4.1.2. Questionnaire
4.1.3. Interviews
4.2. Hypothesis
5. Results
5.1. Public Health Service
5.1.1. Results from the Questionnaire
5.1.2. Results from the Interview
5.1.3. Summarized Results from the Public Health Service
5.2. Public Supply System
5.2.1. Results from the Questionnaire
5.2.2. Results from the Interview
5.2.3. Summarized Results from the Public Supply System
5.3. Construction Company
5.3.1. Results from the Questionnaire
5.3.2. Results from the Interview
5.3.3. Summarized Results from the Construction Company
5.4. Manufacturing Company
5.4.1. Results from the Questionnaire
5.4.2. Results from the Interview
5.4.3. Summarized Results from the Manufacturing Company
5.5. Software Company
5.5.1. Results from the Questionnaire
5.5.2. Results from the Interview
- Inherent risk factor, the base security risk, is calculated for every asset based on four variables: the likelihood of threat, the impact of threat, the vulnerability of the asset towards the threat, and the value of the asset of which the threat is associated with. All four variables are evaluated on a scale between 1 and 5.
- The second risk calculation is the current security risk. Risk is calculated with regard to implemented controls. A threat library is used in this calculation. Every threat is related to several controls from ISO/IEC 27001 which are meant to mitigate it. A calculation is made that considers, on the one hand, controls that are already implemented and, on the other hand, controls that have been defined as possible but have not yet been implemented. This gives a risk factor that can be compared to the inherent risk factor to assess the benefits of the measures that have already been taken.
- The third risk calculation is similar to the second risk calculation. It takes into consideration both implemented and future controls, i.e., controls which are being considered or have already been chosen to be implemented but have not yet been implemented. This calculation is made to evaluate the benefit of future controls.
5.5.3. Summarized Results from the Software Company
5.6. Pension Fund
5.6.1. Results from the Questionnaire
5.6.2. Results from the Interview
- Basic risk score = ((impact of risk) × (likelihood of risk)) + (impact other than financial)
- Quarterly risk score = (basic risk score) − ((effectiveness of mitigating control) × (basic risk score))
- Previous quarterly risk score
- Involvement of pension fund division
- Responsible division
- Description of risk factors
- Possible consequences of risk
- Description of mitigation controls
- Objectives
- Comments
- Reference to a documented process
5.6.3. Summarized Results from the Pension Fund
6. Discussion
6.1. First Aim: Development of a Benchmarking Model for Risk Management
6.2. Second Aim: Application of a Benchmarking Model for Evaluation of Real-Life ISO Risk Management Systems
- Scope and outer boundary issues were found in 2 out of 6 cases.
- Interface issues were found in 3 out of 6 cases.
- Hierarchical issues were found in 1 out of 6 cases.
- Resource issues were found in 2 out of 6 cases.
- Issues regarding risk analysis ability to capture complex systems and business operations were found in 4 out of 6 cases.
- Issues regarding risk assessment ability to capture risk evaluation were found in 4 out of 6 cases.
- Issues regarding setting of risk criteria were found in 4 out of 6 cases.
- Issues regarding residual risk were found in 4 out of 6 cases.
7. Conclusions and Future Work
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- COPOLCO. 2021. Available online: https://www.iso.org/sites/ConsumersStandards/1_standards.html (accessed on 15 February 2021).
- ISO 9001:2015; Quality Management Systems—Requirements. ISO: Geneva, Switzerland, 2015.
- ISO/IEC 27001:2013; Information Technology—Security Techniques—Information Security Management Systems—Requirements. ISO: Geneva, Switzerland, 2013.
- ISO 45001:2018; Occupational Health and Safety Management Systems—Requirements with Guidance for Use. ISO: Geneva, Switzerland, 2018. Available online: https://www.iso.org/cms/render/live/en/sites/isoorg/contents/data/standard/06/37/63787.html (accessed on 9 March 2022).
- ISO 22000:2018; Food Safety Management Systems—Requirements for any Organization in the Food Chain. ISO: Geneva, Switzerland, 2018. Available online: https://www.iso.org/cms/render/live/en/sites/isoorg/contents/data/standard/06/54/65464.html (accessed on 14 July 2020).
- ISO 13485:2016; Medical Devices—Quality Management Systems—Requirements for Regulatory Purposes. ISO: Geneva, Switzerland, 2016.
- ISO 37001:2016; Anti-Bribery Management Systems—Requirements with Guidance for Use. ISO: Geneva, Switzerland, 2016. Available online: https://www.iso.org/cms/render/live/en/sites/isoorg/contents/data/standard/06/50/65034.html (accessed on 9 March 2022).
- Björnsdóttir, S.H.; Jensson, P.; de Boer, R.J.; Thorsteinsson, S.E. The Importance of Risk Management: What is Missing in ISO Standards? Risk Anal. 2021. [Google Scholar] [CrossRef] [PubMed]
- International Accreditation Forum, Inc. International Accreditation Forum—IAF. Find Members, Publications & Resources. 13 July 2020. Available online: https://www.iaf.nu/ (accessed on 7 September 2020).
- ISO—Management System Standards List. Available online: https://www.iso.org/management-system-standards-list.html (accessed on 9 July 2020).
- ISO 14001:2015; Environmental Management Systems—Requirements with Guidance for Use. ISO: Geneva, Switzerland, 2015.
- ISO 31000:2018; Risk Management—Principles and Guidelines. ISO: Geneva, Switzerland, 2018.
- ISO 19011:2018; Guidelines for Auditing Management Systems. IEC: Geneva, Switzerland, 2018. Available online: https://www.iso.org/cms/render/live/en/sites/isoorg/contents/data/standard/07/00/70017.html (accessed on 20 July 2020).
- Talapatra, S.; Uddin, M.K.; Rahman, M.H. Development of an Implementation Framework for Integrated Management System Based on the Philosophy of Total Quality Management. Am. J. Ind. Bus. Manag. 2018, 8, 6. [Google Scholar] [CrossRef] [Green Version]
- Talapatra, S.; Uddin, M.K. Prioritizing the barriers of TQM implementation from the perspective of garment sector in developing countries. Benchmarking Int. J. 2019, 26, 2205–2224. [Google Scholar] [CrossRef]
- Franceschini, F.; Galetto, M.; Cecconi, P. A worldwide analysis of ISO 9000 standard diffusion: Considerations and future development. Benchmarking Int. J. 2006, 13, 523–541. [Google Scholar] [CrossRef]
- Herbst, N.; Bauer, A.; Kounev, S.; Oikonomou, G.; Eyk, E.V.; Kousiouris, G.; Evangelinou, A.; Krebs, R.; Brecht, T.; Abad, C.L.; et al. Quantifying Cloud Performance and Dependability: Taxonomy, Metric Design, and Emerging Challenges. ACM Trans. Model. Perform. Eval. Comput. Syst. 2018, 3, 1–36. [Google Scholar] [CrossRef]
- Kounev, S.; Lange, K.-D.; von Kistowski, J. Systems Benchmarking: For Scientists and Engineers; Springer International Publishing: Cham, Switzerland, 2020. [Google Scholar] [CrossRef]
- Olawumi, T.O.; Chan, D.W.M. Development of a benchmarking model for BIM implementation in developing countries. Benchmarking Int. J. 2019, 26, 1210–1232. [Google Scholar] [CrossRef]
- Van der Voordt, T.J.M.; Jensen, P.A. Measurement and benchmarking of workplace performance: Key issues in value adding management. J. Corp. Real Estate 2018, 20, 177–195. [Google Scholar] [CrossRef] [Green Version]
- Staiger, R.D.; Schwandt, H.; Puhan, M.A.; Clavien, P.-A. Improving surgical outcomes through benchmarking. Br. J. Surg. 2019, 106, 59–64. [Google Scholar] [CrossRef]
- Hartono, E.O.; Abdullah, D. HFLTS-DEA Model for Benchmarking Qualitative Data. Int. J. Adv. Soft Compu. Appl. 2019, 11, 109–131. [Google Scholar]
- Mangla, S.K.; Luthra, S.; Jakhar, S. Benchmarking the risk assessment in green supply chain using fuzzy approach to FMEA: Insights from an Indian case study. Benchmarking Int. J. 2018, 25, 2660–2687. [Google Scholar] [CrossRef]
- Hoffmann, P.; Schiele, H.; Krabbendam, K. Uncertainty, supply risk management and their impact on performance. J. Purch. Supply Manag. 2013, 19, 199–211. [Google Scholar] [CrossRef]
- Björklund, M. Benchmarking tool for improved corporate social responsibility in purchasing. Benchmarking Int. J. 2010, 17, 340–362. [Google Scholar] [CrossRef]
- Moriarty, J.P.; Smallman, C. En route to a theory of benchmarking. Benchmarking Int. J. 2009, 16, 484–503. [Google Scholar] [CrossRef]
- MacGillivray, B.H.; Sharp, J.V.; Strutt, J.E.; Hamilton, P.D.; Pollard, S.J.T. Benchmarking Risk Management Within the International Water Utility Sector. Part II: A Survey of Eight Water Utilities. J. Risk Res. 2007, 10, 105–123. [Google Scholar] [CrossRef] [Green Version]
- Talapatra, S.; Uddin, M.K.; Antony, J.; Gupta, S.; Cudney, E.A. An empirical study to investigate the effects of critical factors on TQM implementation in the garment industry in Bangladesh. Int. J. Qual. Reliab. Manag. 2019, 37, 1209–1232. [Google Scholar] [CrossRef]
- Talapatra, S.; Uddin, K. Understanding the difficulties of implementing TQM in garment sector: A case study of some RMG industries in Bangladesh. In Proceedings of the International Conference on Mechanical, Industrial and Materials Engineering 2017 (ICMIME2017), Rajshahi, Bangladesh, 28–30 December 2017; p. 6. Available online: http://icmime-ruet.ac.bd/2017/DIR/Contents/Technical%20Papers/Industrial%20Engineering/IE-243.pdf (accessed on 1 November 2021).
- Talapatra, S.; Uddin, K. Some Obstacles that Affect the TQM Implementation in Bangladeshi RMG Sector: An Empirical Study; IEOM Society International: Bandung, Indonesia, 2018; p. 13. Available online: http://ieomsociety.org/ieom2018/papers/401.pdf (accessed on 9 March 2022).
- Aven, T.; Zio, E. Foundational Issues in Risk Assessment and Risk Management. Risk Anal. 2014, 34, 1164–1172. [Google Scholar] [CrossRef]
- Klinke, A.; Renn, O. A New Approach to Risk Evaluation and Management: Risk-Based, Precaution-Based, and Discourse-Based Strategies. Risk Anal. 2002, 22, 1071–1094. [Google Scholar] [CrossRef]
- Cox, L.A. What’s Wrong with Risk Matrices? Risk Anal. 2008, 28, 497–512. [Google Scholar] [CrossRef]
- IEC 31010:2019; Risk management—Risk assessment techniques. IEC: Geneva, Switzerland, 2019.
- Aven, T. Improving risk characterisations in practical situations by highlighting knowledge aspects, with applications to risk matrices. Reliab. Eng. Syst. Saf. 2017, 167, 42–48. [Google Scholar] [CrossRef]
- Fellows, R.; Liu, A.M.M. Managing organizational interfaces in engineering construction projects: Addressing fragmentation and boundary issues across multiple interfaces. Constr. Manag. Econ. 2012, 30, 653–671. [Google Scholar] [CrossRef] [Green Version]
- Mikes, A. From counting risk to making risk count: Boundary-work in risk management. Account. Organ. Soc. 2011, 36, 226–245. [Google Scholar] [CrossRef] [Green Version]
- Zerjav, V. Design boundary dynamics in infrastructure projects: Issues of resource allocation, path dependency and problem-solving. Int. J. Proj. Manag. 2015, 33, 1768–1779. [Google Scholar] [CrossRef] [Green Version]
- Lathrop, J.; Ezell, B. A systems approach to risk analysis validation for risk management. Saf. Sci. 2017, 99, 187–195. [Google Scholar] [CrossRef]
- Blood Transfusion Guide—EDQM Publications|EDQM—European Directorate for the Quality of Medicines. 2020. Available online: https://www.edqm.eu/en/blood-guide (accessed on 29 April 2021).
- WHO Action Framework to Advance Universal Access to Safe, Effective and Quality Assured Blood Products. 2020. Available online: https://www.who.int/publications-detail-redirect/action-framework-to-advance-uas-bloodprods-978-92-4-000038-4 (accessed on 29 April 2021).
- ISO 14971:2019; Medical Devices—Application of Risk Management to Medical Devices. ISO: Geneva, Switzerland, 2019. Available online: https://www.iso.org/cms/render/live/en/sites/isoorg/contents/data/standard/07/27/72704.html (accessed on 9 March 2022).
- IEC 62366-1:2015; Medical Devices—Part 1: Application of Usability Engineering to Medical Devices. IEC: Geneva, Switzerland, 2015.
- ISO/IEC 27005:2018; Information Technology—Security Techniques—Information Security Risk Management. ISO: Geneva, Switzerland, 2018. Available online: https://www.iso.org/cms/render/live/en/sites/isoorg/contents/data/standard/07/52/75281.html (accessed on 13 July 2020).
- Leveson, N. A new accident model for engineering safer systems. Saf. Sci. 2004, 42, 237–270. [Google Scholar] [CrossRef] [Green Version]
- Leveson, N.G. Engineering a Safer World. 2011. Available online: https://mitpress.mit.edu/books/engineering-safer-world (accessed on 3 July 2018).
- Leveson, N. A systems approach to risk management through leading safety indicators. Reliab. Eng. Syst. Saf. 2015, 136, 17–34. [Google Scholar] [CrossRef] [Green Version]
- The Global Risks Report 2021. The World Economic Forum. 2021. Available online: https://www3.weforum.org/docs/WEF_The_Global_Risks_Report_2021.pdf (accessed on 12 April 2021).
- The Global Risks Report 2022. The World Economic Forum. 2022. Available online: https://www3.weforum.org/docs/WEF_The_Global_Risks_Report_2022.pdf (accessed on 9 March 2022).
No. | Benchmark Name | Corresponding Risk Management (RM) Principle/Framework/Process Clause in ISO 31000 |
---|---|---|
1 | Scope and outer boundaries of a RM system | Process (clause 6): Scope, context, and criteria (6.3) |
2 | Interfaces (internal boundaries, departments, unclear responsibility) within a RM system | Process (clause 6): Scope, context, and criteria (6.3) |
3 | Hierarchical issues (layer issues, unclear hierarchical safety and security structure) within a RM system | Principles (clause 4): Structured, comprehensive, and dynamic RM Framework (clause 5): Leadership and commitment (clause 5.2) Process (clause 6): Risk assessment (clause 6.4) and risk treatment (clause 6.5) |
4 | Resources available to support the RM system | Framework (clause 5): Leadership and commitment (clause 5.2) |
5 | Risk analysis ability (foundation, method) to capture complexity | Process (clause 6): Risk assessment (clause 6.4) |
6 | Risk assessment ability to capture risk evaluation | Process (clause 6): Risk assessment (clause 6.4) |
7 | Risk criteria setting in risk assessment | Process (clause 6): Risk assessment (clause 6.4) and risk treatment (clause 6.5) |
8 | Treatment of residual risk, risk that is left after risk mitigation | Principles (clause 4): Continual improvements Framework (clause 5): Improvement (clause 5.7) Process (clause 6): Risk assessment (clause 6.4), risk treatment (clause 6.5), monitoring and review (clause 6.6) |
ID | Organization | Business Operation | Accredited ISO Certifications |
---|---|---|---|
A | Public health service | Processing of biological samples | ISO 9001 |
B | Public supply system | Operation of an electricity transmission system | ISO 9001, ISO 14001, ISO 45001 |
C | Construction company | Construction of an infrastructure facility | ISO 9001, ISO 14001, ISO/IEC 27001, ISO 45001 |
D | Manufacturing company | Manufacturing of a medical device | ISO 14001, ISO 13485 |
E | Software company | Software development | ISO/IEC 27001 |
F | Pension fund | Financial investments | ISO 9001, ISO/IEC 27001 |
No. | Question/Topic | A—Public Health Service | B—Public Supply System | C—Construction Company | D—Manufacturing Company | E—Software Company | F—Pension Fund |
---|---|---|---|---|---|---|---|
1 | General information | ||||||
1.1 | Listed (on Nasdaq) | no | no | no | yes | yes | no |
1.2 | Number of employees (European Union classification) | 51–250 | 51–250 | 251–500 | 501–5000 | 11–50 | 11–50 |
1.3 | Number of local sites/offices | 4 | 2 | 7 | 1 | 1 | 1 |
1.4 | Number of countries with subsidiaries | 1 | 1 | 1 | 18 | 2 | 1 |
1.5 | Intl. business operations and export | no | no | yes | yes | yes | yes |
2 | Compliance | ||||||
2.1 | Relevant laws and regulations for business identified | yes | yes | yes | yes | yes | yes |
3 | Certification | ||||||
3.1 | Operations ISO certified | all | all | all | partly | all | all |
3.1.1 | … if yes, by an accredited certification body | yes | yes | yes | yes | yes | yes |
3.1.2 | … if yes, name of certification body | list | list | list | list | list | list |
3.2 | Non-ISO certifications | yes | no | yes | yes | no | no |
3.2.1 | … if yes, which parts | list | list | list | list | list | list |
3.2.2 | … if yes, which accredited certification body | list | list | list | list | list | list |
4 | Policies | ||||||
4.1 | Safety and/or security policy exist | yes | yes | yes | yes | yes | yes |
4.2 | Documented safety and/or security policy exists | no | yes | yes | yes | yes | yes |
4.3 | Ref. to relevant law(s)/regulation(s) in policy documents | list | list | list | list | list | list |
4.4 | Other policy documents relevant to safety/security | yes | yes | yes | yes | yes | yes |
5 | Risk management system | ||||||
5.1 | Formal risk management process in place | yes | yes | yes | yes | yes | yes |
5.2 | Risk assessment conducted | yes | yes | yes | yes | yes | yes |
5.3 | Risk analysis conducted | yes | yes | yes | yes | yes | yes |
5.4 | Internal control | yes | yes | yes | yes | yes | yes |
5.5 | Audits, internal and/or external | yes | yes | yes | yes | yes | yes |
5.6 | Review process | yes | yes | yes | no | yes | yes |
6 | Risk analysis | ||||||
6.1 | Formal methodology used | yes | yes | yes | yes | yes | yes |
6.2 | Use of special software solution for risk analysis | no | yes | no | no | yes | no |
6.3 | ISO guidelines used for doing risk analysis | no | yes | yes | yes | yes | yes |
6.4 | Likelihood of risk assessed | no | yes | yes | yes | yes | yes |
6.5 | Risk evaluated | yes | yes | yes | yes | yes | yes |
7 | Risk assessment | ||||||
7.1 | Tangible assets registered | yes | yes | yes | n.a. | yes | yes |
7.2 | Intangible assets registered | yes | yes | yes | n.a. | yes | yes |
7.3 | Threats identified | yes | yes | yes | n.a. | yes | yes |
7.4 | Consequence of risk assessed | yes | yes | yes | yes | yes | yes |
7.5 | Risk calculated | no | yes | yes | yes | yes | yes |
7.6 | Systematic risk mitigation with controls | yes | yes | yes | yes | yes | yes |
7.7 | Risk calculation after selecting controls—efficacy of controls assessed | no | yes | n.s. | no | yes | yes |
7.8 | Assessment on efficacy and usefulness of risk analysis in terms of cost | no | yes | yes | no | no | no |
7.9 | Risk information used for improvements—someone responsible | yes | yes | yes | yes | yes | yes |
7.10 | Result of risk assessment documented | yes | yes | yes | yes | yes | yes |
7.11 | Result of risk assessment used to learn from it | n.s. | yes | yes | yes | yes | yes |
8 | Risk criteria | ||||||
8.1 | Risk criteria set | no | yes | yes | yes | yes | yes |
9 | Residual risk | ||||||
9.1 | Residual risk assessed | no | yes | no | yes | yes | yes |
No. | Benchmark | Issues Found | Hypothesis (True/False) |
---|---|---|---|
1 | Scope and outer boundaries of a RM system | Outer boundaries of RM system stretched into other health care institutions without compliance with ISO procedures | True |
2 | Interfaces (internal boundaries, departments, unclear responsibility) within a RM system | Boundary issues regarding joint service and infrastructure of the hospital | True |
3 | Hierarchical issues (layer issues, unclear hierarchical safety, and security structure) within a RM system | No issues found | False |
4 | Resources available to support the RM system | No issues found | False |
5 | Risk analysis ability to capture complex systems and business operations | Limited ability to capture complexity | True |
6 | Risk assessment ability to capture risk evaluation | Two-dimensional risk metrics does not capture risk evaluation | True |
7 | Risk criteria setting in risk assessment | Risk criteria unclear | True |
8 | Treatment of residual risk | Residual risk not addressed | True |
No. | Benchmark | Issues Found | Hypothesis (True/False) |
---|---|---|---|
1 | Scope and outer boundaries of a RM system | Risk associated with stakeholders not always addressed | True |
2 | Interfaces (internal boundaries, departments, unclear responsibility) within a RM system | Internal boundaries well defined but bottom-up risk assessment within departments has led to causality between risk factors not being identified | True |
3 | Hierarchical issues (layer issues, unclear hierarchical safety, and security structure) within a RM system | Hierarchical issues found | True |
4 | Resources available to support the RM system | Resource issues found | True |
5 | Risk analysis ability to capture complex systems and business operations | Limited ability to capture complexity | True |
6 | Risk assessment ability to capture risk evaluation | Two-dimensional risk metrics does not capture risk evaluation | True |
7 | Risk criteria setting in risk assessment | Risk criteria sometimes unclear | True |
8 | Treatment of residual risk | Not every known risk is included in the risk assessment and treated therefore left as residual risk | True |
No. | Benchmark | Issues Found | Hypothesis (True/False) |
---|---|---|---|
1 | Scope and outer boundaries of a RM system | No issues found | False |
2 | Interfaces (internal boundaries, departments, unclear responsibility) within a RM system | No issues found | False |
3 | Hierarchical issues (layer issues, unclear hierarchical safety, and security structure) within a RM system | No issues found | False |
4 | Resources available to support the RM system | No issues found | False |
5 | Risk analysis ability to capture complex systems and business operations | No issues reported | Not verified |
6 | Risk assessment ability to capture risk evaluation | No issues reported | Not verified |
7 | Risk criteria setting in risk assessment | No issues reported | Not verified |
8 | Treatment of residual risk | No issues reported | Not verified |
No. | Benchmark | Issues found | Hypothesis (True/False) |
---|---|---|---|
1 | Scope and outer boundaries of a RM system | No issues found | False |
2 | Interfaces (internal boundaries, departments, unclear responsibility) within a RM system | No issues found | False |
3 | Hierarchical issues (layer issues, unclear hierarchical safety, and security structure) within a RM system | No issues found | False |
4 | Resources available to support the RM system | No issues found | False |
5 | Risk analysis ability to capture complex systems and business operations | No issues reported | Not verified |
6 | Risk assessment ability to capture risk evaluation | No issues reported | Not verified |
7 | Risk criteria setting in risk assessment | No issues found | False |
8 | Treatment of residual risk | No issues reported | Not verified |
No. | Benchmark | Issues Found | Hypothesis (True/False) |
---|---|---|---|
1 | Scope and outer boundaries of a RM system | No issues found | Fales |
2 | Interfaces (internal boundaries, departments, unclear responsibility) within a RM system | Internal boundaries sometimes unclear | True |
3 | Hierarchical issues (layer issues, unclear hierarchical safety, and security structure) within a RM system | No issues found | False |
4 | Resources available to support the RM system | Lack of resources | True |
5 | Risk analysis ability to capture complex systems and business operations | Limited ability to capture complexity | True |
6 | Risk assessment ability to capture risk evaluation | Limited ability to capture risk evaluation | True |
7 | Risk criteria setting in risk assessment | Risk setting unclear | True |
8 | Treatment of residual risk | Residual risk partly addressed | True |
No. | Benchmark | Issues found | Hypothesis (True/False) |
---|---|---|---|
1 | Scope and outer boundaries of a RM system | No issues found | False |
2 | Interfaces (internal boundaries, departments, unclear responsibility) within a RM system | No issues found | False |
3 | Hierarchical issues (layer issues, unclear hierarchical safety, and security structure) within a RM system | No issues found | False |
4 | Resources available to support the RM system | No issues found | False |
5 | Risk analysis ability to capture complex systems and business operations | Limited ability to capture complexity | True |
6 | Risk assessment ability to capture risk evaluation | Risk assessment ability to capture risk evaluation is limited | True |
7 | Risk criteria setting in risk assessment | Risk criteria unclear | True |
8 | Treatment of residual risk | Treatment of residual risk unclear and residual risk not always addressed | True |
No. | Benchmark | Corresponding to Risk Management (RM) in ISO 31000:2018 | Risk Issues Found | ||||||
---|---|---|---|---|---|---|---|---|---|
A—Public Health Service | B—Public Supply System | C—Construction Company | D—Manufacturing Company | E—Software Company | F—Pension Fund | Frequency of Risk Issues | |||
1 | Scope and outer boundaries of a RM system | Process: Scope, context, and criteria | x | x | 2 | ||||
2 | Interfaces (internal boundaries, departments, unclear responsibility) within a RM system | Process: Scope, context, and criteria | x | x | x | 3 | |||
3 | Hierarchical issues (layer issues, unclear hierarchical safety, and security structure) within a RM system | Principles: Structured, comprehensive, and dynamic Framework: Leadership and commitment Process: Risk assessment and treatment | x | 1 | |||||
4 | Resources available to support the RM system | Framework: Leadership and commitment | x | x | 2 | ||||
5 | Risk analysis ability to capture complex systems and business operations | Process: Risk assessment | x | x | n.v. | n.v. | x | x | 4 |
6 | Risk assessment ability to capture risk evaluation | Process: Risk assessment | x | x | n.v. | x | x | 4 | |
7 | Risk criteria setting in risk assessment | Process: Risk assessment and treatment | x | x | n.v. | n.v. | x | x | 4 |
8 | Treatment of residual risk | Principles: Continual improvements Framework: Improvement Process: Risk assessment, treatment, monitoring, and review | x | x | n.v. | n.v. | x | x | 4 |
Total no. of risk issues found in RM system | 6 | 8 | 6 | 4 | 24 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Björnsdottir, S.H.; Jensson, P.; Thorsteinsson, S.E.; Dokas, I.M.; de Boer, R.J. Benchmarking ISO Risk Management Systems to Assess Efficacy and Help Identify Hidden Organizational Risk. Sustainability 2022, 14, 4937. https://doi.org/10.3390/su14094937
Björnsdottir SH, Jensson P, Thorsteinsson SE, Dokas IM, de Boer RJ. Benchmarking ISO Risk Management Systems to Assess Efficacy and Help Identify Hidden Organizational Risk. Sustainability. 2022; 14(9):4937. https://doi.org/10.3390/su14094937
Chicago/Turabian StyleBjörnsdottir, Svana Helen, Pall Jensson, Saemundur E. Thorsteinsson, Ioannis M. Dokas, and Robert J. de Boer. 2022. "Benchmarking ISO Risk Management Systems to Assess Efficacy and Help Identify Hidden Organizational Risk" Sustainability 14, no. 9: 4937. https://doi.org/10.3390/su14094937
APA StyleBjörnsdottir, S. H., Jensson, P., Thorsteinsson, S. E., Dokas, I. M., & de Boer, R. J. (2022). Benchmarking ISO Risk Management Systems to Assess Efficacy and Help Identify Hidden Organizational Risk. Sustainability, 14(9), 4937. https://doi.org/10.3390/su14094937