The Development of Risk Assessments and Supplier Resilience Models for Military Industrial Supply Chains Considering Rare Disruptions

Abstract

Background: Supply chain risk and resilience in non-profit-seeking industries involving governmental agencies and quasi-governmental agencies have been under-studied. This paper focuses on the military industrial supply chain to demonstrate the development of risk assessment and supplier resilience models considering one-off disruption events such as the COVID-19 disruption. Methods: We establish relevant resilience-based categories through a literature review, supported by the experiences of supply chain experts within the military industry. We quantify the severity of the identified resilience categories, their detectability, and their occurrence probabilities. The failure modes and effects analysis technique is used to evaluate the risk priorities for the resilience categories to develop a risk assessment model. The risk assessment model is then extended to a supplier resilience model by incorporating specific rare disruption factors, which can act as a scenario planning tool. Results: It is found that (i) the top four resilience sub-categories are financial, topical data, business continuity planning, and supply chain mapping, while cost reduction strategies and green material usage are the least important; (ii) the main areas requiring focus are topical data, supply chain depth awareness, business continuity management, and internal risk management; and (iii) suppliers have least resilience in the areas of ‘topical information’ and ‘business continuity strategy’. Conclusions: The tool developed can help military industrial supply chains identify the main areas to enhance resilience from multiple perspectives of severity, occurrence probability, detectability, and suppliers.

1. Introduction

According to the supply chain resilience report launched by BCI in 2023, nearly 43% of key suppliers do not have business continuity arrangement in place and are vulnerable to supply chain disruptions [1]. The importance of a resilient supply chain (SC) has become acutely apparent during these unprecedented, post-COVID-19 times [2]. The level of supply chain disruption reported by organizations is more than twice as high as that of pre-pandemic levels [1]. Once stable and predictable, consumer and commercial environments have changed to become turbulent [3]. Understanding the key foundations that underpin a resilient and flexible SC has become a business necessity. Organisations need to learn to quickly adapt to ‘new’ threats and respond to close off the risks. The ability to track system vulnerability within a SC provides it with a robust and resilient foundation. The majority of supply chain risk research has focused on mature commercial industries, such as the automotive, electronics, aerospace, fashion, and food industries [4], but much less has been reported on logistics and supply chain risk in non-profit-seeking industries involving governmental agencies, quasi-governmental agencies, and non-governmental agencies [5]. This paper focuses on the military weapon industry within the broad military industrial supply chain (MISC) to demonstrate how a supplier resilience model can be developed based on the lessons learnt from the COVID-19 disruption.

This paper has three specific research objectives (ROs). First, we explore risk types, mitigations, and models, and we establish a series of resilience-based categories (RO1). Second, the findings from the literature review are supported with multiple interviews of SC experts within the military industry along with a survey of key SC professionals to establish the importance of the categories identified, their impact and severity, along with their current controls and monitoring processes (RO2). Third, we develop an adaptable, empirical risk assessment model with clear risk/resilience indicators, thus providing a comprehensive view of MISC resilience. Furthermore, disruption factors, such as cyber-attacks, pandemics, or natural disasters as examples of extreme events, are incorporated into the risk assessment model elevating it to a more holistic supplier resilience model and scenario planning tool (RO3).

The main contributions of this paper are as follows: (i) an empirically derived supplier resilience model is developed for an MISC which involves governmental agencies, quasi-governmental agencies, and non-governmental agencies; (ii) a number of resilience categories are identified, and their severity, detectability, and occurrence probability are measured quantitatively; (iii) from the gap between severity and detectability, the main areas of the resilience categories that need attention are identified; (iv) supplier-dependent risk priority numbers are obtained from the risk assessment model revealing risks that correspond to where the supplier has least resilience; and (v) the risk assessment model is further extended into a supplier resilience model by incorporating rare disruption factors, which can act as a scenario planning tool under specific types of disruption events (e.g., pandemic, cyber-attacks, natural disaster).

The remainder of this paper is organised as follows. In Section 2, the relevant literature is reviewed with an emphasis on supply chain risk, disruption, and resilience. In Section 3, a research framework is presented to explain how individual research objectives are addressed. In Section 4, we present the data analysis and the main findings. Finally, we draw conclusions and propose further research in Section 5.

2. Literature Review

The literature review is presented by four sub-reviews. Firstly, we identify and classify SC and operational disruptions and risks in a broad context since the relevant literature on the MISC is rather limited. Secondly, we review the literature related to military industrial supply chains. Thirdly, we discuss the risk management strategies to overcome risks and disruptions including the lessons learnt from COVID-19. Fourthly, we review and explain risk modelling techniques and summarise the resilience categories in context.

2.1. Disruption and Risk Types in SC and Operational Resilience

Disruptions in the SC, both upstream and downstream (including inbound logistics and outbound logistics), can have lasting impacts, therefore posing significant threats to business operations if not mitigated [6]. It is reported that 75% of organisations face various disruptions annually [7]. However, many companies only understand their vulnerabilities once an unanticipated disruption has occurred as a consequence of limited initial investment into risk identification [8,9].

Risks can be classified into two broad types: endogenous and exogenous. Endogenous risks are those derived from business operations and trading activities. These have been extensively studied and are directly relevant to SC management and operational resilience [10]. The endogenous risks may be categorised according to their impact on an SC. Many of these risks arise at the SC level; however, it should be recognised that multiple endogenous risks arise from the focal firms’ environment and their interdependencies upstream and downstream [10]. Upstream endogenous risks include inventory, lead times, design complexity, price, and on-time/on-quality measurements [10], whereas downstream endogenous risks need to consider variation in demand, including distortion and disruptive technologies [10]. Endogenous risks are further identified when considering financial dependency [10], process stability and effectiveness [9], information topicality [11], and sustainable procurement [12].

Exogenous risks arise from external uncertainties such as political, regulatory, disaster-based, and foreign exchange rate risks [10]. Aslam et al. [13] recognise recent increases in major disasters, such as tsunamis in Asia and the US, and terror attacks, resulting in a heightened requirement for SC resilience and agility. Separately, the pandemic has resulted in strengthened relationships and partnership-forming to overcome such challenges [14]. Revilla and Jesus Saenz [15] find that the most active SC risk management strategies focus their mitigations endogenously and thus build reliability through efficient processes and the elimination of failure modes. Cuvero et al. [16] evaluate supply chain resilience in the provision of Personal Protective Equipment in the UK and propose a framework on risk management strategies including identifying critical suppliers, locating distribution centres, and quickly restructuring healthcare systems. Golan et al. [17] examine vaccine supply chain resilience and state that applying network science tools such as artificial intelligence and digital twins can balance efficiency and resilience after pandemic disruption. Katsaliaki et al. [18] conduct a literature review to summarise the types of disruptions and their impact on supply chains. For example, according to the frequency of occurrence, SC risks that occur regularly may be grouped as follows: supply risks, process risks, demand risks, intellectual property risks, behavioural risks, and political/social risks [18,19].

Most of the risk and disruption types can be categorised by endogenous factors that influence resilience: material planning, product complexity, quality issues, price escalation, and disruptive technologies [10,15]. However, the interconnectedness of endogenous and exogeneous risks should be considered when assessing risk mitigations, and thus SC resilience.

2.2. Military Industrial Supply Chain and Resilience

The MISC can be defined as a cross-functional system consisting of a collection of commercial and governmental partners involving procuring, producing, and delivering military products such as weapon systems. There are a few unique characteristics that differentiate it from common commercial supply chains. Firstly, the military industry is not solely a profit-seeking enterprise: the focus is to achieve a particular state of readiness contracted by governments [20]. As a result, the associated cost-related culture is often secondary and there is less pressure for cost efficiency. Secondly, the MISC involves both commercial and governmental partners including governmental agencies and quasi-governmental agencies which have different objectives [5]. Thirdly, modern weapon systems are often complex, expensive, and international, which are subject to high uncertainty and have huge economic implications [20].

The literature is limited with regard to revealing SC resilience in the military industry sector. Essig et al. [21] emphasise the importance of implementing safety- and security-related instruments as a resilience strategy to achieve efficiency and effectiveness in German defence supply chains. Sokri [22] presents volume flexibility and delivery flexibility to measure the performance of a military supply chain, where volume flexibility represents the ability to change the level of moved products and the delivery flexibility indicates the ability to meet short lead times. Zhou et al. [23] adopt a simulation method to model military supply networks and analyse the networks’ resilience against different disruptions. Burns [5] highlights the complexity of large-scale systems like the MISC and identifies the need for developing improved approaches to manage risks in supply chains involving commercial and governmental parties. Nazeri et al. [24] consider the supplier selection problem in a military supply chain. They propose a multi-objective programming model to determine the quality, quantity, and the risk factors of the best suppliers and identify the Pareto-optimal solution set. Sani et al. [25] emphasise the new trend to enhance resilience in military supply chains by identifying new and proactive strategies in the logistics planning process.

In the MISC, reduced effectiveness often occurs as a consequence of many opportunities not being addressed when the potential for cost minimisation is at stake, such as not consolidating requirements and scheduling deliveries at the last minute [21,26]. As a result of high-paying export customers and government contracts, the military industry does not have the budget pressures often found elsewhere and therefore does not necessarily have the same pressures for improved efficiencies [27]. Inefficiencies have been shown to reduce a company’s resilience when subjected to a disruption [28]. However, cost minimisation and efficient operations do not always ensure flexibility during a disruption [29]. The preparedness for disruptions is often a low priority leading to an ‘aftershock’, with little strategy and therefore ‘failure imagination’ applied [30]. Having experienced severe disruptions, such as the COVID-19 pandemic, there is a need to establish resilience models for the MISC from an empirical research perspective.

2.3. Risk Management Strategies and Lessons Learnt from COVID-19

In this section, we first review the risk management strategies in the literature and then discuss the lessons learnt from COVID-19. The rationale to choose COVID-19 is that it is an unprecedented example of a disruption event and has exposed many problems in supply chains that have not been paid sufficient attention to before.

The existing literature on risk management methods mainly focuses on risk identification, risk assessment, risk mitigation, and risk monitoring for specific supply chain processes. Baghersad and Zobel [31] propose three metrics (the initial loss, the maximum loss, and the total loss over time) to evaluate the resilience performance of an organisation against disruptions such as a pandemic, earthquakes, flood, and work stoppages. Fan and Stevenson [32] summarise a matrix of risk treatment strategies based on the probability and impact of risk factors including risk acceptance, risk mitigation, risk transfer/sharing, and risk avoidance.

The following are a few examples of the studies related to risk management methods. Zsidisin et al. [33] analyse the supply risk assessment techniques for purchasing organisations including proactively assessing the probability and impact of the supply risks in advance, or reactively discovering the risk after a detrimental event occurs. Paulsson et al. [34] classifies the risk exposure in supply chains into 15 different risk exposure categories with different levels of negative impact. The perceived disruption risks can instigate a recovery plan in line with business continuity management. Sawyerr and Harrison [35] highlight the need for adaptation through agility as a key SC mitigation strategy. Similarly, Yu et al. [36] recognise the requirement for a dynamic capability view (DCV), which enables reconfigurations of existing capabilities to enable a firm to address rapidly changing environments. Aslam et al. [13] build on the need for agility by suggesting a supplier needs to be ambidextrous to become both resilient and efficient, recommending that the triple-A model proposed by [37] should be applied in the following order: adaptability, alignment, agility. Revilla and Jesus Saenz [15] conclude that all SC risk management (SCRM) strategies should be designed proactively with relevant partners. This allows for a continuous loop of learning from past disruptions. Often, internal capabilities and infrastructure need to be aligned to enable an accurate balance of strategies [15]. Raja Santhi et al. [38] point out that the application of Industry 4.0 technologies can increase agility, transparency, and resilience in the supply chain and therefore mitigate disruptions such as pandemics, wars, and natural calamities.

Several lessons have been learnt from COVID-19. Industries globally have seen operational disruption as a result of the COVID-19 pandemic, resulting in an increased urge to plan for future threats [2]. Despite the Iceland volcanic eruption, hurricanes Maria and Harvey, and the Fukushima disaster, most organisations still found themselves unprepared for rare disruption events, which calls for the development of a supply chain resilience assessment tool [39]. Due to this unpreparedness, suppliers located in China enforced 3000 force majeure declarations from December 2019 to March 2020 [40]. However, the extent of the preparation required could not be understood fully, as it has been proposed that a pandemic creates a 30% longer-term disruption than that of other major disruptions, therefore increasing the requirement for a resilient operation within the SC [41]. Another lesson that Beattie [2] acknowledges is the requirement to have topical datasets in order to ensure continued and consistent reviews, as opposed to an SC manager having out-of-date datasets on file which are being unused. Choi et al. [40] found that organisations do not have access to vital information across international teams, resulting in a higher impact of the disruption, due to uncoordinated and reactive responses. Following COVID-19, another lesson suggests that resilience issues are too critical to only lie within the responsibility of one department [2]. Gaps between roles and responsibilities can result in some disciplines not being undertaken, and with many boards of directors now discussing resilience as a priority, it must be ensured that these gaps are closed [2]. This suggests that different functions need to work together as multi-functional teams (MFTs) to ensure aligned resilience strategies are put into place. Derbyshire [42] recognises the decision-maker’s diminished ability to perceive the possibility or consequence of rare events like the COVID-19 pandemic. This indicates the importance of evaluating the detectability of risk categories.

Concluding this section, common mitigation strategies and practices currently implemented by firms include standard lessons learnt, reactive responses, and resource cuts. The literature recommends many alternative mitigations to the standard practices found, which ensure the strategic longevity of the mitigations. These include the triple-A model, routine learning, ensuring topical datasets, longer-term responses, and having additional responsibilities within an organisation to take accountability for resilience.

2.4. Risk Modelling Techniques and Resilience Categories

Apart from understanding risk types, another important component of risk modelling is to understand the criteria and measures necessary to establish organisational and SC resilience. SC resilience is often defined as the ability of the supply chain system to bounce back from a disruption event. Golan et al. [43] propose resilience measures including product depreciation, disruption costs, direct losses/customer retention, and recovery rates. Ivanov [41] affirms the necessity to measure inventory dynamics, customer performance level, financial performance, and lead time performance, whilst considering SC disruption time frames to ensure disruption time can be reduced. Additional and new suggestions from Ivanov [44] consider reserved capacities, lead time reservations, and regional subcontracting.

Jain et al. [45] identify 13 enablers of a resilient SC, and ones not already identified are risk and revenue sharing and technological capability, whilst Beattie [2] acknowledges that models should include aspects of crisis management, business continuity, disaster recovery, pandemic planning, site emergency management, risk management, and vendor risk management. It was concluded that vulnerabilities can be mitigated mainly by trading partners frequently sharing information, which builds trust and visibility [45]. However, Basole et al. [10] indicate that there are limited computational models that allow risks to be visualised for firms. Therefore, disruption-related metrics should be considered in addition to solely performance-based metrics.

Lee et al. [37] highlight the requirement for empirical data collection and predictive quantitative methods when considering SC resilience, and Golan et al. [43] build on this by further suggesting data collected must be weighted to be able to determine trade-offs quantitatively as opposed to qualitatively which is the main focus in the literature. Bier et al. [6] conclude that risk and structured models have a positive correlation when treated equally, thus representing the effects of risks being described through the structure of SC networks. An example of a structured model is failure modes and effects analysis (FMEA), an approach that is already prominently used in the defence supply chain to analyse, assess, and optimise risk factors [21].

Broadly, FMEA has been shown to be a powerful tool for proactive SC risk management and could play a major role in supplier risk assessment and selection [46]. Kumar et al. [47] present a failure modes and effects analysis tool to assess supply chain risks for a focal OEM company. They state that global supply chains need scalable risk management tools to ensure supply continuity when major disruptions occur. Chen and Wu [48] propose a modified FMEA method for new supplier selection to mitigate supply chain risks for an IC assembly company. Aleksic et al. [49] apply the FMEA method to a long supply chain in cheese production. The risk priority numbers are calculated to rank the risk factors. Choudhary et al. [50] state that FMEA has been frequently applied for SC risk assessment but needs to be integrated with other techniques to capture other uncertainties and the interdependency of events. In our study, we apply FMEA to evaluate severity, detectability, and occurrence probabilities for individual resilience categories quantitatively, which consequently provides RPN values for comparison. Disruption factors such as cyber-attacks, pandemics, or natural disasters are not suitable to be directly included in FMEA because they are one-off events and it is difficult to quantify the occurrence probability. We therefore extend the risk assessment model built based on FMEA to a supplier resilience model by incorporating specific disruption factors as an additional layer.

In the FMEA framework, three determinants of occurrence, detection, and severity (ODS) are rated on a Likert scale 0–10 and multiplied together to achieve a risk priority number (RPN). This RPN is combined with specific disruption factors to develop a supplier resilience model in this paper.

Table 1. Criteria of three-stage FMEA model (based on [21]).

Determinant	Criteria
Occurrence	How often does the item occur? Are there any possibilities (in terms of action measures) existing so far which might help to prevent the item form occurring?
Detection	Is it assumed that, in reality, the item does occur even more frequently? Are there any possibilities (in terms of action measures) existing so far which might help to detect the item?
Severity	Is there a realistic possibility to cope with the damage being caused by the item? What is the effort (in terms of financial, material, time, and personnel resources) required to cope with the damage?

summarises the criteria against each of the three determinants which can be applied to resilience categories.

Based on the literature review, we classify the resilience factors into 10 main categories. We then divide each category into several sub-categories, which lead to total 22 sub-categories as shown in

Table 2. Identified resilience categories from literature review.

Category	Sub-Category	Description
Topicality	Up-to-date intelligence	Real-time exchange of important and relevant data on potential threats in order to avoid potential disruptions and their impacts [2].
Cost implications	Cost reduction strategies	Proactive cost reduction strategies in place with supplier, to ensure bottom line savings that impact P&L, or transactional and cost-containment savings [21,26].
	On-cost target achievement of supplier	A required representation of cost expectations from a competitive viewpoint. Considering depreciation, industry spending and disruption [29].
	Spend with supplier	Percentage of spend with supplier to be monitored to understand wider picture impacts and the company’s dependency on supplier [29].
Financial stability	Financial health of supplier	To enable financial resilience, status of the business, impact of sales, previous ‘disruption’ experiences from historic performance, and age of the company to be considered [10,41,44].
Market awareness and dependence	Market share	To consider the impact of alternative customers and their dependency on each customer, assessment of contract cancellations during disruption, or reduction in capabilities [51].
	Geographical location of supplier	Understanding of material group geographical location spread by assessing distance from customer/supplier. Considerations of border restrictions. Considerations of risks associated with each geographical area, particularly with historic disruptions [44].
Critical item management	Complexity of product/process	This concerns adaptability and agility of the supply chain, customer, and product/process to ensure replacements and alternatives can be quickly sought during a disruption [52].
	Identification of potential obsolete materials within bill of materials (BOM)	Anticipation of obsolescence within a current process or product, and ensuring contingencies are in place in advance [9].
	Identifying risk of potential disruptive technologies	The opportunity for innovation that significantly impacts the way a business operates and changes the competitive market in this way [10].
	‘Green’/environmentally friendly material usage	The use of environmentally sustainable materials within a product and process [12].
SC mapping	Supply chain tiers/depth awareness	Understanding the end-to-end supply chain, from raw material to customer and risks associated at each level when any disruption types occur [45].
Resource utilisation	Supplier capacity loading	Understanding supplier total capacity to gain awareness of opportunities to flex and mitigate around bottlenecks [30].
	Business continuity strategy	Comprehensive plans in place to ensure continuity despite many possible forms of disruption [34].
Performance implications	Current supplier performance against ‘On Time’ targets	Measure suppliers against on time delivery, determining root cause for consistent late delivery [53].
	Current supplier performance against ‘on quality’ targets	Measure suppliers against quality delivery, determining root cause for consistent defects and non-conformances [10].
	Level of relationship with supplier	The recognition that different suppliers require different relationship types. Recognition that there is a preference to become increasingly collaborative. Ensuring strategies are aligned [14,45].
Risk culture	Risk management internally	Strategic orientation for developing robustness and openness to change among employees [12].
	Risk management of supplier	Supplier’s strategic orientation for developing robustness and openness to change among employees [10].
Material planning implications	Forward visibility of demand	Forecasts provided to suppliers, and their sub-tiers to mitigate the bullwhip effect of a distorted demand picture [54].
	Buffer or raw material stock holding	The principles of JIT have been quickly negated when faced with disruption, therefore having a need for stock holding [9,30].
	Length of lead time on item	A long lead time can lead to a high risk, particularly when poorly planned [54].

. The main categories were derived by ensuring exogenous risks are factored into an external viewpoint for a later stage, yet including endogenous risks due to the ability to measure and mitigate these [10]. Further risks identified in the literature also require measuring, such as location of supplier, market share, and risk culture; therefore, these have also been incorporated into categories or sub-categories [51]. Table 2 summarises the resilience categories including corresponding use cases, with the intention to measure each category. We use empirical research to confirm and complement the resilience categories.

3. Research Methodology

The research framework can be described in relation to the three research objectives. RO1 is largely addressed through the literature review. The identified resilience categories in Table 2 are validated through a survey and interviews in the context of the military industry supply chain. RO2 is achieved via a questionnaire survey. The respondents score the categories for direct input into an adapted FMEA model in order to determine resilience scoring (weightings). A new supplier resilience model is fully developed to satisfy RO3, applying all identified categories from RO1 and weightings from RO2 to adapt the FMEA and add a disruption factor (DF) using information from the interviews. Disruption factors such as cyber-attacks, pandemics, and natural disasters are rare and extreme events. They are difficult to measure for a specific supply chain. In addition, varying disruptions can impact some resilience categories more adversely. We therefore treat the disruption factor as an additional level in the supplier resilience model, which is useful to enable scenario planning. Suppliers can be assessed in the proposed new FMEA model using secondary data from the case company’s ERP system, Dun and Bradstreet financial reports, and internal risk assessments that highlight value, lead time, and financial health. All qualitative methods are used to collate mitigations per category, ensuring a link to the international aerospace industry is captured. The research framework is illustrated in Figure 1.

The semi-structured interview provides a qualitative perspective enabling meaningful responses to be encouraged to enable new ideas and views to be formed and therefore, for example, understanding varying viewpoints on probable disruption impacts and mitigations in line with RO1 and RO3. Seven interviews are undertaken, to allow the literature to be challenged. The interviews are also used to identify the need for technologies to support resilience and to establish different disruption types. Interviewees were selected on a convenience sample basis, ensuring a range of experiences and roles were selected. The interviewees were identified as relevant due to their strategic nature and having wider awareness of current improvement and ongoing projects; however, the interviews serve differing purposes in line with differing ROs.

To ensure RO3 was considered in line with recognising technologies and their influence on resilience, a couple of transformation-type roles were considered within the interviews to explore technological aspects and how critical they are to this research. Also ensuring role coverage across different stages of the product life cycle were considered when interviewing SC professionals. The interviewees were all based in the UK and concern the roles highlighted in

Table 3. List of selected interviewees.

Role—Title	Reason for Selection
Director of Procurement and Supply Chain	Responsible for overall strategy linked to RO1 and RO3
Head of Supply Chain Management and Business Solutions	Strategist for supply chain of the future and data gathering linked to RO2
Project Lead for Supply Chain of the Future	SC Industry 4.0 linked to RO3
Future Capability Industrial Engineer	Future supply chain linked to RO2
Head of Supply Chain Excellence and Collaboration	SC improvements for tools and processes linked to RO1 and RO3
Lead Supply Chain Manager	Production programme and industry experience linked to RO1 and RO3
Supply Chain Manager	Weekly SC interaction, responsible for supplier improvements for RO3 input

.

The survey provides the means for employees to provide their viewpoints on the sub-categories, therefore ensuring RO2 can be addressed. Furthermore, qualitative data are required to ensure respondents have the ability to provide mitigative suggestions in line with RO3. The survey offers mainly closed questions to enable empirical data to be captured as highlighted as one of the gaps within the literature review [55]. Additionally, open questions demonstrate that the data have not solely been derived from the literature and eight interviews.

Participants from the following departments were selected: Industrial Strategy, SCM, Buying, Demand Management, and SC Strategy. This yields 93 participants. A limitation to the survey is that the design requires employees to respond with factual answers, as it cannot be based solely subjectively in order to establish a viable and quantitative answer. It is worth noting that there are numerous factors that can influence the outcome of interviews and questionnaires. For example, one possible risk is the challenge of completing the survey due to the lengthy and thought-provoking/quantitative style of questions. Moreover, the survey was conducted during the summer. As a result, the summer holiday period and short return period may potentially affect the survey response rate.

Once the data are collected, they are directly inputted into the FMEA model. As the FMEA requires a scale out of 10 for severity, detection, and occurrence, to establish the risk priority number (RPN), the survey responses create a fixed ‘impact/importance’ (severity) score and also a current ‘detection’ score. These two figures are multiplied by a 1–10 response concerning ‘occurrence’ which an SC manager scores in line with the provided criteria per individual supplier. This produces an RPN to allow for focus on the highest risks per sub-category applied to a supplier. Therefore, the company’s ability to detect different types of disruption can be assessed, establishing gaps where resilience needs to be improved. The survey has been developed to ensure the employee’s historical experience of disruption can be calculated and used as valid data to create the model. A suitable method to assess the qualitative data is to use an affinity diagram, as this highlights top-level headings with many sub-headings underneath in the form of a structure around a large set of ideas that may overlap [56]. This ensures all potential resilience categories are identified, which can be transferred into the FMEA model, ready for the quantitative data inputs.

4. Data Analysis and Findings

Four themes are identified from the results by considering whether they are supportive, contradictory, or novel. Explanations are provided in line with the identified themes as shown in

Table 4. The themes of the findings.

Theme	Findings
Supply chain resilience category	A comprehensive list of categories from the literature review and interviews. New categories—cyber, contract, and political—are identified from the survey.
Disruption type impacts on categories	Impacts from COVID-19 disruption, with mitigations and lessons learnt. Verification that the categories cover COVID-19.
Factors that impact supply chain resilience weightings	Weightings of importance/severity and detection.
Resilience model development	Risk assessment model. Supplier resilience model as a scenario planning tool.

.

The analysed literature from Section 2 served as a foundation for resilience mapping and underpinning the risk and resilience assessment in an SC context. Interviews, surveys, company data, and action-based research determine this outcome, therefore enabling genuine disruptive and non-mitigative issues to be analysed in this context. The survey returned 59 respondents, providing a 64% response rate during a 1.5-week period, with the distribution of roles and responsibilities outlined in Figure 2.

The outputs are assigned to a series of themes alongside company data and the eight interviews which were conducted to weight the categories and identify mitigation strategies. This forms a triangulation-based approach as some question types and results are replicated in different methods.

4.1. Supply Chain Resilience Category

Due to the limited literature on the military industry sector, the resilience categories have been identified from the general literature, using specific military industry features for a couple of categories: industry budget implications and market share. Therefore, it is required to confirm each category from the military industry perspective. Resilience categories from the literature review have been consolidated into 10 top-level categories and 22 sub-categories in

Table 5. The 22 identified sub-categories.

Reference	Category and Sub-Category
1	Topicality
1a	1. Up-to-date intelligence
2	Cost Implications
2a	2. Cost reduction strategies
2b	3. On-cost target achievement of supplier
2c	4. Spend with supplier
3	Financial Stability
3a	5. Financial health of supplier
4	Market Awareness and Dependence
4a	6. Market share
4b	7. Geographical location of supplier
5	Critical Item Management
5a	8. Complexity of product/process
5b	9. Identification of potential obsolete materials within BOM
5c	10. Identifying risk of potential disruptive technologies
5d	11. ‘Green’/environmentally friendly material usage
6	Supply Chain Mapping
6a	12. Supply chain tiers/depth awareness
7	Resource Utilisation
7a	13. Supplier capacity loading
7b	14. Business continuity strategy
8	Performance
8a	15. Current supplier performance against ‘On Time’ targets
8b	16. Current supplier performance against ‘On Quality’ targets
8c	17. Level of relationship with supplier
9	Risk Culture
9a	18. Risk management internally
9b	19. Risk management of supplier
10	Material Planning
10a	20. Forward visibility of demand
10b	21. Buffer or raw material stock holding
10c	22. Length of lead time on item

. To facilitate the subsequent analysis, we code these main categories and sub-categories with reference numbers as shown in Table 5. In the survey, these categories were explored to provide an opportunity to add additional categories that had not yet been included in the literature. As a result, three additional categories were added: contract management, political environment, and cyber security. The survey results indicated that contracts require regular amendments to enable commercial viability; political support of industries can have a significant impact on a company’s resilience and a cyber-attack can result in stolen supplier proprietary data. Nevertheless, these three additional categories are not used in developing the risk assessment model, because the weightings against each area have not been collected during the survey, and they are therefore recommended for future research.

4.2. Disruption-Type Impacts

To further validate the 22 sub-category findings from the literature review, their potential to be applied to COVID-19 is assessed via the questionnaire responses. Figure 3 highlights a consolidated view of 25 lesson-learnt themes (only those 3% or more are displayed), which are collated from 111 responses (from 58 respondents) when asked to identify an area of improvement from COVID-19. The original 22 identified resilience categories are cross-referenced to these themes to ensure a more detailed measure can be developed for each criterion within the categories. The criteria enable COVID-19’s impact to be assessed as a form of disruption, providing a clear indication of disruption-type impacts and the prominent issues that need the most improvement.

Ivanov [44] finds that COVID-19 poses a very different form of disruption when compared to previous definitions of disruption. This is due to pandemic-based disruptions occurring over a 30% longer time frame, requiring a resilient, sustainable SC to enable longer-term success and survivability, hence requiring a single-point-of-failures strategy to ensure bottlenecks are mitigated.

Furthermore, changes in demand are deemed to be a significant risk during any disruption type, not solely a pandemic, resulting in agility and full visibility required to meet these changes [57]. Six percent of respondents support this finding, as they highlight ‘forecast and accuracy’ to be a critical factor following observations of COVID-19’s impacts on market share. COVID-19 has exposed the military industry considerably, resulting in a requirement for customers to increase visibility of total demand to flow down to their supply chain.

Similarly, another COVID-19 lesson learnt from the literature finds that resilience is too critical to only have the responsibility of one department [2]. ‘Essential activities only’ being undertaken represents a short-term focused environment which 3% of employees highlighted. This represents a general lack of risk mitigation culture and is considered to be a contravention of Aerospace Standard 9100 process risk-based thinking requirements [58]. MISC resilience requires a risk mitigation culture, and a requirement to meet specific customer standards such as Aerospace Standard 9100, which is currently found as a significant cultural gap due to short-term priorities.

All lessons learnt and major impact areas for COVID-19 are confirmed to be covered in the 10 identified top-level categories and 22 identified sub-categories in Table 5.

4.3. Factors That Impact SC Resilience Weightings of Importance and Detection

The adapted FMEA model requires 1–10 scoring. However, during the survey pilot, feedback received regarded the 10-point scale to be overly granular. Therefore, a 7-point scale was used in the survey. Hence, the scores need to be converted into a percentage to provide an input of 1–10 into the FMEA model. Figure 4 compares respondents’ importance ratings against detection ability and highlights the standard deviation for each representing a sample of 58 respondents.

The average ‘importance’ standard deviation is at 20% and the average ‘detection’ standard deviation is 21%. As this is similar in both factors, this demonstrates the reliability and consistency of the respondents’ answers. The top four highest importance sub-categories areas are financial health, topical data, business continuity planning, and SC mapping.

However, a limitation is recognised when comparing the ratings of importance versus detection, due to the survey scales recognising all importance areas as important and none as unimportant, meaning ‘not as important’ would be 0%. Calculating the difference between two ratings (importance rating and detection ability) is useful to determine the improvements required per category, some of which may be minimal; however, some categories may pose significant gaps. If a gap between two ratings is present, resilience cannot exist due to the ability to detect or obtain not being as high as required. The aim should be to have equal scores per sub-category [45]. The category scores have been analysed in terms of the largest gaps between the importance and the detection average scores, as shown in Figure 5 in descending order to highlight the areas that need the most improvement, relative to the importance rating.

4.3.1. Topicality

Topicality has a single sub-category, up-to-date intelligence, which represents the need for not only data accuracy, but also for relevant data to be readily available to assess current trends continuously. This category was found to be the second-most important at 88%, with a detection ability at only 60% resulting in the second highest gap (28%) between detection and importance of all 22 sub-categories. This supports the research by Beattie [2], which suggests that continued and consistent reviews are required to achieve topical datasets and more topical relationships. Additionally, earlier in the literature review, Choi et al. [40] recommended a higher focus on topical datasets for international suppliers. As a result, virtual engagements would further provide a benefit via a formal communication plan. A system or model needs to be developed to ensure the data gathered can be interpreted as useful information.

4.3.2. Cost Implications

The ability to identify cost reduction is the third lowest ranked at 52%, yet it is also the second lowest ranked in importance also at 52%. This result indicates the low importance attributed to cost reduction strategies by procurement and SC employees in the military industry. This confirms the findings in [21,26], which highlight the lack of cost effectiveness of the military industry with many inefficiencies not being addressed, due to the limited pressures of budget cuts in comparison to other industries. However, severe and persistent disruptions like a pandemic may lead to continual cost increases. One mitigation strategy is to set contractual milestones and price points, which are agreed upfront, therefore resulting in minimal price increases throughout a contract.

4.3.3. Financial Stability

A financial health assessment of a supplier is ranked highest in terms of importance at 89%, with detection ability at 68%, meaning a considerable improvement in detection rating ought to be implemented. To further close the gap of 21% on financial stability, the literature recommends assessments of product depreciation and specific current disruption costs associated and direct costs of loss [43]. This will add further understanding of impacts and measurement points, both in advance of and during a disruption.

4.3.4. Market Awareness and Dependence

Similarly, market share also requires assessment through third party data, being considered of an importance at 71%, particularly when considering examples such as Airbus’s reduction in orders due to COVID-19. Secondly, the geographical location of supplier is a simple factor to map out, therefore being the only sub-category that scored higher in detection ability than importance. The respondents of this research suggest an alternative source should always be available in a closer proximity, geographically. The significance of geographical location was stated as important during the interviews, due to border restrictions during COVID-19 and anticipated changes due to Brexit. Another respondent stated that all raw materials should be from a UK source. Nevertheless, due to the importance value being 61%, this is not a priority strategy to be worked on.

However, following interviews it is recommended that organisations are now considering reshoring and selecting UK-based suppliers due to greater visibility and control, which is vital during times of disruption. This can further improve detection abilities when considering supplier infrastructure and structure arrangements, for example, working-level employee location and availability. Nevertheless, local sourcing and options for sourcing are not always entirely available for the military industry due to the complexity of products sourced.

4.3.5. Critical Item Management

Similar considerations of single points of failure can be applied to critical products and processes, of which featured importance scores of 71–79 feature the sub-categories of disruptive technologies, product and process complexity, and obsolescence.

In the literature, Gardner and Colwill [52] find that the more complex the product, the more ownership of the design the supplier has. This supports the requirement for alternative and mitigative options against sole-source items that fit into these sub-categories. Complexities such as functions, parts, software, processes, and characteristics have a significant effect on product realisation and require specific actions to ensure they are adequately managed. However, contradicting the literature, ‘green’ supply is the least important out of all categories at 49%, with an even lower detection ability at 38%, showing that views on ‘gresilient’ practices need to improve to become fully resilient during current times. Similarly, no qualitative data highlighted a respondent’s requirement for green practices to be measured and implemented. As a result, a comprehensive ethical health check is suggested to ensure sustainable priorities and strategies are assessed with suppliers, whilst reviewing the credibility of material usage within the design.

4.3.6. Supply Chain Mapping

Lower supply chain tier/depth awareness has the largest gap of all categories between detection and importance, at 29%. Due to an 85% level of importance, improvements are required to map and assess sub-tier levels. This is also a requirement as a part of the AS9100 to ensure product realisation through a sub-tier management plan, suggested as a key element. Furthermore, interview respondents highlighted that sub-tier mapping requires information from a variety of the categories, including lower-level capacity management and forward visibility of demand. This supports Yoo-chul [59], where sub-tier implications were seen as a result of changes in capacity to adhere to COVID-19 social distancing rules, meaning the total SC needs to be considered collectively.

4.3.7. Resource Utilisation

Effective, planned, and comprehensive resource utilisation is found to be very important, scoring 83% and 85% for capacity loading and business continuity strategy. Detection gaps are scored as 16% and 26%, highlighting a greater focus required on business continuity management (BCM) to ensure the effective use of resources to target single points of failure. BCM had a large gap of 26%, representing a requirement for a more robust BCM plan, recognising different disruption-specific risks and therefore impacts on their workforce. For example, a respondent considered the impact of shielding and the split of skills as needing to be planned for and managed through lessons learnt.

4.3.8. Performance

On-time (OT) and on-quality (OQ) measurements of supplier performance are considered to be low-risk, due to high importance and only a 0.06 gap on average when comparing to detection ability. A major theme of these categories was to ensure an agile response is in place when problems occur and the KPI declines as a result. Basole et al. [10] find that in disruptive circumstances, quality measurements can decline considerably due to other priorities. Therefore, with an importance of 83% for OQ and 79% for OT, it must be ensured that a fast response is in place to ensure agility, adaptability, and alignment, therefore ensuring ambidexterity as supported by Aslam et al. [13]. When considering the relationship level with a supplier, this was found at 84% importance, with an 11% gap for detection. The relationship level can be used as a performance indicator to assess maturity, which is necessary considering most qualitative suggestions and comments surrounding relationships with the supplier. Sapics [14] further identified that relationships strengthened during COVID-19. Similarly, the respondents from our survey further highlighted the importance of developing supplier relationships into partnerships to ensure challenges during disruption can be faced.

4.3.9. Culture of Risk Management

Gaps between importance and detection for risk mitigation, both internally and with the supplier, are found in the top eight categories out of twenty-two. Whilst linked to communication and relationship strategies, qualitative responses have highlighted the requirements for clear definitions between roles and responsibilities to be considered when mitigating risk as too many roles report on the same information. This ensures AS9100 risk-based thinking requirements are addressed and acted upon proactively [58]. However, this contradicts Beattie [2], as it is found that more responsibilities for risk do result in an improved mitigative culture in the business. Therefore, a benefit is identified in creating a multi-functional team, which is empowered to manage all aspects of the SC with a joint goal and back-up plans as to how quickly a supplier can be replaced.

4.3.10. Material Planning

Forward visibility of demand has the fifth highest gap in the twenty-two categories, whilst showing a significant importance of 81%. There is a requirement to mitigate the ripple effect further down the SC in line with Ivanov’s [44] findings. This supports Essig et al.’s [21] goal to ensure procurement-based employees understand demand planning impacts upstream and downstream.

The respondents from our interviews highlight that buffer stock has provided a surprise to the military industry during disruption. Normally lean principles are calculated to follow just in time principles when taking the length of lead time into account. Buffer stock was rated at 74% importance with a 10% gap of detection due to the ability of utilising SAP stock reports. However, to ensure better planning, it has been identified that there are limited roles that plan the demand with supplier interfaces to provide this information. Therefore, a responsibility is required to understand supplier capability to allow for the early identification of risk within the end-to-end SC. This will correlate well with forward visibility and lead time understandings, to ensure aligned buffer stock requirements to the master data on MRP. Furthermore, OT/OQ can be used in conjunction with demand planning to ensure failures can be predicted or mitigated in advance.

4.4. Risk Assessment Model Development

The feedback from interviews and the supplier scorecards were considered and applied to develop an FMEA risk assessment model. Specifically, ‘importance/severity’ and ‘detection’ have been inputted as static elements of FMEA, whereas the ‘occurrence’ factor of the FMEA model is applied to individual suppliers.

‘Detection’ and ‘importance/severity’ ratings from the survey are adapted into detection scoring in a 1–10 Likert scale for FMEA and can be seen in Figure 6, where the colour indicates the scale with higher the number the more red.

As opposed to ‘severity’ and ‘detection’ fixed scores, the ‘occurrence’ factor of the FMEA model is applied to individual suppliers by an MFT to establish resilience. This represents a score that will depend on the context of the supplier against each sub-category, where five suppliers are piloted. It is necessary that occurrence represents the likelihood that a failure may occur, therefore applying current situations via the 1–10-point criteria has been designed through survey, interview, and action research data. Figure 7 highlights the categories and scoring per supplier, whilst Figure 8 shows the criteria used by the MFT for each supplier, where the colour indicates the scale with higher the number the more red. The use cases for the criteria can be viewed in Table 2. These include recommendations and mitigations per category, which have been derived from all qualitative findings and have been interpreted and placed into the categories by the researcher. A respondent from our interview highlighted the significant benefit of using human judgement in a resilience-based model, as intellect needs to be applied to ensure consistent measuring across subjective interpretations of the criteria.

Once inputted, a score of 1–10 will be created on a top-level spreadsheet for occurrence and then multiplied by the static ‘importance/severity’ and ‘detection’ scores, creating an adapted version of the RPN. This allows an RPN to be calculated for each individual sub-category in Figure 9 (in which Supplier 1 is chosen as an example), highlighting in red where top priority focus is required to ensure a reduction in risk. An average resilience score has been calculated to determine where suppliers have least resilience, which is currently shown as ‘topical information’ and secondly ‘business continuity strategy’.

4.5. Supplier Resilience Model

By incorporating a specific DF into the risk assessment model, we obtain the supplier resilience model in Figure 9, which represents the multiplier that the specific disruption brings, including cyber-attacks, pandemics, or natural disasters. The multiplier is determined from interviews and company data, as an example deducing that a lack of topical data would reduce the risk score if a cyber-attack occurred, allocating a 0.8 multiplier. The final score of each resilience category can be calculated by multiplying the RPN with the multiplier of the DF. This enables the managers to prepare for scenario planning under a specific type of disruption events (e.g., pandemic, cyber-attack, natural disaster). For example, a pandemic is not impacted by the use of ‘green’ materials, so the multiplier remains a 1, but ‘topical data’ is the highest at 1.4 due to the impact the lack of topical data would cause. The scale of the multipliers has been developed with interviewees’ viewpoints on disruption types and the literature, resulting in multipliers of 0.8, 1, 1.2, and 1.4.

To enable sampling against the model, Supplier 1 has been selected with the aim to investigate high risk, and it requires a deep dive/root cause analysis to be undertaken to ensure resilience. The results are shown in Figure 9. Topical data, SC awareness, and business continuity score risk priority numbers of 288. These areas are also proven through company data and action research, highlighting that this supplier has older and non-existent processes, no supplier audits, and requires succession planning for business continuity. Therefore, the high risks found in the model verify the data from the customer.

Once the multipliers are selected for ‘pandemic’, this results in SC awareness to increase to 403 and market share to be the second highest risk at 353. This multiplier highlights areas of focus during times of disruption for an SCM, as these areas will require further root cause analysis to be undertaken, to ensure the risk can be reduced and mitigated.

5. Conclusions

This paper presents the development of a risk assessment model and a supplier resilience model for an MISC. We firstly establish the resilience-based categories by identifying the risk types and topics from the literature and incorporating the lessons learnt from COVID-19. In total, 10 resilience categories and 22 sub-categories are identified. Multiple interviews, company data, and a questionnaire survey confirm the identified categories to improve SC resilience, and three additional categories are identified from the military industry.

Twenty-two sub-categories from the literature review are fully analysed using the data collected from the MISC. Based on the survey, we quantify the importance (severity) of the identified categories, their detectability, and their occurrence probability. This resulted in the sub-categories being importance-weighted in which 85% of the findings against sub-categories support the literature when considering the importance level, with sustainability priorities and cost implications being considered as less important than found in the literature. Significantly, SCMs and strategy-based roles consider cost to be of less importance than other procurement-based roles. Demand planning and buying roles consider cost of higher importance, which is found to be attributed to their manufacturing stakeholder links highlighting the importance of end-user and, therefore, cost-reduction strategies. The main areas that require focus are topical data, supply chain tier/depth awareness, comprehensive business continuity management, and internal risk management, in line with recommendations from [57].

Noting the one-off characteristic of rare disruption events, we treat the disruption factor as an additional layer to be combined with the risk assessment model obtained from FMEA, which yields a supplier resilience model. This model can be used as a scenario planning tool for extreme disruption events such as a pandemic, natural disaster, cyber-attack, and industry action. The model adaption offers a familiarity to users, due to FMEAs being used within MISCs, with occurrence, detection, and severity (ODS) undertaken against the SC instead of a product or process. The consolidation of data can be considered as a significant contribution, as the depiction of all SCM areas have been rarely consolidated and prioritised in such a form. The proposed model ensures all categories can be analysed by both importance and detection, ensuring the wider picture is quantitatively visible for procurement experts in MISCs and more general supply chains.

This study has some limitations. For example, it focuses on one military industry case study, which may narrow its general applicability. Another limitation is the focus on COVID-19 from respondents due to its topicality and relevance at the time, as opposed to general disruption types, and this may have influenced the responses during the research.

During the research, three new categories of cyber security, political influence, and contracts were identified. They need to be incorporated to establish the full view of SC resilience impacts in any further research. Further research could extend the supplier resilience model to create a flag system to highlight sub-categories’ mean scores and to remove the FMEA threshold application. For example, when a supplier has a non-high flagged RPN but is still considerably above the mean of its category, it would illustrate that a major improvement is required.