Towards a Framework of Operational-Risk Assessment for a Maritime Autonomous Surface Ship

: Global research interest in the domain of maritime autonomous surface ships (MASS) is dramatically increasing. With new prototypes planned to be set to the seas where various operational modes (OMs) are claimed, the issue of the safety evaluation of an MASS, and criteria for selecting the appropriate OM for given conditions remain open questions. This paper proposes a four-step risk-informed framework to assess risk in a scenario for an MASS operating at one of three OMs: manual control (MC), remote control (RC), and autonomous control (AC). To this end, the concept of risk priority numbers (RPNs), adopted from failure mode and effects analysis (FMEA), is utilized. The required parameters to deﬁned RPNs are obtained in the course of analyzing a model MASS accident with expert knowledge. The applicability of the proposed framework is demonstrated via a model MASS case study. Results reveal that, in the same scenario, the risk of MASS varied across the analyzed OMs. On the basis of the aggregated results for each operational mode, suggestions for OM switching are put forward.


Introduction
Autonomous ships were termed as maritime autonomous surface ships (MASS) with four types of degree of autonomy (DoA) by the Maritime Safety Committee (MSC) of International Maritime Organization on its 99th session. An MASS could be operating at one or more DoA for the duration of a single voyage [1]. That is, a different DoA indicates different operational modes. For example, an MASS and its remote-control center synchronously go through a number of different operational modes following different DoA.
In the literature, operational modes (OMs) for a MASS are categorized into four types, namely, manual control, autonomous control, remote control, and fail to safe [2][3][4][5][6][7][8]. First, manual control represents the mode where the ship is handled by the crew on board, as is, e.g., the case during berthing and approaching port, passing navigationally demanding waters, and dealing with emergency situations [3,9]. Second, autonomous control indicates that the ship handles itself with an advanced control system onboard, e.g., an autonomous navigation system [6]. In this mode, there may be two submodes, autonomous execution, and autonomous control or autonomous problem solving [3,10]. In the former submode, the ship follows a predefined track, and fully automatically performs navigational and lookout tasks without more advanced reasoning and decision making or guidance from the shore, but with periodic and brief status reports sent to the shore operators [7,10]. In the latter submode, the ship may make intelligent decisions on its own, e.g., on the basis of Energies 2021, 14, 3879 2 of 12 calculated algorithms or preprogrammed behavior patterns [11]. Third, remote control means that the ship is taken over by a remote-control center, e.g., a shore control center (SCC). In this mode, remote control can be divided into three submodes, namely, direct control, indirect control, and situation handling [10,12]. Lastly, fail-to-safe mode is one of the main safety principles (see [13,14]), similar to the minimal risk conditions (MRCs) proposed by the DNV [15]. The specific fail-to-safe mode depends on what problem the ship encounters and other environmental or ship parameters [3]. Porathe [4] roughly provided three submodes, i.e., drifting, anchored, and station keeping, while DNV [15] listed 10 potential MRCs. Three conventional operational modes are the interest of this study, as shown in Table 1.

Manual control (MC)
The MASS is handled by onboard crew. This is similar with that of conventional ships. [3,9] Ship with automated processes and decision support Remote control (RC) The MASS is controlled by an operator in a remote-control center, e.g., shore-based control center. In this mode, an operator can directly control, indirectly control, and remotely handle a situation. [10,12] Remotely controlled ship without seafarers on board Autonomous control (AC) The MASS controls itself, e.g., with the autonomous navigation system onboard. In this mode, the MASS autonomously controls its behavior or solves problems if any exist. [3,10] Fully autonomous ship This paper focuses on the risk assessment of potential OMs to change that may provide risk-based support for switching OM. In other words, how to switch among the three operational modes of manual control, remote control without crew onboard, and autonomous control to reduce risk. However, few studies were conducted on providing support for changing OMs. To our best knowledge, there are no standard and criteria to measure risk level for MASS. Hence, to determine the complete risk level for MASS with different OMs is very challenging [8]. To fill this gap, we propose a four-step framework to quantify such risks in these three OMs on the basis of traditional failure mode and effects analysis (FMEA). Failure modes (FMs) in the case study were derived from model MASS tests using the 24 Model for accident analysis.
The rest of this paper is organized as follows. Section 2 introduces the FMEA and 24 Model. Section 3 illustrates the framework for risk assessment of MASS in a given operational mode. Section 4 presents a case study. Contributions and limitations of this study are discussed in Section 5. Section 6 concludes the paper.

FMEA
FMEA was first developed as a formal design methodology in the 1960s by the aerospace industry with obvious reliability and safety requirements [16]. In the past several decades, FMEA was applied for risk assessment on manufacturing processes [16], electronic systems [17,18], the design of aircraft engines [19], and the propulsion systems of ships [20,21]. Risk assessment in FMEA conventionally develops an RPN, which is obtained by finding the multiplication of the probability, the severity, and the probability of not detecting failure mode. For obtaining the RPN of a potential failure mode, the three  [16,22,23]. Similarly, the integer numbers, 1 to 10, as ratings are designed for this study, described in Tables 2-4.

10
Failure onboard or onshore is hazardous and occurs without warning. It suspends the operation of the system and/or involves noncompliance with international or national regulations.

9
Failure onboard or onshore involves hazardous outcomes and/or noncompliance with international or national regulations or standards.
8 Onboard or onshore system is inoperable with loss of primary function.

7
Performance of onboard or onshore system is severely affected, but still functions. The onboard or onshore system may not operate.

6
Performance of the onboard or onshore system is degraded. Comfort or convince functions may not operate.

5
Moderate effect on performance of onboard or onshore system. Onboard or onshore system requires repair.

4
Small effect on performance of onboard or onshore system. The system does not require repair.

3
Minor effect on the performance of onboard or onshore subsystem or system.
2 Very minor effect on the performance of onboard or onshore subsystem or system.

1
No effect. Table 4. Ratings for the detection of a failure mode for a MASS system.

Rating Description for Likelihood of Detection (D)
10 Onboard or onshore subsystem or system does not detect a potential cause of the failure or subsequent failure mode, or there is no system or subsystem for such detection. 9 Very remote chance that onboard or onshore subsystem or system detects a potential cause of failure or subsequent failure mode.

Rating Description for Likelihood of Detection (D)
8 Remote chance that onboard or onshore subsystem or system detects a potential cause of failure or subsequent failure mode. 7 Very low chance that onboard or onshore subsystem or system detects a potential cause of failure or subsequent failure mode.
6 Low chance that onboard or onshore subsystem or system detects a potential cause of failure or subsequent failure mode.

5
Moderate chance that onboard or onshore subsystem or system detects a potential cause of failure or subsequent failure mode. 4 Moderately high chance that onboard or onshore subsystem or system detects a potential cause of failure or subsequent failure mode.
3 High chance that onboard or onshore subsystem or system detects a potential cause of failure or subsequent failure mode.
2 Very high chance that onboard or onshore subsystem or system detects a potential cause of failure or subsequent failure mode.
1 Onboard or onshore subsystem or system almost certainly detects a potential cause of failure or subsequent failure mode.

24 Model
The 24 Model is a linear and systematic accident-causation model [24,25]. This method analyzes an accident from two levels, i.e., individual and organizational, and from four causes, i.e., immediate cause, indirect reasons, radical cause, and root cause. Additionally, it accounts for the effect of external factors. The 24 Model is also applied in rail transportation [26] and the maritime industry [27].
Given an accident, we adopted the 24 Model to analyze the immediate cause(s) and external factors of the accident, which are identified as failure modes for developing a scenario. In this context, we assumed that a scenario was caused by one or mutual FMs that were unsafe or by unsafe conditions related to the onboard side, the onshore side, or external environmental factors.

Framework
The proposed framework consists of four steps:

Step 1: Identify Potential Failure Modes
We used the accident-analysis approach for the 24 Model to identify potential failure modes related to MASS on the basis of historical data, e.g., accident reports.

Step 2: Evaluate Three RPN Parameters of FM in Given Operational Mode
An FM's three RPN parameters, i.e., O, S, and D in a given operational mode are evaluated by expert judgements using the crisp numbers in Tables 2-4. For all identified FMs, such evaluations of expert p construct evaluating matrices O p , S p , and D p , respectively, as follows: where i is the number of FM, i = {1, · · · , m}, j is the number of OM, i = {1, · · · , n}. Given k experts involved in the evaluation, we have k evaluating matrices for each parameter. Then, we constructed average evaluating matrix O, S, and D as follows: The averaging results are inputs to calculate the RPNs of each FM in the three OMs, as follows: where RPN q is the RPN of FM q. Subscript q is the number of FMs. Subscript j is the number of the operational mode, i.e., j = 1 for manual control, j = 2 for remote control, j = 3 for autonomous control. O qj is the probability of FM q in operational mode j, S qj is the severity of FM q in operational mode j, and D qj is the probability of not detecting FM q in operational mode j.

Step 3: Calculate RPN of a Scenario in a Given Operation Mode
To calculate the RPN of a scenario in a specific OM, a formula is proposed as follows.
where t is the total number of FMs in the scenario. RPN j is the RPN of a scenario in the operational mode j.

Step 4: Analyze Results and Provide Suggestions
From Step 3, the results could be further analyzed to provide feasible solutions for decision or policy makers. To this end, we used the number of FMs in the scenario to normalize the RPN of the scenario; the resulting sRPN is shown in Equation (9). We also set sRPN ranges to provide corresponding suggestions on switching OM as shown in Table 5.

Case Study
In this case study, we present how to apply the proposed framework in Section 2 to assess a MASS's operational risk in a given scenario. Results are expected to guide decision or policy makers in selecting the appropriate OM or taking measures to reduce risk in the current OM. The scenario was derived from a model MASS test using the 24 Model. In order to evaluate the three RPN parameters of FMs with respect to the three considered OMs, nine seafarers were interviewed.

Step 1: Identify Potential Failure Modes
A model MASS has been trialed in Qinhuai River, Nanjing since October 2019. Figure 1 shows a potential accident of this model MASS trail in Qinhuai River. For more details on this model ship, please refer to [28,29]. During its voyages, a ship-bank allision accident was recorded in the OM of AC (see Figure 2).

Case Study
In this case study, we present how to apply the proposed framework in Section 2 to assess a MASS's operational risk in a given scenario. Results are expected to guide decision or policy makers in selecting the appropriate OM or taking measures to reduce risk in the current OM. The scenario was derived from a model MASS test using the 24 Model. In order to evaluate the three RPN parameters of FMs with respect to the three considered OMs, nine seafarers were interviewed.

Step 1: Identify Potential Failure Modes
A model MASS has been trialed in Qinhuai River, Nanjing since October 2019. Figure  1 shows a potential accident of this model MASS trail in Qinhuai River. For more details on this model ship, please refer to [28,29]. During its voyages, a ship-bank allision accident was recorded in the OM of AC (see Figure 2).       On the basis of the accident recorded in this trial, we applied the 24 Model to identify FMs. The identified accident paths and elements are shown in Figure 3, in which A1, A2, A3, B1, and B2 are immediate causes, C1 is an indirect reason, and F1 and F2 are external factors. However, from the accident, there were neither radical (safety-management system), nor root (safety culture) causes. On the basis of the accident recorded in this trial, we applied the 24 Model to identify FMs. The identified accident paths and elements are shown in Figure 3, in which A1, A2, A3, B1, and B2 are immediate causes, C1 is an indirect reason, and F1 and F2 are external factors. However, from the accident, there were neither radical (safety-management system), nor root (safety culture) causes. On the basis of the above accident analysis, five FMs were identified: 1 , GPS information loss; 2 , improper assessment of ship position; 3 , deviation in course; 4 , improper sensing; 5 , negligence of watchkeeping.

Step 2: Evaluate Three RPN Parameters of FM in Given Operational Mode
The three RPN parameters of the five FMs identified in Step 1 with respect to the three OMs were evaluated by nine seafarers, whose profiles are shown in Table 6. In the MUNIN project, Kretschmann et al. [30] assumed that the employment plan for a shore control center involves 5 to 8 people for a remotely controlled MASS in one shift, in which 5 people for 24/7 operation and 3 people for one shift operation are needed. Hence, the number of seafarers was assumed to be reasonable to judge OM switching for a MASS. The evaluations on the three RPN parameters of the five FMs were averaged using Equations (4)- (6), and results are presented in Table 7. According to Equation (7), Table 8 shows their RPNs in the three OMs.

Step 2: Evaluate Three RPN Parameters of FM in Given Operational Mode
The three RPN parameters of the five FMs identified in Step 1 with respect to the three OMs were evaluated by nine seafarers, whose profiles are shown in Table 6. In the MUNIN project, Kretschmann et al. [30] assumed that the employment plan for a shore control center involves 5 to 8 people for a remotely controlled MASS in one shift, in which 5 people for 24/7 operation and 3 people for one shift operation are needed. Hence, the number of seafarers was assumed to be reasonable to judge OM switching for a MASS. The evaluations on the three RPN parameters of the five FMs were averaged using Equations (4)- (6), and results are presented in Table 7. According to Equation (7), Table 8 shows their RPNs in the three OMs.

Step 4: Result Analysis and Suggestions
RPNs in Figure 4 show that, under a certain scenario, an MASS is expected to encounter different levels of risk when operating under three considered OMs. On the basis of the sRPN results, suggestions are presented.
On the one hand, accounting for FMs, e.g., FM 1 -GPS information loss, and FM 4 -improper sensing, changing from AC to RC immediately is highly unlikely. FM 1 means that the communication link via GPS satellites is cut off. Even if the communication is acceptable, FM 4 indicates that the information of the perception sent from the onboard side to the onshore side may be a challenge to generate sufficient situation awareness. Maintaining such situational awareness may be a challenge if communication quality is unsatisfactory, which, in turn, increases the operational risk in RC. This means that changing from AC to RC may not reduce risk, but also bring new risk.
On the other hand, although there is no nearby crew who can embark on the model MASS in a timely manner, some onboard experimenters can immediately take over the ship and quickly handle the emergency. Hence, changing from AC to MC seems possible for the model MASS. In the scenario, the current OM is AC. Figure 4 shows that the sRPN in AC is 144.23, which belongs to Range 2. According to Table 5, if changing OM is possible, the onboard side should switch to an OM in which the sRPN is less than Range 2. However, values of sRPN in the two other OMs were also in the corresponding Range 2, i.e., 156.88 ∈ (125, 512], 151.24 ∈ (64, 343]. This means that, even though changing from AC to MC or RC is possible, the sRPN would still be within Range 2. On the basis of the above analysis, changing from AC to RC or MC was unavailable. In this context, according to Table 5, the model MASS should control the involved FM(s) by reducing occurrence or severity, or improving detection capability in the current OM, i.e., AC. For example, when FM 1 -GPS information loss occurred, the model MASS should have used input from another satellite, e.g., the BeiDou Navigation Satellite System, to obtain the ship position. When FM 4 -improper sensing happened, the model MASS could have automatically adjusted the gain in radar to obtain proper quality in perception. These emergency responses may be feasible for onboard crew who could take measures in a timely manner to reduce or eliminate the consequence(s) of these FMs, which is the disadvantage of MASS in AC. Therefore, the proper postaccident behavior of MASS within AC is vital to overall safety (Wróbel et al. [31]).

Study Contributions
This study contributes in the following three aspects. First, the proposed four-step framework may not only be helpful for a MASS system to judge OM switching to reduce operational risk, but also be used as a demonstration to guide or train future operators in SCC to evaluate operational risk with respect to the three considered OMs.
Second, in MASS, there are no criteria based on operational risk in terms of RPN to generate corresponding suggestions for risk mitigation. Although any elaboration on MASS safety levels are inevitably incomplete due to the current lack of historical or empirical data (Wróbel et al. [32]), it is essential to develop risk-acceptance criteria, especially for MASS (Utne et al. [33]). In this study, the values of the criteria for evaluating the RPN of a scenario were different in terms of different OMs. These different ranges for the same suggestion in three OMs indicate that the AC criteria are the strictest, followed by those for RC and MC, which may provide insight for developing future risk-acceptance criteria for MASS.
Third, mitigation measures were proposed according to the sRPN range in each OM. If the obtained sRPN is located in Range 1, we suggest to keep the OM but pay attention on the increased risk. If the obtained sRPN is located in Range 2, we suggest to switch to an OM in which the sRPN is lower than Range 2. If the obtained sRPN is located in Range 3, we suggest to switch to an OM in which the DoA is less than the current DoA. However, if changing OM is not possible due to, e.g., communication delay, technical impossibility, time limitation, or because it is not necessary, measures may be taken to mitigate or even eliminate some or all involved FMs to reduce the RPN or risk in the current OM. In this sense, two directions to reduce the RPN of the involved FM are also given. One way is to reduce the occurrence and/or control the severity of the involved FM, and another is to improve the capability of detection of the involved FM.

Study Limitations
This study has two main limitations of the study. The first is the shortcoming of evaluating three RPN parameters of FMs. The evaluation of O, D, and S of the FM in a given OM relies on expert elicitation, which may bring about interpersonal and intrapersonal uncertainty. In the future, rather than fully relying on subjective data, the three RPN parameters of some FMs, e.g., the occurrence of FM 1 (GPS information loss) and FM 4 (improper sensing) could be objectively obtained.
The second is the deficiency in the suggestions for switching OM. There are three main reasons for this deficiency: first, in the context of autonomous shipping, there exists no standard or regulation to determine whether the operational risk in terms of RPN is acceptable. If the RPNs in the three considered OMs are acceptable, it is not necessary to change OM even if there are differences among them. Drawing up and carrying out such regulations internationally rely on the International Maritime Organization. On the other hand, if their RPNs are unacceptable, it could consider other models, e.g., fail to safe. Second, it does not consider factors affecting such changes, e.g., communication, time, crew, and cost (Ewelina and Montewka [34]). If communication between the ship and SCC fails, it is hard to imagine that remote control can successfully or immediately take over the ship. Third, even though one OM is determined to change, this does not consider whether such change brings new risks or not. When changing from AC into RC, does the operator onshore quickly and correctly become acquainted with the situation? Does this process result in a greater risk of cyberattacks? These are still open issues.

Conclusions
This paper proposed a four-step framework to quantify the operational risk of a maritime autonomous surface ship (MASS) in three operational modes (OMs), i.e., manual., remote, and autonomous control. In this framework, FMs were identified from accident analysis using the 24 Model. Then, three RPN parameters of the identified FMs were evaluated with respect to three considered OMs by experts using crisp values. The average of the experts' judgement was generated as input for calculating RPN of FM in a given OM. The RPN for the scenario in the given OM was calculated by adding the RPN of the involved FMs. That is, the RPN for a scenario in an OM was the summation of the RPN of FMs involved in that OM. On the basis of RPNs in three OMs, we provided some suggestions on OM switching.
In the case study, five failure modes (FMs) were defined using 24 Model on a model MASS accident: GPS information loss, improper assessment of ship position, deviation in course, improper sensing, and negligence of watchkeeping. These five FMs as potential FMs for MASS accident scenario were further analyzed following the proposed framework. Results showed that switching to manual or remote control is not useful to reduce risk. Identified FM(s) should be controlled by reducing occurrence or severity, or detection in the current OM should be improved, i.e., autonomous control. Nevertheless, for MASS with onboard crew, once the two failure modes of GPS information loss and improper sensing occurred, changing to manual control would reduce such a risk in a timely manner.