Using Operational Scenarios in a Virtual Reality Enhanced Design Process

Abstract

Maritime user interfaces for ships’ bridges are highly dependent on the context in which they are used, and rich maritime context is difficult to recreate in the early stages of user-centered design processes. Operations in Arctic waters where crews are faced with extreme environmental conditions, technology limitations and a lack of accurate navigational information further increase this challenge. There is a lack of research supporting the user-centered design of workplaces for hazardous Arctic operations. To meet this challenge, this paper reports on the process of developing virtual reality-reconstructed operational scenarios to connect stakeholders, end-users, designers, and human factors specialists in a joint process. This paper explores how virtual reality-reconstructed operational scenarios can be used as a tool both for concept development and user testing. Three operational scenarios were developed, implemented in a full mission bridge simulator, recreated in virtual reality (VR), and finally tested on navigators (end-users). Qualitative data were captured throughout the design process and user-testing, resulting in a thematic analysis that identified common themes reflecting the experiences gained throughout this process. In conclusion, we argue that operational scenarios, rendered in immersive media such as VR, may be an important and reusable asset when supporting maritime design processes and in maritime training and education.

1. Introduction

Working at sea is a challenging occupation with notoriously unpredictable working conditions. This includes long hours, isolated work, inconsistent connectivity to land-based resources and high-stress tasks. Maritime operations in Arctic regions further exacerbate these conditions and introduce unique safety challenges for ships’ navigation teams including a lack of accurate navigational information, extreme weather conditions, inherent technology limitations, heavy reliance on Arctic-specific knowledge, and longer waiting times for rescue services [1,2]. Traffic in the northernmost shipping routes is expected to continue increasing as glacial ice melts, opening new routes that were previously not accessible [3]. Since 80–90% of most maritime accidents can be connected to human operation, and often to suboptimal design, there is a special need to secure the human-centered design (HCD) of maritime workplaces operating in dangerous conditions, such as the Arctic [4,5]. The “Safe Maritime Operations under Extreme Conditions: the Arctic case” (SEDNA) project has explored these challenges and developed an integrated risk-based approach to safe Arctic navigation [6]. This paper reports on HCD and the collaborative process used to produce and demonstrate workplace design proposals for ships’ bridges using virtual reality-reconstructed operation scenarios (VRROS).

Maritime navigation can be described as a safety-critical sociotechnical system. A safety-critical system is one that with any failure could result in loss of life, significant property damage or damage to the environment [7]. Sociotechnical systems (STS) are goal-driven and consist of technical, personnel, work design, and environmental subsystems [8]. A sociotechnical system should be studied from a holistic approach, with the aim of understanding the interaction and dependencies between the individual parts of the system [9]. Maritime navigation consists of multiple safety-critical components, including human and non-human actors (i.e., navigational aids). The successful functioning of this system also requires human actors to possess both technical and non-technical skills [10]. Any failure or breakdown in the system can lead to a potentially devastating situation. To maintain a safe system, the non-human actors must support the human actors to achieve their tasks. Unfortunately, the maritime industry is technology-driven, despite decades of research advocating for human-centered design (HCD), and a lack of user-centered approaches often leading to more complex systems [11,12,13,14,15,16].

Designing for safety-critical systems requires special attention to the requirements, selection, training and certification of users, in this case, operators or navigators on ships’ bridges [17]. Designers are people who work in the field of design, usually possessing a wide range of skills, knowledge and awareness, with the ultimate goal of improving existing situations into preferred ones [18]. It has been acknowledged that designers are finding themselves working on important but unfamiliar tasks that traditionally exist but that are outside their remit [17,18,19]. In parallel, maritime human factor (HF) research has not fully adapted its systems to support and integrate design practitioners in its processes. Furthermore, reports on many HF methods and tools tend to be published for academic and scientific audiences only, while the intended users are practitioners, service providers, or mariners (in this case). This results in very little uptake regarding research results and applications for both new techniques and user-centered solutions [20]. The HF and design domains remain slightly disjointed, leading to a research-practice gap in maritime navigation. This has left the maritime industry lagging behind its transportation counterparts (e.g., aviation, automotive, rail), generally maintaining traditional designs, systems, and attitudes [21]. This gap can be attributed to several factors, including:

• Lack of understanding of user needs and the integration of use context into the design process.
• Lack of accessibility of the methods and results, written primarily for academic target audiences.
• The novelty of design thinking and practice-based methods to maritime research.
• New technology development and uptake in the maritime industry being driven by technology developers instead of end-users.
• A slow-moving regulatory framework in the maritime industry.

This research-practice gap should be resolved through a collaborative and more systemic or holistic approach. Collaborative design is the process by which actors from different disciplines disseminate knowledge about the design process to achieve a shared understanding and use this collective understanding to create new products or designs [22]. Systemic design is the integration of systems thinking and human-centered design to assist designers with complex design projects (e.g., a ship’s bridge) [23]. An example of this combined successful approach in the maritime context is the Ulstein Bridge Concept design research project, which created an integrated redesign of a ship’s bridge [24,25]. Although the designers had no experience as mariners, they were able to integrate the context of a ship’s bridge into the design process through a collaborative and ethnographic approach. The resulting product was a user-centered concept of a future integrated bridge design that focused on safety in a complex system. This was a successful approach, but it was challenging to execute without significant funding, a shipping company as a sponsor and unlimited access to a ship’s bridge. Unfortunately, the regulatory framework in the maritime industry does not support radical innovations such as the Ulstein Bridge Concept; therefore, there has been almost no change in ship bridge design, with little to no integration of the technological systems onboard since the completion of this project.

Instead, technological systems continue to be developed that “assist” with navigational tasks, yet their lack of integration into existing systems and the lack of a systemic approach end up making the system merely more complex [9,26]. Successfully designing solutions for a ship’s bridge requires access to the user context while also maintaining awareness of the rapid development of new design concepts [27]. Adopting a more collaborative, systemic, user-centered approach in the maritime industry requires early intervention and iterative processes involving an interdisciplinary team. The maritime industry is in a unique position to learn from our transportation counterparts who have already overcome many of the technological hurdles faced today by the maritime industry, and integrated HCD at a much earlier stage in the design process. It is important to draw upon experiences and lessons learned in other industries to avoid some of their mistakes and benefit from the successes. An example of ongoing research to promote knowledge transfer between domains is the Horizon2020 SAFEMODE project. This project is attempting to strengthen synergies between the aviation and maritime industries, to inform the risk-based design of systems and operations in both domains and improve human performance [28]. There is also a need for consolidation efforts to be made from the best practices from the automotive industry, particularly in relation to human-automation interaction.

1.1. Current Maritime Research Approach

Simulation technologies can be used to simulate content supporting maritime design. The current, standard technology for maritime education and training (MET) is a full mission bridge simulator (FMBS). FMBS, or high-fidelity simulators, have proven to be extremely useful for research studies and the training of future mariners [29,30,31]. Although providing an effective means of research and training, there is a long list of limitations associated with the use of FMBS, including cost, availability, and lack of flexibility. Further, these simulators do not commonly facilitate human-centered design processes [27]. As a solution to these limitations, immersive technologies, including augmented reality, virtual reality, and mixed reality, have created a new space for advanced maritime research and training applications [21]. In particular, the VR market has recently gained traction within maritime applications, including efforts from ship classification societies (e.g., Lloyd’s Register) to maritime startups (e.g., Immerse) [32]. Maritime stakeholders are increasingly recognizing the potential benefits associated with more flexible and cost-effective solutions that can be used for maritime training, research, and development [33]. Although immersive technology has been available for decades, it has not been fit for use in real-world maritime applications and is therefore not widely implemented [21]. This paper highlights the use of VR technology as rendered through a game engine. Other types of immersive reality are outside the scope of this paper.

1.2. Virtual Reality Research

A recently published systematic review of VR literature identified the two most common fields of application for VR training studies as industrial (i.e., maintenance and assembly tasks, procedural training, etc.) and safety and emergency preparedness [34]. Additional domains that are using VR for training and education purposes include healthcare, firefighting, and other means of transportation (i.e., aviation, aerospace). These domains can benefit from VR as real-world training can be expensive, inflexible, and, in some cases, dangerous. Given the present speed in advancement and innovation of VR systems, it is expected that more affordable and available systems will continue to be developed. Although the context is less widely explored, the maritime industry also has the potential to benefit from VR technology and there are several research initiatives underway to determine the possibilities of VR in the maritime sector [21,34]. As an example, immersive VR was tested by Hjellvik et al. for educating marine engineers, as an alternative to a desktop simulator [35]. The findings indicate that the immersive VR experience led to improved post-test scores in a specific marine engineering task, in comparison to a desktop simulator. The authors advocate for the further research and development of VR and immersive technologies as maritime training solutions [35]. A Finnish university has also completed work in developing immersive safety training scenarios in VR through MarSEVR (Maritime Safety Education with VR Technology) and, later, Immersive Safe Oceans Technology [32,33]. Utilizing scenarios as part of the VR testing framework aligns with the goals of this paper and is to be further discussed. These initiatives have shown that although the technology is still in the early stages of development for maritime applications, VR offers a wide range of solutions for challenges present in MET [33].

As the potential benefits of VR become more evident, it is important to acknowledge that VR research is still in its infancy and is lacking many critical components to evaluate the reliability, validity, and generalizability of its methods and results. Many VR studies have failed to explore the impact of prolonged VR use for training and skill retention, instead, focusing on the short-term uses of VR within an experimental setting [34]. In addition, future VR research should clarify the development process or framework applied so that others can further develop solutions in a more systematic way [34]. The work presented in this paper supports these research initiatives and aims to fill existing gaps related to the lack of frameworks for concept development and user testing when employing VRROS.

1.3. Scenarios as a Tool

Traditional ergonomics methods, including task analysis, cognitive work analysis, and user profiling, have used “scenarios” or “scenario-building” to identify human factor (HF) issues that might impact design [36]. From a design perspective, scenarios have also been used to provoke stimulating ideas, assist with prototyping, and communicate design concepts and fieldwork within a research team [24,36]. In these disciplines, scenario-building has been used to construct and review past user tasks; for example, in a “scenario analysis”, it presents a way to describe current tasks or to explore possible future work or design possibilities [36]. In more recent years, Lurås (2016) has developed the layered-scenario mapping technique, which builds on existing methods from HF and design disciplines and combines them into a useful and collaborative tool supporting the design processes [24]. This technique was developed to (1) offer a framework to use when interpreting information about the situation for which the model is being designed, (2) facilitate the sharing of data collected and insights among the team, and (3) to present the situation at the level of granularity necessary to gain an in-depth level of understanding [24]. The layered scenario-mapping technique was applied in this work throughout the process of selecting and developing the scenarios to meet the needs of the SEDNA project. The objective of this study was to be able to recreate the scenarios as a VRROS, therefore benefiting from scenario-based methods, while using immersive media to experience it. VRROS could serve as a reusable asset in maritime design processes, maritime training, and education.

1.4. Virtual Reality-Reconstructed Scenarios (VRROS)

VRROS, or virtual reality-reconstructed scenarios, is a term that emerged during the SEDNA project to describe the approach used to explore and evaluate the use of scenarios in immersive media. The purpose of using VRROS was to create a realistic experience of being onboard a ship’s bridge that could be replicated, reused, easily edited, and that could serve as a tool for discussion. Scenario-based testing and training in VR are used successfully in medicine, aviation, automotive, industrial, production, and many other domains [34,35,37,38]. However, scenario-based testing in maritime applications is still largely unexplored, with only a handful of maritime organizations using scenarios, these being primarily focused on inspection procedures and safety training [33]. The VRROS developed and tested in this study support these initiatives and can be adapted for use in any application or work domain. The novelty of this approach is that VRROS can also be used to support the design process, allowing for early user intervention and providing a collaborative discussion tool between stakeholders, resulting in an iterative, user-driven design process for maritime applications.

Three VRROS were developed and tested. The VRROS consisted of a reconstruction of either a real accident or a common operation (to be discussed further in Section 2). Therefore, the user could not change the outcome of the scenario; instead, they could comment on the different parts of the scenario. The user was able to virtually move around the ship’s bridge, using the VR controls to explore and experience the scenario from different viewpoints. The process of developing the VRROS, user testing, and the potential for use in MET and research will be discussed throughout this paper.

1.5. Aim of the Paper

One of the challenges in maritime research is that maritime user interfaces for ships’ bridges are highly dependent on the context in which they are used, and a rich maritime context is difficult to recreate in the early stages of user-centered design processes [25,39]. The SEDNA project has removed the constraints of regulation and existing technological solutions, and has instead applied user-centered, design thinking throughout the project’s life cycle to develop, test, and apply novel concepts for navigation using VR. The removal of these constraints allowed this project to develop user-driven solutions that could contribute to a safer and more sustainable shipping industry. This paper presents an example of a VRROS operation-centered design process used to connect end-users, designers, and human factor specialists in a joint process. We will discuss the design process and resulting user testing, exploring how operational scenarios can be used as a tool for both concept development and later for user testing of the concepts.

2. Materials and Methods

A pragmatic approach was adopted throughout this work, utilizing several different qualitative methods and tools to develop and test the VRROS. The process started with an ethnographic inquiry to understand the user’s context onboard a ship’s bridge; this led to the identification of critical and common operations that could be useful as scenarios. Next, the scenarios were developed, through interviews with subject matter experts and data from the FMBS, to create a realistic scenario in VR. Finally, the scenarios were tested on end-users through a think-aloud protocol in which the data were sent directly back to the design team to improve the VRROS experience. Figure 1 provides a summary of the research approach; each part of the diagram will be discussed in the following sub-sections.

2.1. Selection of Operational Scenarios

Re-creating ship operations in any form (simulator, VR, AR, etc.) is a complex task requiring a detailed understanding of the user’s context and the situated interaction between the user and their environment [40]. To do this successfully, multiple methods must be adopted within a collaborative approach. The beginning stages of this work consisted of an ethnographic approach comprising field studies onboard icebreakers, observations of ice operations, and interviews with navigators to develop and test ideas and concepts. Ethnography allows the research team to have contextual insight into the many types of complex operations completed in ice-covered waters [41]. Once the research team had experienced the specific types of operations possible in Arctic navigation, this was supplemented with a review of accidents occurring in Arctic waters conducted with experts in the subject matter. The aim was to have a combination of real accident scenarios to recreate and an additional “typical scenario” representing common operations in Arctic waters. The scenarios had to meet the following criteria to be included:

• The availability of sufficient information from the accident report to understand the sequence of events and the critical decision points.
• The ability to recreate the accident in the FMBS (i.e., the availability of the geographical area in the simulators).
• The ability to demonstrate different types of ice navigation challenges (i.e., ice properties, convoys, submerged rocks, etc.).
• A scenario that could potentially have been avoided with the availability of more accurate information about the surroundings.
• A potentially common scenario in Arctic waters (frequency).
• The scenario includes tasks that users deem to be high risk (criticality) [24].

The three scenarios selected were the Vega Sagittarius and MV Explorer accidents, and a three-vessel convoy situation. The Vega Sagittarius and MV Explorer were real incidents, and the convoy scenario was a generic, common operational scenario developed by the subject matter experts.

The Vega Sagittarius accident was selected because the navigators missed critical information about the navigational situation. They observed and prioritized the icebergs in their visual field but failed to locate and avoid a submerged rock, leading to poor awareness of the situation.

The MV Explorer accident was selected because it demonstrated the importance of having the correct information about ice properties for decision-making. A common challenge in the Arctic is the ability to properly assess ice properties, particularly those of multi-year ice. This is further complicated by the fact that accurate assessment is generally based on years of experience, including cues from the vessel related to vibration and noise.

It was not possible to find an accident scenario that involved a convoy and that met the criteria listed above for inclusion in the project. However, the eventual scenario was selected because icebreaker assistance is frequently needed in ice-filled waters. The need for assistance can arise from different situations, for example, a planned escort of a vessel with a lower ice-class than that recommended for the waters, or to rescue a vessel stuck in the ice. Convoy operations are also interesting and challenging to complete in Arctic waters, and operators can potentially benefit from experiencing them in VR.

2.2. Development of Operational Scenarios

Each of the scenarios was further developed by simulator instructors at Chalmers using the layered-scenario mapping technique. This provided a framework to physically map out and discuss the critical aspects of the scenario, including the vessels’ position, mode of operation, the actors involved, communication (when and to whom), position on the bridge, equipment used, and the information and functionality necessary to carry out each task [24]. The two accident scenarios were replicated to the best of our ability from the available accident reports produced by the Maritime Accident Investigation Branch (MAIB) and were then discussed among the subject matter experts to determine additional information relevant to the accident. Once the internal research team was satisfied with the map, it was taken onboard a local icebreaker and was used as a validation and communication tool to ensure all information was captured, and new insights were recorded.

Once the project team was satisfied with the scenarios, they were implemented into the FMBS at Chalmers. The implementation process involved: locating or creating the designated geographical area in the simulator, adding ice, icebergs, and any other relevant obstacles in the geographical areas according to the accident report, re-creating the route of the vessel according to the accident report coordinates and timestamps in Wärtsilä’s NTPro 5000 simulator application, and finally playing out the scenario and ensuring it matches as closely as possible with the accident report or operation. The next step was to place a subject matter expert or “actor” on the bridge of the FMBS to experience the scenario from start to finish. This allowed the Oslo School of Architecture and Design (AHO) design team to question the subject matter expert about important contextual elements within the scenario using the layered scenario-mapping technique, including equipment, information, and critical decision points. All necessary navigational information was recorded and sent to AHO for VR concept development.

2.3. VR Concept Development

To compare the simulation of the current ship’s bridge technology within the SEDNA framework, this project recreated the simulated tested scenarios in a VR environment. The Ocean Industries Concept Lab established a VR scenario based on data from the FMBS. First, an existing VR simulator was used to recreate the physical environment from the simulator using VR. This simulator supported the realistic rendering of an “oceanscape”, which is a term used by researchers at AHO, defined as the landscape outside the window of a ship, including the light conditions and weather. New 3D assets, such as ships, ship’s bridge interiors and ice, that closely matched the established simulation were added. These assets were developed using the 3D studio MAX CAD software. The open-source OpenBridge design guideline (http://www.openbridge.no/guideline.html, accessed on 17 August 2021) was used to recreate all user interfaces in a new, consistent design. Sequential navigation data, exported directly from the FMBS, served as the basis for the VR scenarios. The data values from the simulator datasets were interpolated, providing the basis for realistic animation of the vessels’ movements and other values in the virtual graphic user interfaces on the virtual bridge. This included data such as the ship’s speed, heading, engine power load, and ship position. Accurate position data and video of user interfaces, including the electronic chart display and information system (ECDIS) and marine radar, were cropped to allow map and radar data to be inserted into new virtual interfaces. Real, physical environments were recreated using 3D models and were extracted and generated based on altitude and depth measurements from satellite-based map data.

A PC interface allowed us to modify the scenario from a connected screen when a user was immersed in VR. The scenarios were packaged as a standalone application for each scenario, with an external interface allowing test personnel to control the simulation outside VR. The participants could move around the VR scene and control applications using the VR controllers. Each scenario was accompanied by a description of the scenario and the available functionality at central scenario segments. The following equipment was needed to run the scenarios:

• The VR hardware used in the testing was a powerful PC, Intel Core i9-9900K CPU, equipped with an NVIDIA GeForce RTX 2080Ti graphic card and 32 GB RAM.
• The VR headset used in the test was the HTC Vive Cosmos with the following specs: dual 3.4” diagonal screen, resolution of 1440 × 1700 pixels per eye (2880 × 1700 pixels combined), 90 Hz refresh rate, maximum 110 degrees field of view (https://www.vive.com/eu/product/vive-cosmos/features/, accessed on 18 August 2021).
• Chalmers VR “lab” equipment: handheld Sony video camera, Apple iPad, large screen, powerful PC (specs listed above), VR headset (HTC Vive Cosmos)
• Tools used to communicate and share information: Box, VR scenario developer (SteamVR, Valve Corporation https://store.steampowered.com/app/250820/SteamVR/, accessed on 18 August 2021), Zoom, TeamViewer, Skype.

2.4. User Testing

The user testing was completed in Gothenburg, Sweden at the Chalmers University of Technology in 2020. Each scenario was tested for usability and usefulness, through a qualitative assessment using the think-aloud protocol with professional mariners. Think-aloud methods allow the participants to talk out loud or verbalize their thoughts while completing a specific task [42]. The goal of this method is to understand a participant’s cognitive processes. Both concurrent and retrospective verbal reports were completed throughout the data collection. The concurrent report involved the participants speaking out loud throughout the scenario about what they saw and were challenged with pre-determined specific questions if necessary. The retrospective verbal report was completed post-scenario and required that the participants reflect upon their experience. In addition to the think-aloud protocol, HF specialists were observing the participants throughout the entire scenario. The test setup allowed the HF specialists to have the same viewpoint as the participants through the TV screen, while also observing their body language and movements (Figure 1). Any interesting observations were noted and added to the participants’ test sheets. The participants were able to move freely around the bridge virtually to experience the scenario from different viewpoints; however, it was not possible to change the outcome of the scenario. Therefore, there was no need for the participants to have any experience with VR, reducing the familiarization time usually required when using new technology.

This paper reports on the user feedback from the VR perspective of the scenarios, and the result of using VRROS as a potential method for HF and designers working with MET. The user testing phase allowed HF researchers to collect and analyze qualitative data from end-users and report directly back to the design team at AHO. This process was extremely efficient, due to the remote work technologies that have been exploited during the COVID-19 pandemic. This process provided the ability to test and adapt the VRROS through remote access to computers. The feedback from user testing was incorporated directly after each scenario was completed, meaning that each subsequent scenario was already improved from the previous one.

2.4.1. Recruitment

Purposive sampling, also known as judgment sampling, is a non-random technique used when the researcher needs the participants to have certain qualities, skills, knowledge, or experience [43]. Purposive sampling was used to recruit participants, primarily through word of mouth, who had the designation of Master Mariner certification at some point throughout their career, and who had recent experience with navigational equipment or were enrolled in the fourth-year Master Mariner program at Chalmers University. The COVID-19 pandemic impacted the ability to recruit and test the desired sample. Participants were therefore primarily recruited internally at the Chalmers University of Technology.

2.4.2. Ethics

When participants arrived at the test lab, they were provided with information about the study and a brief description of the test protocol. The participants were also briefed about the potential risks related to VR use, including dizziness and nausea, and were told that they could stop the test at any point. Each participant was given a unique ID number which was used throughout the test to ensure confidentiality. The consent form was explained to the participants and, once they felt comfortable, they signed, and the test began.

2.4.3. Demographic Information

A total of twenty-two professional mariners evaluated the three scenarios. Nine participants evaluated the Vega Sagittarius scenario, nine participants evaluated the MV Explorer scenario, and seven participants evaluated the convoy scenario. Several participants were re-used in the convoy scenario, as the COVID-19 pandemic prevented recruiting participants from outside Chalmers University. All participants were of Swedish nationality; twenty-one were male, and one, female. The age range was between 18 and 64 years. Nine participants were fourth-year Master Mariner students, eleven participants were Chalmers University employees, and one participant was currently serving onboard ship. All participants met the criteria to be able to participate in the study. In addition to demographic information, the participants were asked about their previous experience using virtual reality. Participants reported that they either had no experience at all with VR systems (11 responses) or a little experience using VR systems (11 responses).

2.4.4. Procedures

A pilot study was completed prior to the official data collection to ensure that the first scenario was technically accurate for user evaluation. The pilot study consisted of a walkthrough with a subject-matter expert of the Vega Sagittarius scenario, to find any potential issues with both the equipment and the test procedure. The official data collection was completed by two HF researchers in the Chalmers University VR lab. Both HF researchers were present throughout the data collection to observe, record the answers from the participants, and assist with the VR controls and maneuvering through the scenario. Participants were asked about simulator sickness (dizziness, feeling unwell, nausea) multiple times throughout the testing period. No participants reported any symptoms of sickness at any point throughout testing. Once consent and familiarization were complete, the participant completed a tablet-based demographic pre-test questionnaire. When the simulation was finished, the researcher provided an overview of the scenario and explained what was to be expected from the VR experience.

Only one scenario was tested at a time (per participant). The scenario began when the participant was seated and relaxed (Figure 2). The researchers had the same visual field as the participants had in VR through the large TV screen, which allowed for seamless communication when referring to specific objects. The researchers allowed the participants to get comfortable on the virtual bridge and encouraged them to explore at their own pace. Qualitative data were obtained through a think-aloud protocol. Each scenario tested different bridge designs and the overall VR experience.

The Vega Sagittarius scenario lasted for 31 min and focused primarily on having access to crucial information about the surrounding environment (Figure 3). This additional information about icebergs, submerged rocks, and safe navigating areas could have potentially prevented this accident from occurring.

The MV Explorer scenario lasted for 44 min and focused primarily on having access to information about ice conditions (Figure 4). This scenario also included a radar overlay on the ocean surface, the ability to plot a path through the ice, a night-vision zoom camera, and a searchlight.

The convoy scenario lasted for 67 min and included two cargo ships, in addition to the ice breaker on which the VR user is located (Figure 5). The main workstation is placed on the starboard side of the bridge. As with the other scenarios, the user is not able to control the ship but can freely move and teleport around the bridge. This scenario focused on the complexity of icebreaking and convoy operations. When several vessels are close to each other, it is critical that the navigators are informed about the movement of the other vessels and the distances between the vessels. In a convoy situation, it is particularly important to monitor the distances between the vessels, and the relative speed differences. For example, if the last vessel in the convoy is moving faster than the vessel in front, it is only a matter of time before there will be a collision. In addition to carefully monitoring the distances and speeds between the vessels, it is just as important to keep watch outside to physically see the other ships moving and the ice moving around the ship.

Once the scenario was complete and the researchers were satisfied with the qualitative data, the researcher helped the participant remove the VR equipment and reacclimate to reality. The retrospective verbal report was then completed, which required the participants to reflect upon their experience. They were asked to recall any additional comments or give feedback about the VRROS experience.

2.5. Data Analysis

The development of VRROS was an exploratory process. There is no existing procedure to develop and test these types of novel concepts. The data analysis was, therefore, an aggregation of various data collected throughout this process, including reflections from the project team during the various stages of research; team meetings that documented the experiences and lessons learned and, primarily, the qualitative data from user testing. The data obtained throughout the VRROS development were assessed using a thematic analysis that was continuous, iterative, and reflexive [44]. The research team reviewed the findings after each part of the research approach and identified general codes representing the development of the VRROS, and the result of VRROS as a user testing method. For the user testing, qualitative data from the twenty-two participants were analyzed. The data were first transcribed individually by each HF specialist, and then compared for consistencies or discrepancies. This process was adopted to be able to cross-check the transcripts and ensure that the data were correctly interpreted. Once the data were coded and collated, several themes were identified that captured both the processes and implications of VRROS development and the final user experiences. The themes in this paper were derived using an inductive approach, meaning that the themes are strongly data-driven and are not strongly connected to a specific research question [45,46]. To finalize and refine the data analysis, several themes were collapsed into each other to create sub-themes, as there was some repetition between the categories, and not enough evidence to support others. This is said to be an expected part of thematic analysis, as it is an ongoing, organic process [45].

3. Results

The main result from this work includes a description and reflection about the experiences and lessons learned from developing and implementing VRROS for Arctic navigation. A secondary result was from the user testing, including a summary of how the participants experienced the VRROS. A short summary of these results is presented in this section, followed by a detailed discussion about the implications of the work in Section 4. Several themes emerged through the development and testing of the VRROS. The main theme that emerged was that VRROS could support the design processes, which was verified by several sub-themes (

Table 1. Themes that emerged from the development and testing of VRROS.

Main Theme	Evidence
VRROS supporting maritime design processes	Designing for safety-critical operations requires that designers have specialized knowledge in the specified domain, which is hard to obtain. VRROS allow a faster exposure and an understanding of a particular context. Operational scenarios provide a collaborative, flexible opportunity to support the maritime design process. VRROS have the potential to support new work practices in MET and to promote interdisciplinary work.
Sub-Themes	Evidence
VRROS as a collaborative tool	Can be used as a strategic tool to create past, existing, or future scenarios. Allow people from all disciplines to capture the essence of the working environment. VRROS can be used as a support to make long-term decisions or to simulate an existing or future scenario, or concept. There is a need for new methods and practices in the maritime domain; this paper provides a new approach to MET. Technology today allows for interdisciplinary work to be completed more easily than ever before. Developing novel concepts and solutions that are wanted and needed by end-users is impossible without an interdisciplinary team. Novel concepts and new technologies could be tested first by using VRROS to check basic usability, prior to their implementation in real life. This could improve safety by allowing users to be involved much earlier in the design process, reducing many of the problems associated with new technologies on ships.
Flexibility of the process	Ability to make design changes immediately (if necessary) to a scenario using cloud/remote technology. Possibility to recreate any ship bridge design, and to test any concept or task. Compared to FMBS, VR is inexpensive. As VR is further developed, it will become less expensive and more adaptable. Repeatable scenarios can support experimental studies and training efforts.
Potential of VRROS	VRROS have the potential to help with MET for navigators, engineers, new cadets, experienced mariners, design students, and even project members who have not yet been exposed to a maritime environment. No participants experienced any form of malaise while using the VR headset, even those participants prone to simulator or motion sickness.

).

4. Discussion

The development and testing of three Arctic navigation VRROS have been completed as a joint collaborative effort. This approach started with establishing operational scenarios, developing and implementing them in the Chalmers FMBS, developing the VR concept, and finally, conducting user testing in VR. The success of this approach was possible through an iterative interdisciplinary research effort. The main results are discussed in the following sections, followed by a discussion of the methods.

4.1. VRROS Supporting Maritime Design Processes

Maritime navigation can be described as a safety-critical sociotechnical system. Designing for this type of system requires special attention to the requirements, selection, training and certification of users, in this case, operators, or navigators on ships’ bridges [17]. The complex nature of maritime navigation, the distributed technical systems onboard, and the lack of standardization lead to a challenging design process. In order to undertake an HCD process, designers must get close to the users and the use environment, which, in the maritime industry, can prove challenging [27]. HF researchers must evaluate and try to improve the performance of a person or team within the complex sociotechnical navigation system that primarily involves tacit knowledge. Each profession must create solutions to improve the work situation, without a proper understanding of or exposure to the work environment. The end-users themselves see a ship’s bridge as a single integrated unit and not as separate design artifacts, which contrasts with the context of the design team [27]. This mismatch has resulted in an industry plagued with safety issues and a poor user acceptance of new technology [15,47]. There is clearly a need for tools to support the maritime design process so as to incorporate HCD.

Today, in many cases, it is unknown whether a new bridge design or automated feature was even wanted or needed by the end-users. To understand this, key stakeholders must be involved from the beginning of the innovation process, adopting a more holistic, systemic approach. The Ocean Industries Concept Lab researchers have been pioneers in promoting HCD in a maritime context, and have developed a model for “design-driven field research” [41]. This model is based on extensive field research, using an ethnographically inspired approach that focuses on three elements—data mapping, experiencing life at sea, and design reflection [41]. This model was used to develop the VRROS and proved to be successful according to the participants in the user testing. Etienne Gernez wrote his doctoral thesis on a human-centered, collaborative, field-driven ship design that leveraged the design-driven field research model proposed by Nordby and Lurås (2014) [11]. His findings promote the connection between the architecture of the ship and the operations of the ship, through integrating end-user experiences into and throughout the design process [11]. Maritime research needs to adopt this approach to systematically move forward toward safer and more sustainable solutions for navigation. Operational scenarios can be used by anyone in any discipline to explore, discover, and understand the context in which they must work. Operational scenarios provide a flexible, collaborative opportunity to support the maritime design process and help promote interdisciplinary work and HCD in the maritime industry.

4.1.1. VRROS as a Collaborative Tool

Another factor contributing to the success of this process was the usage of, and collaboration among, individual competencies within the interdisciplinary team. The team consisted of industrial and interaction designers, software developers, master mariner students, human factors specialists, and engineers. This combination of expertise is lacking in the maritime industry, as it remains heavily driven by technology development and implementation [9,11]. The push toward HCD, and a more integrated approach to ship systems and ship design, has been ongoing for the last two decades [15,16,48,49]. Unfortunately, few approaches demonstrate how to do this in practice. Additionally, the regulatory framework in the maritime industry presents major obstacles, slowing the uptake of such an integrated approach [9,50]. Given the complexity of this work, the collaboration among experts from different domains is invaluable, allowing individuals to draw on their respective skills, experiences and knowledge [11].

In addition to the collaboration of personnel in developing the VRROS, the VR technology itself can serve as a collaborative tool [37]. The SEDNA project, and the approach presented throughout this paper, provide a successful example of an interdisciplinary approach and the merging of work domains. It can thus be expected that interactions between people will become easier and faster as the tools and approach explored in this paper are further developed. Designers, HF specialists, engineers, master mariners, and any stakeholders can work together using the same “base”. VR as a tool can provide a foundation and simple structure to discuss with any stakeholder, at any level of detail or expertise. Using VRROS as part of HF and design research has the potential to enhance existing MET practices. VRROS also provide the ability to collaborate either in person or virtually. In this project, we collaborated virtually, due to the COVID-19 pandemic, through sharing content on cloud-based platforms. Technology allowed two teams operating from different countries, Sweden (Chalmers) and Norway (AHO), to remotely work together, share data, and adapt solutions in real-time. Even though this was the first time that this methodology was adopted and tested, there were few issues related to the remote working conditions. We believe this remote-working methodology will be the future solution for interdisciplinary projects with team members located in different geographical locations.

Using scenarios (i.e., scenario-building, the scenario technique, layered scenario mapping, etc.) as a tool to collaborate within interdisciplinary teams has also proven to be an asset to HF and design research methods [24,36,37]. Scenarios are the most useful in domains with an unpredictable future or outcome. They can be used as support to make long-term decisions or to simulate an existing or future scenario or concept. It is anticipated that the maritime industry will experience more change in the next two decades than it has in the past 100 years [9,51]. Therefore, the maritime industry is an obvious case study for adopting scenarios as a tool to explore the unknown. In this study, three operational scenarios were developed and tested through a collaborative, iterative, human- and design-driven approach. The two accidents, and a complex Arctic operational scenario, were recreated to explore the use of VRROS and to test novel AR concepts in a safe and repeatable way. The use of scenarios invited participants to be a part of a story, which allowed them to engage in a unique approach offering a more immersive and reflective experience. Additionally, VRROS provide a starting point and can be expanded for any type of supplementary testing in the form of a “plug-and-play”. For example, adding physiological measures (i.e., eye-tracking) to the current approach is a simple add-on to the current framework. There is an opportunity to develop scenarios as a tool to test new concepts, designs, and technologies through a more repeatable experimental protocol for future studies.

4.1.2. Flexibility

Simulators have been used for MET since the 1950s. The use of simulators has expanded from developing and training basic navigation skills and passage planning to complex bridge and cargo operations, ship-to-shore training, and, more recently, non-technical skills and team management [29,30]. While there are many positive aspects to using high-fidelity simulators for MET, there are also limitations, including the high cost, high demand, and therefore limited availability of the simulator. In maritime academies, the FMBS are usually fully booked for the training of maritime professionals, students, or research projects. Furthermore, given that the FMBS is mainly developed for training purposes, it is built as a replication of an actual bridge environment. This limits the flexibility and adaptability associated with testing new concepts and technologies. Given the fast pace of technology development in the maritime industry and changing work practices, we must find flexible and adaptable solutions to supplement the FMBS [52]. Although VR technology is still unable to replace reality or an FMBS, it is getting closer. The technology is constantly being improved to a point where it is currently a realistic option to investigate [21]. We argue that VRROS should not replace simulators; in fact, simulators are a necessary part of the process of developing VRROS. However, VRROS may represent a viable option to support the gaps related to using FMBS for all aspects of MET.

The development and testing of the VRROS led to a realization about the potential flexibility of this testing method, compared to the current FMBS testing regime. This process essentially removes the cost, difficulty, and necessity away from the FMBS to a desktop gaming simulator, enabling a more efficient user-testing process, and therefore a more efficient design process. VRROS offer similar benefits to FMBS, including the ability to train or educate using playback, and offers post-simulation debriefing with additional flexibility. The design team can generate new concepts for either a new bridge design, feature, or event between each scenario, and the HF researchers can obtain feedback directly from the end-users. VRROS help promote HCD and invite users to be involved from the beginning of the design process, providing them with a platform to communicate with researchers from various domains. Using VRROS can foster innovation and thinking outside the box, as the solutions are not fixed; they are dynamic and are easily adapted.

4.1.3. Potential of VRROS

While completing user-testing, many participants identified that this technology and the use of VRROS could also be used for other aspects of MET. In the first two years of maritime education, there is little opportunity for shipboard experience, either on the bridge or in the engine room. VRROS could serve as an accessible method of introducing students to a ship’s environment, to better prepare them for their future. In addition to exposure for students, there is also an opportunity to use VRROS to support career development for professional education. Chalmers University and many other maritime institutions offer courses in continuing education and re-certification. In many cases, an FMBS is necessary for parts of the training, which is costly, difficult to schedule, and inflexible. VRROS could support the FMBS aspect of this continuous training by providing flexible solutions for specific training cases (i.e., icebreaking and a convoy situation). Further, no participants experienced any form of negative physical effects (lightheadedness, nausea, eye fatigue or strain) associated with using VR, even those participants prone to simulator- or motion-sickness. This was a positive result, indicating that the technology is getting to a point of readiness for adaptation to various MET applications.

This work also aligns with an initiative recently completed by the Ocean Industries Concept Lab to expose design students to the ship’s bridge environment, which had a positive outcome on learning. In response to the COVID-19 pandemic, design researchers used the VRROS as a substitute for student fieldwork in one of the design courses offered by the university. This was an innovative solution to allow the students to still be able to gain a situational understanding of the environment they were designing for (ship’s bridge) since the current lockdown restrictions prevented them from visiting an actual ship’s bridge. The VRROS provided them with a sense of scale, space, and time, and allowed them to revisit their ideas as much as they wanted since the VRROS allow unlimited access, unlike a real ship’s bridge. The possibilities of VRROS also extend far beyond the maritime domain. The approach presented in this paper is a “plug-and-play” system that is easily modified, offering opportunities for any scenario or work environment. This study provided insight into the possibilities for the use of the VRROS from the perspectives of education, training, and research development.

4.2. Limitations and Strengths of the Method

The approach adopted throughout this paper involved a combination of design and HF methods, leading to a new type of development and testing procedure. This resulted in an unusual style of paper with limited empirical results, as it exists as something between a methods paper and an experimental study. Therefore, one limitation is that there is no way to validate the results or the approach itself. Further work must be completed to evaluate the process and the usefulness of developing VRROS and user testing. As this approach is so novel, and is in the earliest stages of development, there are still plenty of opportunities to expand and adapt the existing framework to meet the needs of MET. We believe that the unique combination of HF and design methods applied within this study have helped bridge the gap in maritime research practice. This approach offers tangible tools to help foster interdisciplinary MET research. The individual methods reduced the impact of the limitations associated with each method and enhanced their respective strengths. We believe that a more holistic, adaptable, and flexible testing approach was discovered.

VR usage as a training medium is a limitation. The success of VR testing is heavily influenced by the quality of the hardware and content. Until recently, the hardware has remained a major limitation for VR testing, particularly for operations in challenging environments [21,35]. Although the hardware tested in this project (HTC Vive Cosmos) is one of the best available, it still has limitations. The weight of the headset is 702 g. Several participants commented on the weight after wearing the headset for 30–45 min. This project only had access to one VR headset, which limited the researchers to testing only one person at a time. Multiple VR headsets would allow for testing several people in the same scenario, while also assessing communication and teamwork. To maintain acceptable quality and user experience, there is a need to adjust the VR geometry and textures to reach the minimum specifications for framerate (in this case, 90 fps) to reduce motion sickness. There is still much work to be done to improve the integration of VR into MET, in particular, more research on the educational efficacy and user acceptance of VR, which will help determine its place in maritime design and MET.

5. Conclusions

This paper explored whether VRROS could be used for maritime design and MET. This approach involved ethnographic work to select three appropriate scenarios for Arctic navigation, the development and implementation of the scenarios in the FMBS, the VR concept development, and finally, the user testing of the three VRROS. The qualitative data collected throughout this process of both scenario development and testing were analyzed using thematic analysis. The results revealed that VRROS can support maritime design and MET as a collaborative, flexible, and reusable asset. VRROS can support prototyping and design, promoting early user intervention, and an iterative design process, allowing input from all maritime stakeholders. VRRO also have potential applications for safety training, education, and scenario-based testing, promoting the diversity and advancement of VR technology. The results are particularly relevant for safety-critical and hard-to-reach work domains. VRROS offer a promising tool to support interdisciplinary work, as well as the opportunity to improve safety in the maritime domain.

As immersive technologies are further developed, it is valuable to continue to explore the possibilities of VRROS and better understand how they could fit within the MET framework. The potential of VRROS is endless and should be further exploited. There is a need for more interdisciplinary research to develop tools and solutions, based on user needs, to improve the working environment for seafarers and ensure the safe implementation of new technologies. This is only possible through combining knowledge from different disciplines and working together in an iterative process. In conclusion, we argue that operational scenarios, rendered in immersive media such as VR, may be an important and reusable asset in maritime design processes, and support future MET initiatives.