The Development of a Steering Angle Estimation Model for Evaluation of Simulated Ship Handling Training for Remote Operators of Autonomous Ships

Abstract

Through technological development, ships are being automated, reducing the number of human operators. Accordingly, the responsibility of humans becomes more significant, making a single operator’s proficiency count. Simulated ship-handling training evaluates trainees’ proficiency using specific criteria to verify proficiency. However, the present criteria are confined to training scenarios, and it is hard to determine whether trainees finally achieved expert-like ability. This research conducted probabilistic estimation on experts’ average steering angles. The paper contains the corresponding explanations for each step of the research methods, from the data preprocessing step to the probabilistic steering angle estimation. The research findings include the experts’ trendline of average steering angles and the sample trainee’s evaluation results.

1. Introduction

Even though technology drives ships to be highly automated to support human factors, on-scene or remote ship operators are inevitably responsible for safe ship handling [1,2,3,4,5]. Accordingly, ship-handling training methods should also follow the trend of maritime technology development [6,7,8]. As ship control primarily relies on rudder steering rather than engine adjustment, except for highly urgent emergencies, the simulated training should also measure how skillfully the navigator steers the rudder in a given situation [9]. Trainees are considered well-trained when they achieve appropriate ship-handling skills like professional navigators [10].

Many researchers have pointed out how vital ship handling is, considering that maritime accidents happen due to inappropriate maneuvering [11,12,13], energy losses from excessive rudder controls [14], and even maintaining the quality of goods [15]. However, evaluation methods for simulated ship-handling training have been merely based on scenarios, such as the restriction of exceeding the maximum value [16,17,18], watching the trainee’s physical condition [19], or robust methods under succeeding or failing rules [20]. Here, this research considered a more specific evaluation model, not only accepting that a trainee has passed the simulation test if they keep within the limit line but also by being closer to the experts’ performance.

Collecting professional navigators’ data in each situation is challenging for many reasons. Data usage is restricted by the company’s policies or privacy issues, the actual ship test boarded with professional navigators is complex [21], and a simulation test with experienced navigators also has risks due to the limited chances of simulation [22]. Accordingly, this research used the university training ship’s data and applied probabilistic estimation methods to populate the uncollected range of data to find specific patterns diverted along the different altering angles.

Therefore, this research aims to develop a probabilistic steering angle estimation model using the actual navigator’s steering data from the target ship to obtain a sufficient professional navigators’ database. The specified process is enumerated, including data collection, preprocessing, probability calculation, and comparing the trainee’s results with the experts’ steering patterns.

2. Materials and Methods

In this research, the proposed methods are to extract a single ship’s handling data. A single ship was selected to collect data involving actual navigators’ decisions. Then, data analysis, hidden Markov chain generation, and data estimation were conducted. Figure 1 shows the workflow of the proposed methods.

2.1. Data Collection and Preprocessing

2.1.1. Selection of the Target Ship

The target ship was a university training ship, as shown in Figure 2. The training ship SEGERO was built in 2018, with an LOA of 133 m, a width of 19.4 m, and a gross tonnage of 9196. The navigators have experienced years on board merchant ships and have served on this ship from the moment that this ship was built.

2.1.2. Steering and Course Variance Data

Data were collected from April 2022 to January 2023 using a voyage data recorder. Throughout 14 voyages, the saved records were backed up periodically by connecting to the equipment’s storage. The data were kept in text files 15 s long with all the connected sensor data. A total of 233,759 files and 3,506,399 s of data were extracted and arranged in tabular format. In the extraction, basic text analytics for text patterns was used to select the specific values of the sensors. The collected data include the “ship’s course”, “GPS position”, “rudder angle order”, “ship’s speed”, and “steering system operation mode”.

Unlike the AIS data, the data from the voyage data recorder were just about noticed. Hence, the missing value was interpolated linearly because the length of the missing parts was in a few seconds, thus not requiring additional data smoothing techniques. As this research focuses on manual steering, the collected data were sorted for only “manual steering mode”. Then, the timelines with ship speeds less than ten or over fifteen knots were eliminated to restrict the speed variable. Afterward, the course alteration angle per second was calculated by separating consecutive timelines into groups and merging the data into the whole dataset.

2.2. Hidden Markov Chain

In generating discrete-time Markov chains, the research focused on correctly generating matrices. Separately, for each variable, steering angle, and course variance, frequency counting was conducted, and the overcounts in non-consecutive timelines were removed. When the Markov chains were generated based on the matrices, the emission frequency of course variance angles for each steering angle was counted. Figure 3 shows the steering angle and course variance of the Markov chains in the illustration.

In Figure 3, the steering angle could transition to the next state or sometimes to another state over the next state, which means that the steering angle changed slowly or rapidly. Likewise, the transition of course variance means the course-altering speed. As the ship’s steering angle and course changed slowly along the rudder and the ship’s movement [23], the expected shape of the Markov chain for both variables was a line shape.

When both sequence data were preprocessed and arranged in seconds, the total length in time was 802,914 s. Transition probability matrices were calculated by designating states, counting the transition frequency, and transforming it into probability. The “steering angle” states are in every degree from −35 to +35 degrees, and the “course variance angle” states are in every 0.1 degrees from −2.0 to +2.0 degrees. Afterward, the transition from state to state was counted in matrices, as the pseudocode explains in Algorithm 1.

Algorithm 1 Transition frequency counting

1: n ← length[StateData] 2: m ← length[SequenceData] 3: si, sj ← StateData 4: qk ← SequenceData 5: TransitionMatrix(i,j) = 0, i = 1, j = 1 6:   for i to n do 7:    for j to n do 8:     for k to m do check 9:      if qk = si and qk+1 = sj then 10:       Update TransitionMatrix(i,j) using TransitionMatrix(i,j)+1 11: return TransitionMatrix

Where “n” is the number of states, “m” is the number of sequences, “StateData” is the array vector that includes every state, and “SequenceData” is the experts’ sequential data changing in time and the index numbers of elements “i”, “j”, and “k”. Then, the transition frequency matrices were transformed into probability matrices. Afterward, the emission probability matrix from steering angle to course variance was generated upon the counted frequency, as the pseudocode explains in Algorithm 2.

Algorithm 2 Emission frequency counting

1: n ← length[RudderStateData] 2: m ← length[AlterStateData] 3: t ← length[SequenceData] 4: ri, rk ← RudderStateData 5: aj, ak ← AlterStateData 6: EmissionMatrix(i,j) = 0, i = 1, j = 1 7:   for i to n do 8:    for j to m do 9:     for k to t do 10:      if ri = rk and aj = ak then 11:       Update EmissionMatrix(i,j) using EmissionMatrix(i,j) + 1 12: return EmissionMatrix

Where “n” is the number of steering angle states, “m” is the number of course variance states, “t” is the length of the sequences as well as the length of time, “RudderStateData” and “AlterStateData” are the array vector that includes every state of steering angles and course variance angles, and “SequenceData” is the experts’ sequential data of rudder angles and course variance angles observed second and the index numbers of elements “i”, “j”, and “k”. Similar to transition frequency calculation, the algorithm counts which course variance was observed for each steering angle. Then, the frequency values were transformed into probability values. Based on the prepared database, the research conducted the random-walk simulation of course alteration and the steering angle estimations.

2.3. Original Course Alteration

In this research, the altering point is where the course variance stops. When the course stops varying at the local minimum value and rises to the local maximum value, the changed angle is starboard-directed altering. In the opposite case, it is portside-directed altering. An illustrated explanation is shown in Figure 4.

The above methods extract the consecutive altering as a single alteration. Thus, the non-consecutive use of tiny angle changes would be separated as short course alterations. The original trajectories of the target ship’s navigators are shown in Figure 5, using distinguishable colors for each alteration.

The course alterations in Figure 5 show the goal of this research because even when the data collection period is over a year, there is still a shortage of data variety. Hence, the proposed model estimates the steering angle to solve this quantity and coverage problem. Before estimation, the research validated the estimation accuracy of the average steering angles for each course alteration. Below, Figure 6 depicts the average steering angles for each course alteration in the original data. Even though the boxplots visualize the trend lines dimly, applying them as evaluation criteria for the simulated ship-handling training is insufficient.

3. Results

For the preprocessed data, the transition matrices (steering angle and course variance) and the emission matrix between them were calculated and generated into Markov chains. The scarce course-altering data were populated from random-walk simulated course alterations. Then, the research conducted steering angle estimation for alterations.

3.1. Hidden Markov Chain of Steering and Course Variance

The steering angle Markov chain and course variance Markov chain were generated with line shapes as expected. Values close to zero are aggregated, others away from the center are dispersed, and emission probabilities are connected in blue lines in Figure 7.

3.2. Steering Angle Estimation

3.2.1. Random-Walk Simulation of Possible Course Alterations

The above Markov chains are in discrete seconds. The random-walk simulation was conducted in ten million seconds to cover most altering cases. A total of 166,305 course alterations were extracted, as shown in Figure 8, and their trajectories are shown in Figure 9 using distinguishable colors for each.

Figure 9 depicts the various course alterations in trajectories, with the ship proceeding from the same position with the initial course of zero degrees. Each trajectory was generated by trigonometric functions using random-walk simulated course alterations with a thirteen-knot speed, the initial speed set in the simulated ship handling training.

3.2.2. Average Steering Angle Estimation

In the validation process, the hidden Markov model Viterbi algorithm was used for the estimation. A total of 39,534 original alterations were used, and the course-altering angle and average steering angle for each alteration were calculated in the validation dataset. Using the ten-fold cross-validation method, the validation dataset was separated into a “train dataset” and a “test dataset” only to calculate the estimation accuracy. The estimation accuracy for the exact corresponding value without allowing angle error was less than 0.5 accuracy, but when allowing error angles from one to ten degrees, the accuracy rose to 0.9789. The estimation accuracy with allowing angle error is arranged in

Table 1. Estimation accuracy of “Average steering angle” for each allowed angle error.

Angle Error (deg)	1	2	3	4	5	6	7	8	9	10
Estimation Accuracy	0.5484	0.8478	0.9349	0.9609	0.9715	0.9753	0.9771	0.9779	0.9784	0.9789

.

The estimation accuracy was calculated upon cross-validation by separating the training and test datasets for objective results. The estimated average steering angles vary over course alteration angles and time. Figure 10 shows the populated experts’ course-altering and steering angles in elapsed times without outliers for clear visualization.

4. Discussion

The proposed model is to generate more sufficient professional navigators’ data through random-walk simulation and probabilistic estimation. The findings are discussed in the following subsections.

4.1. Expert Navigators’ Average Steering for Each Course Alteration

Using the Markov model, simulating the following course variance per second repeatedly leads to probabilistic generations of random ship handling. The course alteration in Figure 8 shows probable ship handlings of the target ship’s navigators. Hence, the accumulated trajectories in Figure 9 have a higher density in regular angles than the dramatic ones. The average steering angle estimation accuracy was greater than ninety percent when allowing the angle error to be over three degrees.

4.2. Sample Evaluation of Ship-Handling Training

As the purpose of the proposed model is to evaluate the trainee’s ship-handling simulation results, short simulation experiments were conducted. The scenario is for the ship handling training requiring participants to follow the route as closely as possible. The course-altering angles vary from 5 to 39 degrees, and the distances between the waypoints vary from 0.1 to 0.72 miles. The initial speed was set to 13 knots; without controlling the engine, only manual steering control was allowed. A total of 70 participants conducted the training. Out of the 70 participants, the result of the best participant who made the slightest distance deviation from the charted route is visualized in Figure 11, and the average steering angles are marked in Figure 12.

The simple ship-handling training focused on steering ability, which prevents the ship from deviating from the charted route. When the trainee’s trajectory showed the minor average deviation of 5.750 cables in Figure 11a, the rudder steering angles changed dramatically in Figure 11b. This fact infers that training evaluation should approach various aspects besides the specific threshold-based evaluation. Figure 12 overlaps the trainee’s results on the experts’ average steering trends. As the diamond marks colored by each elapsed time denote, some marks are matched in colors, and others are not.

Table 2. Comparison of the “Average steering angle” between the trainee and experts.

No.	Altering Angle (deg)	Elapsed Time (s)	Trainee’s Average Steering Angle (deg)	Range of Experts’ Average Steering Angle (deg)	Result
1	−2	32	1.7	−5.0 ~ 0.3	Failed
2	7	38	0.2	0.0 ~ 4.6	Passed
3	−15	45	−3.2	−7.9 ~ −2.7	Passed
4	−10	77	−0.6	−4.0 ~ −1.1	Failed
5	−12	65	0.9	−4.3 ~ −1.4	Failed
6	13	73	1.4	1.2 ~ 5.6	Passed
7	−30	82	−4.8	−9.0 ~ −4.2	Passed
8	9	106	0.6	0.0 ~ 3.8	Passed
9	50	100	8	7.4 ~ 13.0	Passed
10	−38	112	−2.5	−7.0 ~ −3.7	Failed
11	−19	109	−1.6	−3.9 ~ −1.6	Passed
12	8	160	0	0.0 ~ 1.5	Passed
13	19	161	0.7	0.4 ~ 3.7	Passed
14	10	123	0.1	0.0 ~ 1.8	Passed
15	−7	125	−0.3	−1.8 ~ 0.0	Passed
16	34	145	2	1.4 ~ 7.2	Passed

shows specific results comparing the trainee’s training data to the experts’ trend data.

According to the other evaluation methods using thresholds, an average deviation of 5.750 cables may be allowed to pass the training. However, in Table 2, the trainee failed in four out of sixteen course alterations to keep average steering angles within the range of the experts’ average steering angles. The result denotes that twenty-five percent of the course alteration is not close to the experts’ yet. The simulated ship-handling training could be evaluated more objectively when the trainee passes the threshold-based criteria and achieves a similar steering ability to experts.

5. Conclusions

The background of this research was the development of unmanned and autonomous ships. Due to the present trend of increasing the responsibility of navigators and decreasing the required number of human operators, there are questions about automated systems in terms of safety [24]. In addition, the responsibility in urgent situations stays on the human operator; it is necessary to develop a proficient evaluation model for future navigators. Hence, the authors focused on evaluation methods of simulated ship-handling training, especially on average steering angle in altering. Unlike other research using threshold-based criteria, the proposed model achieved comparable expert data as evaluation criteria. Probabilistic methods accurately estimated and populated non-covered expert data, and the trendlines of average steering angle recovered. Afterward, sample evaluation was conducted using the trainee’s simulated ship-handling training result.

The research used the hidden Markov model beside the neural networks in the estimation. As this research aims to populate the expert data for educational purposes using a simulated environment, the process of the proposed model has to be explainable and expendable. In the proposed methods, the data generated during the process stay. Hence, the additional evaluation criteria, such as the experts’ steering sequences, can be used further. Moreover, the HMM model is based on the transition of states from actual navigators’ steering and course alteration. Hence, the result of any course alteration would be in the range of actually conducted ones. Taking those as an advantage, even the neural network models could have better estimation performance; as mentioned earlier the research used the hidden Markov model.

Limitations and future works remain for further consideration. As a limitation of the present research, even though the proposed model populates the experts’ data, the data are still restricted to the original ship’s movement tendency. The tendency is inevitably influenced by the environmental factors of the target ship, such as the water area and traffic route. In future work, the ship’s speed has to be sophisticatedly measured in the probability calculation, because the course alteration differs along the speed. External forces such as wind and current should also be considered as well. Additionally, the model still requires a single target ship’s original data for each type of ship. Understanding this limitation, the authors will improve the model for real applications for objective ship-handling ability evaluation.