Dynamic Multi-Period Maritime Accident Susceptibility Assessment Based on AIS Data and Random Forest Model

Abstract

Maritime accidents, such as ship collisions and oil spills, directly affect maritime transportation, pollute the water environment, and indirectly threaten life and property safety. Predicting the maritime accident susceptibility and taking measures in advance can effectively avoid the accident probability and reduce the risk. Therefore, this study established dynamic multi-period (monthly, yearly, and five-yearly) maritime accident prediction models based on the random forest (RF) algorithm and Automatic Identification System (AIS) data for susceptibility assessment. First, based on historical maritime accidents and influencing factor data, we generated the feature matrixes and selected the conditioning factors using the Pearson correlation coefficient. Then, we constructed the accident susceptibility models using the RF method and evaluated the model performances based on the accuracy, recall, precision, F1-measure, ROC, and AUC values. Finally, we developed accident susceptibility maps for different period scales. The results show that the monthly, yearly, and five-yearly models performed well according to the validation values. And the three-period susceptibility maps show similar patterns. The high-susceptibility areas are close to the shore, especially from the Shanghai shore to the Guangxi shore. In addition, the ship density and bathymetry are the most critical factors among the ten influencing factors in the three models, contributing around 25% and 20% of the total information. These models and maps can provide technological support for maritime accident susceptibility assessment on a multi-period scale, which can be helpful for route planning and resource allocation in marine management.

1. Introduction

In the context of global economic integration, social and economic development increasingly relies on maritime transportation. Over 80% of the international trade in goods is carried by sea, which is even higher for most developing countries [1]. However, because of the increased number of ships, coupled with the extreme weather because of climate change and the growing uncertainty of the shipping environment, such as channel erosion narrowing and increasing typhoon and storm surge events, maritime accidents, such as ship collisions and grounding accidents, often occur [2,3]. Maritime accidents refer to accidents that cause casualties, property losses, and environmental pollution damage during navigation, berthing, and operation [4]. The disruptions in ports and shipping lanes that result from maritime accidents mean that food, energy, medicine, and other essential items do not reach those in need [1]. From 2011 to 2018, a total of 10,442 maritime accidents (including minor accidents and non-transport-ship accidents) occurred in China, resulting in 2537 deaths, 2042 shipwrecks, and a direct economic loss of CNY 5.238 billion [5]. Therefore, predicting and evaluating the maritime accident susceptibility in advance is urgent, which can effectively avoid the accident probability and reduce the risk.

Susceptibility analyses are helpful for probability prediction, zonation, and follow-up management [3,6], which are parts of risk assessment. Maritime susceptibility accident and risk assessment methods initially included early case studies [7,8] and Formal Safety Assessment [9,10], and then later went on to include probability and mathematical statistics [11], computer simulation [12], Bayesian networks [13,14,15], grey clustering [8], fuzzy fault tree analysis [16], and other methods [17]. Recently, people have begun to use machine learning algorithms to solve related problems. Topics include “whether a region is an accident-prone area” [3,18,19], “whether a region is an accident-prone area for different severity” [3], and “dynamic monitoring ship safety in extreme weather events” [2]. These studies provide methods and atlases for maritime risk prediction and monitoring. However, there are several problems to be solved. First, in the process, the general conclusion is that the ship flow is critical for the maritime accident occurrence. However, due to the unavailability of ship flow data, the long-term forecasting susceptibility is generally based on fixed-period ship flow data, which ignores the changing of ship flows [3]. In addition, the topics are generally for multi-year forecasting or short-term monitoring. There needs to be more dynamic monthly and annual forecasting models in most sea areas in the world for maritime safety decision-making planning.

This work aims to generate dynamic multi-period maritime accident susceptibility models using dynamic Automatic Identification System (AIS) data, dynamic weather data, static environmental features, and the RF algorithm. The monthly maritime accident prediction (MMAP) model, yearly maritime accident prediction (YMAP) model, and multi-yearly maritime accident prediction (M-YMAP) model are established in this work. We illustrate our methodology by applying it to the East China Sea and South China Sea. The paper is organized as follows. Section 2 describes the materials used in this work, including the study area, maritime accident data, and accident-influencing factors. Section 3 describes the methodological framework, including the establishment of the model, selection of the features, preparation of the training data, and evaluation of the models. Section 4 presents the main results. Section 5 and Section 6 discuss the results and conclude the article.

2. Materials

2.1. Study Area

The study area (3–31° N, 106–127° E, with an area of ∼9.4 × 104 km²) is located on the coast of China, including most of the East China Sea and South China Sea (Figure 1). It borders Zhejiang, Fujian, Guangdong, Guangxi, and Hainan provinces. This area is an important sea passage between the Northwest Pacific and Uttara Patha Ocean. In addition, the area is also faced with a high maritime accident risk.

2.2. Maritime Accident Data

We catalogued 3627 maritime accidents from the Maritime Safety Administration (MSA) of China between 2015 and 2021 [20]. After removing the sickness of crew members and no external cause of machine damage, which were irrelevant for this study, we found a total of 3558 maritime accidents caused by the ship navigation environment. We collected information about the damage occurrence dates, locations (latitudes and longitudes), severities, and types for all of these records. The spatial distribution of the filtered set of maritime accidents is presented in Figure 1. The results show that the maritime accidents are distributed widely in the study area, particularly around the near-shore areas.

Figure 2a shows that 2017 had the most accidents. August, September, and October were frequent, accounting for 32.21% of the total accidents. Fishing vessels accounted for the highest proportion of all the accidents, accounting for 43.12% of the total accidents, followed by dry cargo ships and bulk carriers, for which the proportions accounted for 18.27% and 12.88%, respectively. The MSA [3,4] classifies accidents into four levels based on the severity: minor accidents, ordinary accidents, major accidents, and severe accidents, and it classifies them into ten types based on the cause factors: “touch rocks”, “stranding”, “collision”, “touch”, “fire and explosion”, “sank”, “operational pollution”, “damage by waves”, “wind”, and “others”, according to the accident type. Figure 2 shows the frequency of the accidents for each severity level and accident type between 2015 and 2021. The average annual number of accidents was 500; ~71.02% of these were minor accidents, followed by ordinary accidents, for which the proportion accounted for 25.04%. Among these, collision accounted for the highest proportion, accounting for 23.51% of the total accidents, followed by sank and stranding, for which the proportions accounted for 13.32% and 12.90%, respectively (except others).

Table A2. The frequencies of different accident types (except Others) in each month between 2015 and 2021.

Month	Others	Collision	Sank	Stranding	Fire and Explosion	Touch Rocks	Wind	Touch	Damage by Waves	Operational Pollution
1	108	48	38	38	21	11	3	4	2	1
2	83	27	28	31	22	10	0	3	1	0
3	98	69	34	46	24	16	1	10	0	2
4	108	66	41	35	10	16	7	8	0	1
5	98	38	32	26	18	7	3	2	1	1
6	105	24	28	35	12	7	1	3	2	2
7	125	25	43	44	18	16	21	4	9	2
8	161	66	37	42	27	23	32	5	3	2
9	126	70	42	51	28	25	9	4	5	0
10	150	61	56	50	24	23	15	3	3	2
11	100	53	44	30	20	11	2	3	4	1
12	144	50	51	31	11	23	3	5	3	1

and Figure A1 in Appendix A show the frequencies of the different accident types (except others) in each month between 2015 and 2021. It can be seen that accidents caused by wind and damage by waves show more seasonal patterns and are more concentrated between July and October. Typhoon disasters are more likely to occur in the study area from July to October, with strong winds and rough waves resulting in more frequent ship disasters and wave damage.

2.3. Accident-Influencing Factors

Humans, the vessels, the environment, and management are the core factors for ship safety. As people are subjective factors and cannot be quantitatively evaluated, they were not judged in this study. In addition, the Vessel Traffic Service (VTS) coverage in the study area has basically reached 100%; thus, the maritime supervision capacity is assumed to be the same. Finally, we summarized the relevant factors and found that ship and environmental data, such as the ship density, ship type, temperature, waves, etc., are the most widely used maritime influencing factors in relevant studies worldwide [2,3,8,18,19], as shown in

Table 1. Datasets and corresponding accident-influencing factors.

No.	Data	Resolution (Original)	Unit	Description	Source
F1	ShipDensity	-	pc	The number of ships in a unit	https://www.msa.gov.cn/ (accessed on 20 March 2023)
F2	AveLength	-	m	The average length in a unit	https://www.msa.gov.cn/ (accessed on 20 March 2023)
F3	AveWidth	-	m	The average width in a unit	https://www.msa.gov.cn/ (accessed on 20 March 2023)
F4	FishRatio	-	ratio	The ratio of fishing vessels in a unit	https://www.msa.gov.cn/ (accessed on 20 March 2023)
F5	Bathymetry	1′ (~2 km)	m	The bathymetry of the grid	https://www.ngdc.noaa.gov/mgg/global/etopo5.HTML (accessed on 23 March 2023)
F6	DisShore	-	km	The distance from shore	https://gadm.org/download_world.html (accessed on 24 March 2023)
F7	MaxTemp_Days	0.25° (~27 km)	°C	The number of days of temperatures exceeding 35 °C	https://cds.climate.copernicus.eu/portfolio/dataset/reanalysis-era5-single-levels (accessed on 28 March 2023)
F8	MinTemp_Days	0.25° (~27 km)	°C	The number of days of temperatures lower than 0 °C	https://cds.climate.copernicus.eu/portfolio/dataset/reanalysis-era5-single-levels (accessed on 28 March 2023)
F9	MaxPre_Days	0.25° (~27 km)	mm	The number of days of precipitation exceeding 50 mm	https://cds.climate.copernicus.eu/portfolio/dataset/reanalysis-era5-single-levels (accessed on 28 March 2023)
F10	MaxWind_Days	0.25° (~27 km)	m/s	The number of days of the wind speed exceeding 17.2 m/s	https://cds.climate.copernicus.eu/portfolio/dataset/reanalysis-era5-single-levels (accessed on 28 March 2023)
F11	Maxcloud_Days	0.25° (~27 km)	ratio	The number of days of the cloud height exceeding 0.8	https://cds.climate.copernicus.eu/portfolio/dataset/reanalysis-era5-single-levels (accessed on 28 March 2023)
F12	MaxWave_Days	0.25° (~27 km)	m	The number of days of the wave height exceeding 2.5 m	https://cds.climate.copernicus.eu/portfolio/dataset/reanalysis-era5-single-levels (accessed on 28 March 2023)

.

2.3.1. Ship Features

The ship density (ShipDensity) [3,18] refers to the number of ships per unit area, somewhat reflecting the busyness in a water area. This study uses the number of vessels per unit to present the ship density. The AIS data between January 2016 and April 2021 obtained from the MSA [20] were used to obtain the ship trajectories and corresponding times. In addition, we found that fishing vessels accounted for the largest proportion of accidents (see Section 2.2). Therefore, we considered the fishing vessel (FishRatio) ratio as an influencing factor. Moreover, the average length (AveLength) and the average width (AveWidth) in a unit were also considered. All the above data were obtained from AIS data.

2.3.2. Static Environmental Features

The distance from the grid to the shore (DisShore) [18] was considered a risk for grounding-related hazards. We extracted a high-resolution world landmass shapefile from gadm.org (https://gadm.org/download_world.html (accessed on 24 March 2023)) and calculated the distance using the Geographical Information System (GIS) tool. The bathymetry [19] under the water surface indicated the probability of encountering the cause factor stranded, and it was extracted from the NOAA (https://www.ngdc.noaa.gov/mgg/global/etopo5.HTML (accessed on 23 March 2023)) at a spatial resolution of 1 arc minute.

2.3.3. Dynamic Weather Features

The ERA5 (https://www.ecmwf.int/en/forecasts/dataset/ecmwf-reanalysis-v5 (accessed on 28 March 2023)) dataset provided dynamic weather feature information from 2015 to 2021 at a spatial resolution of 0.25°. In this study, we selected three precipitation factors (the average precipitation: AvePre; the max precipitation: MaxPre; the number of days of precipitation exceeding 50 mm: MaxPre_Days), three wind factors, three cloud factors, six temperature factors, and three wave factors. Detailed information can be found in Table 1.

The bathymetry and dynamic weather features were resampled from the original resolution to our study unit of analysis (that is, 5 $\mathrm{m}\mathrm{i}\mathrm{l}\mathrm{e}\mathrm{s}\times 5 \mathrm{m}\mathrm{i}\mathrm{l}\mathrm{e}\mathrm{s}$ grids). The ship features are also statistics in 5 $\mathrm{m}\mathrm{i}\mathrm{l}\mathrm{e}\mathrm{s}\times 5 \mathrm{m}\mathrm{i}\mathrm{l}\mathrm{e}\mathrm{s}$ grids.

We calculated the influencing factor values for all the grids in the study area between 2016 and 2020. Figure 3 shows the value distribution. The maximum and mean ships of the grids are 933,623 and 3128, while the median is only 10, which indicates that most areas in our study have fewer ships, while some areas are the busy routes for ships. The MaxTemp_Days and MinTemp_Days have the same situations as that of the ship density. In contrast, other factors have similar values between the mean and median metrics. There were 160 days of wind speed exceeding 17.2 m/s between 2016 and 2020 for all the grids, with an average of 32 days per year, a standard deviation of 126, and the maximum days exceeding 749 days.

3. Methodology

Based on historical maritime accident data, the influencing factors, and the RF method, this study constructed three models: the monthly maritime accident prediction (MMAP) model, yearly maritime accident prediction (YMAP) model, and multi-yearly maritime accident prediction (M-YMAP) model. The three models mainly consist of the following seven steps: (1) the generation of feature matrixes; (2) the selection of the conditioning factors; (3) the generation of the accident and non-accident samples; (4) the generation of accident susceptibility models based on the RF method; (5) the evaluation of the performances of the models; (6) the development of accident susceptibility maps; and (7) an analysis of the importance of the influencing factors.

3.1. Random Forest Model

This study used the RF method to process this problem. An RF is a kind of ensemble learning that belongs to the bagging type [21,22]. By combining several weak classifiers, the final results are voted or averaged, which ensures the high accuracy and generalization performance of the results of the whole model. A random forest is composed of decision trees, which are the basic units of a random forest. Past research has found that the RF is very good in terms of the sensitivity of accidents; thus, the RF was chosen as the machine learning model [23,24].

3.2. Generation of Feature Matrixes

A feature matrix is a data table representing features and their corresponding values. The study area contained 31,269 grids. In the three models, ten dynamic influencing factors were extracted from the grids at different time scales as the feature matrixes, except for the static environmental features.

In the MMAP model, we established 31269 × 5 ×12 samples for 60 months from January 2016 to December 2020 (five years). For each month, we assessed the statistical variables, such as the ShipDensity statistic (the number of ships) and MaxTemp_Days (the number of days of temperature exceeding 35 °C), in a grid in a month.

In the YMAP model, we established 31,269 × 5 samples for five years from 2016 to 2020. For each year, we assessed the statistical variables, such as the ShipDensity statistic (the number of ships) and MinTemp_Days (the number of days of temperature lower than 0 °C), in a grid in a year.

In the M-YMAP model, we set the time scale as five years. We established 31,269 samples from 2016 to 2020. In this model, we assessed the statistical variables for five years, such as the ShipDensity statistic (the number of ships) and MinTemp_Days statistic (the number of days of temperature lower than 0 °C totally), in a grid from 2016 to 2020.

In this study, the ship features and dynamic weather features were processed in Python. We first encoded the AIS data using the Python Pyais package (pyais PyPI). The AIS data in an hour contain 5 million pieces of information. Therefore, to increase the efficiency and reduce the data volume, we extracted data for the 5th, 15th, and 25th of each month to represent the full month. Then, we statistically organized these influencing factors into different time scales. Static environmental factors (bathymetry, DisShore) were directly extracted from ArcGIS. Finally, all data were integrated to form feature matrixes in different time models.

3.3. Feature Selection

Before constructing the prediction model, executing a multicollinearity analysis is essential to ensure that no multicollinearity exists among the chosen factors [25]. The Pearson correlation coefficient is often used to quantify multicollinearity, which is calculated as follows:

$$r_{ij}=\frac{1}{n-1}\frac{\sum _{k=1}^m(x_{ki}-\overline{x_i})(x_{kj}-\overline{x_j})}{{\sigma }_i{\sigma }_j}$$

(1)

where $r_{ij}$ represents the correlation coefficient between factor $i$ and factor $j$; $x_{ki}$ and $x_{kj}$, $\overline{x_i}$ and $\overline{x_j}$, and ${\sigma }_i$ and ${\sigma }_j$ represent the values, mean values, and standard deviations of factor $i$ and factor $j$, respectively; $m$ represents the sample size; and $k$ represents the $k\mathrm{t}\mathrm{h}$ sample. The value of $r$ is between −1 and +1. A larger r value means the correlation between two factors is greater. In this study, one of the two factors could be excluded when the correlation coefficient was larger than 0.7 in the correlation matrix [24,26]. The relation between the variables was considered significant when the p-value was below 0.05.

3.4. Construction of Training and Testing Datasets

The application of the random forest to maritime accident susceptibility modeling is a binary classification issue, which includes accident and non-accident. From Figure 1, we can see that the accident area is much smaller than the non-accident area. There exist extreme imbalances between the positive and negative samples in the three models. There are 6909 accident samples in the MMAP model, 1820 in the YMAP model, and 1150 in the M-YMAP model, which account for 0.4%, 1.2%, and 3.7% of the total samples in the three models, respectively. RF algorithms can perform well in classification based on unbalanced samples [24,27]. In the three models, we set the ratio of positive and negative samples to be 1:2. From all the negative samples, 6909 × 2, 1820 × 2, and 1150 × 2 negative samples were randomly selected for the three models. Finally, we constructed the three models’ datasets with 20,727, 5460, and 3450 samples, which accounted for 1.1%, 3.5%, and 11.0% of the total samples in the three models, respectively. The three model datasets were divided into training sets and testing sets with a ratio of 8:2. The datasets were input into the abovementioned models. In addition to the validation using the 20% testing data, we also applied January, February, March, and April 2021 (blind test events) into the M-YMAP model to validate the model accuracy.

3.5. Evaluation Metrics

Assessing the model performance is necessary for constructing a machine learning model. In this study, we used the accuracy, precision, recall, F1-measure, receiver operating characteristic (ROC) curve, and area under the ROC curve (AUC) to evaluate the models’ prediction accuracies. The confusion matrix is the basis of the accuracy (A), precision (P), recall (R), and F1- measure (F1-m), and it contains four possible outcomes for a model: true positive (TP), true negative (TN), false positive (FP), and false negative (FN) [23]. TP and TN denote the numbers of accident and non-accident samples accurately classified, respectively. At the same time, FP and FN represent the numbers of misclassified samples and non-samples, respectively. Based on the confusion matrix, the accuracy, precision, recall, and F1-measure were calculated following the equations shown below:

Accuracy represents the proportion of accident sample points that are correctly classified among all the samples:

$$\frac{TP+TN}{TP+TN+FP+FN}$$

(2)

Precision denotes the fraction of true accident instances among the samples classified as positive:

$$\frac{TP}{TP+FP}$$

(3)

Recall refers to the ability to correctly classify accident points:

$$\frac{TP}{TP+TN}$$

(4)

And the F1-measure refers to the ability to minimize the misclassification of non-accident points when trying to correctly classify accident points:

$$\frac{2TP}{2TP+FP+FN}$$

(5)

Higher values of the accuracy, precision, recall, and F1-measure scores represent the better performance of the maritime accident prediction model. The AUC is a quantitative benchmark of the model accuracy, and the value belongs to the range of 0.5–1. The higher the AUC, the better and more accurate the model performance. AUC values in the ranges of 0.9–1, 0.8–0.9, 0.7–0.8, 0.6–0.7, and 0.5–0.6 can be categorized as excellent, very good, good, average, and poor performances, respectively [28].

4. Results

4.1. Correlation Analysis of Explanatory Factors

Figure 4 shows the Pearson correlation coefficients between the accidents and all the influencing factors. Most of the factor–accident pairs passed the significance test (p-value < 0.05), except for the AveLength, AveWidth, and MaxPre_Days in the YMAP model and the MaxPre_Days in the M-YMAP model. The ShipDensity had relatively high correlation coefficients with the response variables among all the factors in the three models, and the values reached 0.32 and 0.41 in the YMAP and M-YMAP models, which fully proves the key role of the ShipDensity in maritime accident prediction. In addition, there were strong positive correlations between the AveLength and AveWidth (r = 0.98) in the three models, as well as between the MaxWave_Days and MaxWind_Days (r = 0.81, 0.75, and 0.76, respectively, in the MMAP, YMAP, and M-YMAP models). Based on the correlations, we removed one of the factors to eliminate multicollinearity, which exceeded 0.7 with another factor. Finally, two factors, the AveWidth and MaxWind_Days, were excluded from further analyses.

4.2. Model Performance Analysis

Evaluating the model performance is a necessary step. From Figure 5, we can see that all three models performed well (AUCs > 0.9). The AUC value is the largest for the M-YAMP, which is 0.966. The MMAP and YMAP have the same AUC value, which is 0.957. The values of the accuracy, precision, recall, and F1-measure are also similar in the three models. The numbers are 0.906, 0.877, 0.849, and 0.862 in the MMAP model, 0.910, 0.885, 0.847, and 0.866 in the YMAP model, and 0.913, 0.879, 0.864, and 0.871 in the M-YMAP model, respectively. Additionally, Figure 5b shows the ROC curves for the three models on the testing dataset, which all performed well. The model performances show that these three models have high predictive capacities.

4.3. Generation of Accident Susceptibility Maps

4.3.1. Generation of Accident Susceptibility Maps

After validating the three models, we generated their corresponding accident susceptibility maps. The susceptibility was reclassified into five classes: 0–0.2: very low; 0.2–0.4: low; 0.4–0.6: moderate; 0.6–0.8: high; and 0.8–1.0: very high [24]. We compared the predicted and observed accident points to evaluate the model performances. Figure 6a–c show the MMAP model results. The susceptibility map of January 2017 shows that the high-susceptibility area is located close to the shore, especially from the Shanghai shore to the Guangxi shore. The August 2017 and August 2020 susceptibility maps show a similar pattern, while the Xisha and Nansha have high susceptibility compared with January 2017. Additionally, 97.6% of the existing accident points were located in the very high–high-susceptibility-class (H-class) regions in the MAMP model in January 2017, and the number reached 98.63% and 95.56% in August 2017 and August 2020, which indicates that accident points can be identified with high probability using the MMAP model. When looking at the YMAP and M-YMAP susceptibility results, we can see that the yearly and multi-yearly susceptibility maps have the same pattern as the monthly maps. The H-class areas are close to shore for the yearly and multi-yearly susceptibility maps, while the areas of the H-class are larger than those of the monthly results. Meanwhile, the YMAP and M-YMAP models also obtained their significantly predicted capacities, as shown in Figure 6f,h. The average percentage of observed accidents at very high susceptibility levels from 2016 to 2020 exceeded 80% using the YMAP model, and these numbers reached 82% using the M-YMAP model.

4.3.2. Generation of Accident Susceptibility for Blind Data

We tested the robustness of the established MMAP model by performing a blind test using January–April 2021 maritime accidents, which were not used in the training process. We chose the accident point and randomly chose a 2-fold non-accident point for performing the testing, and we repeated this process ten times. The performance results are shown in Figure 7. The mean values of the AUC for four months are 0.956, 0.955, 0.947, and 0.951, respectively, and the mean values of the accuracy, recall, precision, and F1- measure exceed 0.8. From Figure 8, we can see that the susceptibility maps also show relatively good results. In January 2021, most accidents fell into the H-class, with a percentage of 85.23%, which were 81.78%, 79.41%, and 79.41% for February, March, and April 2021. From January to April 2021, most accidents matched well with the predicted high-susceptibility areas. From the blind test of the MMAP model, we can see that the model has reasonably good results when applied to future maritime accidents.

4.4. Influencing Factor Analysis

Figure 9 shows the relative importance of each accident-influencing factor in the three models. The conditioning factors in the three models had similar sorting. The ShipDensity was the most critical factor in the three models. The percentage was around 25% of the total information contribution, accounting for 23.91%, 25.10%, and 26.47% in the MMAP, YMAP, and M-YMAP models. Bathymetry was the second-highest factor in the three models, exceeding 20% of the total information contribution. Stranding accidents usually occurred in the low-water-depth areas. In addition, the DisShore was the triggering factor of the touch-rock accidents. It was ranked fourth, third, and third in the MMAP, YMAP, and M-YMAP models. Furthermore, the MaxWave_Days was the most critical factor among the dynamic weather features, which shows that the influence of waves and wind (the wind factors have strong positive correlations with waves) on the occurrence of accidents cannot be ignored. Temperature and precipitation had relatively less importance in the models among the influencing factors.

5. Discussion

In this study, we established multi-period maritime accident susceptibility models based on the RF algorithm. The models showed good performances according to the accuracy measures in the testing data results (Figure 5 and Figure 7). Some studies [3,19,27] on maritime accident risk and susceptibility predicted using machine learning algorithms have been carried out. Yang [3] used machine learning technology based on AIS data to assess whether a grid area is an accident-prone area and to predict the accident severity in a grid of the Fujian Sea area from 2008 to 2020. Nourmohammadi [19] proposed spatiotemporal deep learning models for predicting the occurrences of future daily, weekly, and monthly accidents based on the water depth, V-pass, and weather datasets of the territorial sea of South Korea from May 1, 2018, to April 30, 2019. The above studies ignored the future predicted time scale and dynamic changing of the conditions [3], or the prediction rate was not adequately rapid for short-term predictions using data over one year [19]. Therefore, our study produced multi-time-scale maritime susceptibility models using the RF method with dynamic AIS data and weather features.

5.1. Cost–Benefit Analysis

This study set the ratio of accident and non-accident samples to be 1:2 to avoid overfitting via the high sample imbalance based on most studies [3,24]. Under this setting, we obtained MMAP, YMAP, and M-YMAP models with excellent performances and predictive capacities. Taking the MMAP model as an example, the percentages of observed accidents falling in the H-class were 88.24%, 81.78%, 79.41%, and 79.41% for January 2021, February 2021, March 2021, and April 2021 (Figure 8). However, the accompanying problem is that the area judged as H-class is relatively large in the future prediction (Figure 8), accounting for 7.67% of the study area in January 2021, which means that we need to search a large-area grid to find one accident grid. In the following, we analyze the changing of the predictive capacity and sensitivity classification by adjusting the ratio of accidents and non-accidents in the MMAP model. In addition, we defined the cost–benefit ratio (cbr) to measure how many grids need to be checked to detect an accident grid, which is similar to the inverse-recall metric. The equation is shown as follows:

$${cbr}^s=\frac{{acc}_n^s+no{-acc}_n^s}{{acc}_n^s}$$

(6)

where ${cbr}^s$ is the cost–benefit ratio of susceptibility class m, and ${acc}_n^s$ and $no{-acc}_n^s$ are the numbers of accident and non-accident grids in class $s$.

Figure 10 shows the model results (i.e., the evaluation metrics, the different susceptibility class area percentages of the study area, the percentages of the observed accidents at different susceptibility classes, and cbrs) changing with the ratio of non-accident and accident samples in the MMAP models in January 2021. When increasing the ratio of non-accident and accident samples from 2 to 19, we found that there was no significant change in the accuracy or AUC metrics. In contrast, the precision, F1-measure, and recall metrics dropped a lot (Figure 10a), and the percentage of observed accidents falling in the H-class also decreased, from 88.24% to 41.18% (Figure 10c). In addition, the percentage of area judged as H-class dropped from 7.67% to 0.99% (Figure 10b), and the cbr also decreased for the H-class (Figure 10d) when the ratio changed from 2 to 19. From these analyses, we can see that when we increase the ratio of non-accidents and accidents, the cbr can decrease, while the total accidents identified in the H-class will be lower. When we need to extract 80% of the accidents in the H-class, the minimum cbr is 50 for ratio 2; when we need to extract 70% of the accidents, the minimum cbr is 50 for ratio 2; when we need to extract 70% of the accidents, the minimum cbr is 49 for ratio 5. In some aspects, we try to minimize the costs and maximize the benefits. However, in maritime risk management, the higher goal is to decrease accidents by increasing the intensity of the investigation. Therefore, we recommend choosing the appropriate parameters according to the needs based on the cbr in Figure 10d.

5.2. The Limitations of This Study

Several assumptions and limitations are acknowledged in this study. First, due to the limitation of the accident data, we could not find more information about the accidents. There was a total of 1407 “Others” type of disasters, of which there were 859 accidents directly marked as “Other” in the dataset, which we could not find more information on to divide them into the nine accidents; for the other 548 accidents, most of them are marked as “Ship damage”, and we could not find the causes for the damage. Therefore, we put them all in the “Others” type of disaster. Second, AIS data contain 5 million pieces of information in an hour. To increase the efficiency and reduce the data volume, we extracted data for the 5th, 15th, and 25th of each month to represent the full month. Third, due to the AIS data being unavailable in the future, this study does not give future susceptibility maps. Some researchers [29,30] have generated methods to predict the ship flow. In the future, we will combine these methods into our models to assess the accident susceptibility in the future. What is more, in order to avoid overfitting via the high imbalance of the sample, we set the ratio of accident samples and non-accident samples to be 1:2, which resulted in the cost–benefit ratio (cbr) being too large (Figure 10d). In the next step, we will collect more detailed accidents to improve the models. In addition, we suggest that maritime management choose the appropriate parameters to cope with the need.

6. Conclusions

In this study, we established a monthly maritime accident prediction (MMAP) model, yearly maritime accident prediction (YMAP) model, and multi-year maritime accident prediction (M-YMAP) model for maritime accident susceptibility assessment based on the RF method. The conclusions are summarized as follows:

• The results showed good performances according to the accuracy, recall, precision, F1- measure, ROC, and AUC values in the testing data and blind data;
• In addition, the monthly, yearly, and five-yearly susceptibility maps show similar patterns. The high-susceptibility areas are close to the shore, especially from the Shanghai shore to the Guangxi shore;
• Meanwhile, the conditioning factors in the three models had similar sorting. The ship density and bathymetry were the most critical factors in the three models, contributing around 25% and 20% of the total information.

A cost–benefit analysis was performed in the discussion part, and we suggest that maritime management choose the appropriate model parameters according to their needs. The associating output results can be used for route planning and resource allocation in marine management.