Next Article in Journal
Sensor-Based Measurement Method to Support the Assessment of Robot-Assisted Radiofrequency Ablation
Previous Article in Journal
Passive Biotelemetric Detection of Tibial Debonding in Wireless Battery-Free Smart Knee Implants
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Unmanned Surface Vehicle Thruster Fault Diagnosis via Vibration Signal Wavelet Transform and Vision Transformer under Varying Rotational Speed Conditions

1
Department of Mechanical Engineering, Korea Maritime & Ocean University, Busan 49112, Republic of Korea
2
Interdisciplinary Major of Ocean Renewable Energy Engineering, Korea Maritime and Ocean University, Busan 49112, Republic of Korea
3
Advanced-Intelligent Ship Research Division, Korea Research Institute of Ship & Ocean Engineering, Daejeon 34103, Republic of Korea
4
Maritime R&D Center, LIG NEX1 Co., Ltd., Seongnam-si 13488, Republic of Korea
5
Marine Domain & Security Research Department, Korea Institute of Ocean Science and Technology, Busan 49112, Republic of Korea
*
Author to whom correspondence should be addressed.
Sensors 2024, 24(5), 1697; https://doi.org/10.3390/s24051697
Submission received: 4 February 2024 / Revised: 23 February 2024 / Accepted: 28 February 2024 / Published: 6 March 2024
(This article belongs to the Section Sensors and Robotics)

Abstract

:
Among unmanned surface vehicle (USV) components, underwater thrusters are pivotal in their mission execution integrity. Yet, these thrusters directly interact with marine environments, making them perpetually susceptible to malfunctions. To diagnose thruster faults, a non-invasive and cost-effective vibration-based methodology that does not require altering existing systems is employed. However, the vibration data collected within the hull is influenced by propeller-fluid interactions, hull damping, and structural resonant frequencies, resulting in noise and unpredictability. Furthermore, to differentiate faults not only at fixed rotational speeds but also over the entire range of a thruster’s rotational speeds, traditional frequency analysis based on the Fourier transform cannot be utilized. Hence, Continuous Wavelet Transform (CWT), known for attributions encapsulating physical characteristics in both time-frequency domain nuances, was applied to address these complications and transform vibration data into a scalogram. CWT results are diagnosed using a Vision Transformer (ViT) classifier known for its global context awareness in image processing. The effectiveness of this diagnosis approach was verified through experiments using a USV designed for field experiments. Seven cases with different fault types and severity were diagnosed and yielded average accuracy of 0.9855 and 0.9908 at different vibration points, respectively.

1. Introduction

In maritime environments, systems are more susceptible to faults than in other environments due to environmental perturbations [1,2], corrosive conditions [3,4], and floating debris [5,6,7]. Moreover, recent research on navigational efficiency [8,9], use of renewable energy [10,11,12,13,14], situation awareness [11,15], improved communication methods [16,17], etc., applied to surface platforms is broadening spatial and temporal boundaries for unmanned surface platforms. Hence, with the anticipated expansion of unmanned vessels’ operational scope in the near future, if a vessel becomes immobilized due to a malfunction in the middle of the ocean, the time and cost for retrieval can be substantial. Therefore, it is imperative to undertake research to automate the condition monitoring and fault detection that were previously carried out by onboard ship engineers.
For conventional ships, maintenance regulations are well organized. A daily checkup on sensor values and checklist confirmation is routinely performed. Also, periodic maintenance, overhauls, and regulations of necessary replacement parts on board enable a ship to maintain long-term operation and maintenance [18]. However, most Unmanned Surface Vehicles (USVs) follow scheduled maintenance and reactive strategies, and most fault-related research is heavily focused on commercial-class surface vessels [19,20,21,22].
Within the components constituting USVs, the integrity of the propulsion system is a decisive factor for the successful completion of missions. Yet, owing to its direct interaction with the marine environment, this system is inherently susceptible to a broad spectrum of failures, prominently including sudden disruptions induced by external factors. The capability to efficiently identify and categorize these faults during maritime operations is crucial, as it enables the formulation of adaptive strategies in alignment with the system’s level of autonomy and redundant thrust capabilities [23]. Given these considerations, this research aims to diagnose the types and magnitude of underwater thruster faults of USVs.
However, if the system is not fabricated with consideration for fault diagnosis, it is challenging to measure parameters such as rotational speed, power consumption, and voltage in real-time without making alterations to the system. These parameters, which are directly connected to the vehicle’s control and propulsion, are difficult to apply as diagnostic parameters without prior integration in the system design. Additionally, for smaller vehicles not subject to the mandates of the Automatic Identification System (AIS), the lack of suitable Global Navigation Satellite System (GNSS) equipment poses an obstacle to the effective real-time measurement and utilization of dynamic state data [24].
Therefore, this study aims to employ vibration-based fault diagnosis as a non-invasive and sensitive method that allows the acquisition of high-quality data in real-time without any alterations to the system or internal wiring. Vibration sensors offer a cost-efficient alternative for acquiring diagnostic features when compared to other parameters such as power consumption, supply voltage, propeller blade rotation speed, and high-accuracy GNSS sentences. Fault diagnosis based on frequency analysis of vibration data has been frequently utilized in various fields due to its simplicity and the ability to observe the characteristics of different frequency components associated with faults [25,26]. The most commonly used thrust source for surface vehicles, the underwater thruster, is a type of rotating equipment that operates on rotational motion. Thus, the use of vibration-based fault diagnosis becomes straightforward in the ease of identifying the fundamental frequency (first-order vibration, 1X component), which occurs at the same speed/frequency as the rotor [27] (p. 42).
While the direct attachment of vibration sensors to thrusters facilitates the acquisition of characteristic data conducive to fault diagnosis, implementing sensors within the hull’s interior is a more pragmatic strategy for general applications. The hull’s interior comprises structures with disparate stiffness and mass, leading to a complex array of resonance frequencies and harmonics [27,28] (pp. 275–279). This complexity makes the collected data notably noisy and challenging to analyze. Hence, under these conditions, the identification and isolation of faults with varying thruster speeds pose an intricate problem [29]. Nevertheless, data for this study was acquired according to a driving profile that entailed a ten-second acceleration from a static state to maximum rotational speed, followed by a subsequent ten-second deceleration phase. This driving profile was used not just to facilitate the diagnosis of faults across the entire spectrum of the thruster’s possible rotational speed outcomes but also to enable a variation in the spectrum of frequencies applied for fault detection. This approach enhances the capability of structural frequency analysis, allowing for the examination of not only the thruster but also the joint condition or damage of other structures, thereby extending its functional scope beyond diagnosing faults at fixed, specific speeds [29,30] (pp. 249–259).
To address the inherent challenges in fault diagnosis via hull vibrations and to diagnose faults at varying rotational speeds of USVs’ propulsion systems, the Continuous Wavelet Transform (CWT) was applied to convert time-series vibration data into scalograms [30,31]. Although a number of methodologies have been applied for fault diagnosis at a constant rotation speed of USV thrusters [32,33,34], analyzing varying rotational speeds demands a methodology that simultaneously accounts for attributes in both the temporal and frequency domains. CWT, renowned for its ability to encapsulate both the physical characteristics and time-frequency domain nuances, has been extensively researched and validated within the realm of Physics-Informed Neural Networks (PINNs) [35]. Hence, scalogram images obtained as an output of CWT were applied as an input for deep neural network (DNN) classifiers. Leveraging the widely applied and validated DNN techniques in image classification through transfer learning. A particular emphasis was placed on the application of the Vision Transformer (ViT) model, which is known for its ability to increase global context awareness and transfer learning efficiency [36]. This paper also proposes to present the critical considerations and analytical reflections required during experimental procedures involving varying thruster rotational speeds and the subsequent application of a DNN classifier. A USV with wireless DAQ (Data Acquisition) and a remote-control system was fabricated to experiment with the viability of the approach. The experimentation was conducted at sea, encompassing the collection of both normal status data and data under simulated fault cases. Common fault scenarios that are attributed to external factors were selected as fault cases [32,37,38,39]. The experimental results of this study aim to demonstrate the proposed method’s effectiveness as a non-invasive technique capable of not only identifying the type but also the extent of the faults. The study illustrated some results of wavelet transformations conducted on vibrations transmitted to the interior of a hull in marine environments and analyzed the physical characteristics across the time-frequency domain. The study quantitatively evaluates the acuity of a wavelet-based DNN classifier and discusses potential avenues for future research.

2. Related Works and Backgrounds

This chapter examines fault diagnosis research, focusing on underwater thrusters and unmanned maritime platforms. In addition, this chapter intends to provide an overview of previous research conducted on the platforms analogous to the ones used in this study and supplement the conceptual basis of the vibration-based approaches employed in this research.

2.1. Related Works

2.1.1. Fault-Related Research on Unmanned Marine Platforms

Unmanned marine platforms can be categorized into various types, with three primary groups depicted in Figure 1. USVs navigating the water’s surface, Autonomous Underwater Vehicles (AUVs) functioning wirelessly, and Remotely Operated Vehicles (ROVs) that operate underwater tethered by cables. Due to its unique operating methods, areas of activity, and structural differences, each platform requires different methodologies and approaches for fault detection and diagnosis.
ROVs commonly adopt an over-actuated system (more actuators than the minimum required for controlling all its degrees of freedom) for enhanced maneuverability, stability in various underwater tasks, and increased payload capabilities. Their design allows for power supply via an umbilical cable, affording greater flexibility in the number of thrusters utilized compared to other marine platforms and offering the substantial benefit of ease in recovery in the event of a malfunction. Consequently, there has been a vigorous pursuit of research in fault tolerance using remaining thrusters in cases of individual thruster failures [23,39,40].
AUVs, particularly those designed as torpedo-type, are generally non-over-actuated. Even AUVs with specialized designs and objectives are conservatively designed in terms of thruster quantity due to the limitations of continuous energy supply for prolonged operations. As wireless platforms, AUVs encounter communication challenges with operators, leading to active research in risk analysis and reliability, focusing not only on fault diagnosis but also on sustained operational stability [39,41].
In contrast, USVs, unlike their submergible counterparts, benefit from the feasibility of real-time positioning via GNSS and radio communications, making their retrieval comparatively straightforward. Hence, research in this domain has primarily focused on navigation and guidance rather than fault diagnosis-related issues involving field testing [9,42,43]. However, as recent technological advancements extend the spatio-temporal capabilities of USVs, they will also increase difficulties associated with retrieval distance and cost. As such, automated condition monitoring, fault detection, and prognosis will become compelling areas for research.

2.1.2. Underwater Thruster Stand-Alone Fault Detection and Diagnosis Research

Numerous studies have been reported on detecting and diagnosing underwater thruster faults and their stand-alone experiments. Fault diagnosis was performed using a small underwater thruster’s current and underwater acoustic data as features, employing a 1D-CNN (Convolutional Neural Network)-based deep learning structure and visualization through t-SNE (T-distributed Stochastic Neighbor Embedding) [33]. A thruster model was driven based on the control input, rotation speed, and current of the thruster, and the residual between the thruster model and actual data was used as a feature to compare classification performance through an MLP (Multi-Layer Perceptron) and LSTM (Long Short-Term Memory)-based classifier [44]. A fault diagnosis method was proposed using both current-rotational speed correlation analysis and support vector machines [45]. A test bed and data collection system for thruster experimentation were used and research was conducted on fault diagnosis using underwater acoustic data. Clustering and frequency analysis-based methods were used to identify fault features [34].
This study aims to diagnose faults through signals from vibration sensors attached to the hull. Hence, this research requires a method of vibration-based fault diagnosis that is sensitive and capable of handling noise, taking into account the structural influence and the damping effect of the fluid-contacting bottom surface. Hence, USV, which is capable of DAQ and field tests, was introduced to acquire actual noisy vibration data.

2.2. Backgrounds

This section provides an overview of previous research on the platform analogous to the one used in this study and supplements the conceptual basis of the vibration-based approaches employed in this research. The platform is designed for Fault Detection and Diagnosis USV (FDD USV) and has been revised for different research purposes. The FDD USV used in a series of previous studies and for this research is shown in Figure 2.
In prior studies involving FDD USV, extensive analyses were conducted on data parameters such as rotational speed, power consumption, supply voltage, and vibration to determine thruster malfunctions due to external causes. It was determined through experimental validation that, among these parameters, vibration presented the highest degree of sensitivity for fault identification [46]. Further research in FDD USVs also involved the application of dimension reduction and entropy-based numerical transformation techniques for each fault case [47]. The significance of vibration data was not only underscored in the context of FDD USV’s previous works but also broader applications involving the fault diagnosis of various rotational machines [26]. When factors such as propeller blade damage, temporary obstructions, or biological fouling occur, they cause a shift in the center of gravity away from the geometric center aligned with the propeller blade’s rotational axis. This shift results in a misalignment of the geometric center, leading to an imbalance in the rotating body [46]. This rotational imbalance acts as a vibromotive force, which is then transmitted through the structural components of the hull and detected by vibration sensors mounted inside.
The previous experiments with FDD USV were conducted in calm and controlled environments like engineering water tanks, focusing on results under consistent control inputs (rotational speeds). These conditions ensured isolation from external marine disturbances such as wind or waves, allowing fault identification within specific rotational speed ranges. In contrast to certain land-based rotational machinery, USVs frequently encounter scenarios where their rotational speeds are subject to continual variations during operation. Consequently, this research incorporates a fault diagnosis methodology that effectively copes with environmental perturbations and adapts to environments with fluctuating rotational speeds. This is achieved through the application of CWT and ViT techniques, demonstrating an advanced approach suitable for the dynamic operating conditions of USV.

3. Methods

This chapter provides a detailed explanation of the components that make up the workflow to achieve the objectives of this study, as well as an overview of the entire workflow process.

3.1. Fault-Induced Vibromotive Force

The study explicitly targets faults like propeller blade damage and debris entanglement. Such faults cause a shift in the center of mass of the thruster blades, leading to rotational unbalance (rotor imbalance) and vibration. The concept of rotor imbalance is illustrated in Figure 3, with formulas representing the centrifugal force due to mass displacement written in Equation (1) and the resulting vibrations detailed in Equation (2).
F c = m r ω 2
V = F c k s t r
In the formula, F c represents the centrifugal force caused by imbalance, r is the distance between the geometric center and the shifted center of mass, ω denotes angular velocity, and m indicates the imbalance mass resulting from damage or entanglement, referring specifically to the weight of a blade in case of blade breakage. V Signifies the vibration caused by imbalance, while k s t r represents the structure’s dynamic stiffness. The magnitude of vibration caused by rotor imbalance is proportional to the imbalance mass and the square of the angular velocity [46]. The vibration frequency corresponds to the rotational speed of the body, denoted as the first-order vibration, fundamental vibration, 1X component, etc. In a normal thruster, the rotor imbalance is minimal, leading to a smaller magnitude of the first-order vibration. However, in the event of entanglement or relevant faults, the first-order vibration’s magnitude increases. An illustration of the difference between normal conditions and fault conditions is shown in Figure 4. The figure illustrates the difference between normal and faulty thruster raw vibration data and their first-order vibration. Following the onset of a fault, the vibration signal reveals an enhanced periodicity that aligns with the blade’s rotational speed. The Fourier transform outcomes further substantiate the amplification of the first-order vibration, indicating a direct correlation between the fault condition and the specific vibrational pattern observed.

3.2. Wavelets and Continuous Wavelet Transform

Wavelet is a short-lived wavelike oscillation localized in time. To be defined as a wavelet, two essential conditions must be satisfied [30]
  • Admissibility condition
As shown in Equation (3), it means that the wavelet has no zero-frequency component or that the wavelet ψ ( t ) must have a zero mean.
ψ ( t ) d t = 0
Interestingly, sinusoidal functions, which are fundamental elements of the Fourier transform and serve a contrasting purpose, also have a zero mean over the entire interval. The second condition that distinguishes sinusoidal functions from wavelets is shown in Equation (4).
2
Finite energy
The wavelet ψ ( t ) must have zero energy.
| ψ ( t ) | 2 d t <
Having finite energy, wavelets possess the property of being localized in time. This is in contrast to Fourier transforms, which decompose infinite sinusoidal functions. Because wavelets exist within a localized time frame, they can capture information in both time and frequency domains. This study employs the CWT, a wavelet application, to generate scalogram images. The mathematical representation of CWT is formulated as Equation (5). This method is integral to the research’s objective of analyzing data across both time and frequency dimensions, providing a comprehensive analysis encompassing physical characteristics caused by faults.
T ( s , τ ) = f ( t ) ψ s , τ * d t = f ( t ) , ψ s , τ  
Here, f ( t ) is the signal of interest to be transformed and ψ s , τ * is a complex conjugate of the wavelet transform. Wavelet ψ s , τ scaled (s) and translated (τ) over time, which can be represented as shown in Equation (4). Adjustments in the scale parameter s , facilitate the acquisition of frequency data, while modifications in the translation parameter τ , enable the gathering of temporal domain information. Wavelet ψ s , τ in Equation (6) is a normalized form without consideration of complex-part.
ψ s , τ = 1 s ψ ( t τ s )
T ( s , τ )   is a convolutional result between the signal of interest f ( t ) and ψ s , τ . This outcome reflects the degree to which the scaled and translated wavelet ψ s , τ contributes to composing the signal. As such, T ( s , τ ) serves as an indicator of the correlation between the original signal f(t) and ψ s , τ , with higher values (attributes to the brightness in the scalogram) denoting more remarkable similarity and lower values (darkness in the scalogram) indicating lesser similarity. The generation of a scalogram image is achieved by varying the values of s and τ and plotting these convolution results in alignment with time-frequency space.
The choice of wavelets varies across applications, with this study employing the Morlet wavelet, known for its real part’s resemblance to a sinusoidal wave overlaid with a Gaussian distribution. This wavelet has been widely used in different fault diagnosis applications due to its similarity to mechanical vibration patterns [48]. The formulation of the Morlet wavelet is presented in Equation (7).
ψ ( t ) = π 1 / 4 ( e i 2 π f 0 t e ( 2 π f 0 ) 2 / 2 ) e t 2 / 2
An example of a scalogram generated from the results of the CWT is shown in Figure 5.

3.3. Vision Transformer (ViT)

In the realm of image analysis, deep neural network (DNN) methodologies have been rigorously developed, enabling the utilization of pre-trained models across a broad spectrum of applications. Among these developments, the ViT, which utilizes transformer architecture: a structure initially emerging in natural language processing, has been adapted for vision-related tasks and is gaining traction in various disciplines. The ViT marks a departure from traditional CNN usage in vision tasks, instead leveraging the Transformer’s multi-head attention (self-attention) architecture to achieve significant performance. The ViT partitions an image into patches, processes each patch as a token, equivalent to a word in a sentence, and then employs a standard transformer architecture to process these tokens. The architectural framework of ViT is illustrated in Figure 6 [36]. Vibration scalogram datasets for both normal and various fault states from maritime experiments were stored according to their respective case labels. Then, the network’s head was then replaced with a new one tailored for the thruster fault diagnosis task, followed by fine-tuning on this new dataset.
In the ViT framework, the most distinctive aspect is segmenting images into patches and their subsequent arrangement into sequences, known as position embedding. This attributes ViT to various benefits, notably global context awareness and enhanced efficiency in transfer learning. ViT has been empirically validated to outperform many conventional CNN-based DNN classifiers under specific prerequisites [36,49]. Figure 7 depicts the dataset following a rotational speed-varying driving data profile intended for use in this research and sample data.
The driving profile entails an acceleration from a static state to maximum rotational speed, followed by a subsequent deceleration phase. As shown in Figure 7b, the magnitude of vibration does not increase in tandem with increasing rotational speed. Furthermore, the presence of the structure’s main natural frequency plays a significant role in analyzing the vibratory temporal response. Hence, ViT was selected as the foundational architecture for the image classifier in this study, attributing to its capacity to assimilate characteristic information relative to location.

3.4. Experiment Settings

This research verified the methodology’s effectiveness through sea trials using a modified FDD USV capable of remote, real-time data acquisition. The control system configuration of the FDD USV used in this study is depicted in Figure 8.
The workflow of the remote control and the DAQ system through the FDD USV is illustrated in Figure 9.
The design was separated into two primary sections: the DAQ section for data collection and transmission and the control section for overall management. The control section incorporated a Motion control board with individual thruster control, path tracing control, dynamic positioning functions, and a power control board for managing various sensors and the device’s power distribution. The motion control board was integrated with the NI compact DAQ system and a microcontroller unit (MCU) to facilitate remote data acquisition at user-defined times and intervals.
The detailed experimental procedure is as follows. Initially, the power and communication status between the ground control console and the FDD USV are checked to ensure smooth operation. Then, the function of the thruster’s operational state and essential navigation performance are checked, including the status of the GNSS equipment used to track the platform’s position during experiments. Additionally, the operation of the rendezvous mode, which returns the platform to the launch position if the navigation control loop stops for any reason or if communication from the shore is interrupted for a certain designated period, is checked.
The DAQ process is primarily conducted from the onshore control console. The control console connects to the NI DAQ equipment’s IP address and control section IP address through an AP device that relays the connection between the shore and the FDD USV. The control section controls navigation systems and thrusters to execute the desired data profile. The NI DAQ equipment operates via the National Instruments’ FlexLogger 2021 R1 program and is connected to the control section using a hardware triggering method to repeatedly capture data at consistent intervals. This hardware triggering allows navigation software to execute control for the data profile while starting data logging. It stops the DAQ equipment’s logging function once control for the data profile is completed. The navigation program and thruster control software are programmed in C language on the control section’s Micro Controller Unit (MCU) and the GUI program on the onshore control console, which executes control commands developed in C#.
Additionally, to automate the acquisition of repetitive data and dataset creation, a macro program was developed in C# to automate the generation of data files through the FlexLogger program, execution of the NI DAQ equipment, thruster operation according to the data profile, stopping of DAQ equipment, and extraction of acquired data. The program to convert the acquired data into CSV format, preprocessing, and iterating the Continuous Wavelet Transform (CWT) was written in MATLAB R2023b. The program stores the data for each fault case in separate folders and automatically processes and performs CWT on the data within each folder, creating scalograms for each. As a result, separate folders containing only the scalogram images for each fault case are created. Finally, folders named after each fault case can be processed through a MATLAB-written ViT transfer learning program to obtain classification results.
Specific faults, such as blade breakage and entanglements, typically resulting from external factors, were synthetically induced to simulate the dataset for the experiment. Simulated faults were applied to the starboard side thruster for all cases. The data acquisition was executed by a data profile comprising ten-second intervals of acceleration and deceleration each. Figure 10, in the study, provides a visual representation of both the normal and propeller fault cases utilized in these experiments. The results of the CWT corresponding to the sequence shown in Figure 10 are presented in Figure 11.
Figure 10b,d,f depict cases simulating breakage faults in propellers, with damages of 7%, 14%, and 21% relative to the radius of the propeller. Figure 10c,e,g represent entanglement-type faults, showcasing scenarios with a thin rope and a rope of different diameters entangled, as well as a net entanglement.
Figure 11 shows that all cases depict first-order vibration aligning with the rotational speed variations in the data profile, yet breakage cases and entanglement cases have discernable differences. In scenarios involving breakage, a pronounced enhancement of the first-order vibration is evident. In contrast, during entanglement events, the first-order vibration remains detectable; however, characteristics such as its inability to exceed a certain threshold due to speed-induced overload become apparent. Additionally, entanglements involving turbulent interactions with water lead to a noticeable increase in residual vibration components, yielding distinct and insightful outcomes across varying circumstances. Still, distinguishing between the normal state shown in Figure 11a and the minor faults depicted in Figure 11b,c remains challenging. Furthermore, in the case of breakage, the emphasis on the first-order vibration makes it difficult to differentiate between them visually.
The study faced challenges in maintaining a uniform sample size across the datasets due to difficulty replicating severe faults, experimental disruptions caused by spontaneous naturally occurring faults, as shown in Figure 12, and adverse weather conditions. The dataset sizes for each case are listed in Table 1.
Each dataset outlined in Table 1 was converted into scalogram images and subsequently transformed into RGB images of dimensions 384 × 384 × 3, aligning with the ViT’s input layer specifications. These images were segregated into training, testing, and validation sets for targeted applications within the ViT framework. Utilizing a pre-trained ViT model, adaptations were made to the input and output layers to align with the transfer learning objectives of this research. Figure 13 illustrates the comprehensive framework for fault diagnosis, as designed to achieve the objectives of this research.

4. Results and Discussion

The analysis of the results will focus on the classification outcomes and accuracy of the wavelet and Vision Transformer (ViT), as well as the confusion matrix. The study concentrated on the results from two sensors located inside the hull. The attachment positions of these two sensors are depicted in Figure 14. One sensor, labeled as vib No.4, was affixed upper deck, near the structure for fixing the starboard side thruster that simulated faults. The other sensor, vib No.9, was attached inside the container housing the DAQ system.
Only the vibration data from sensors vib No.4 and vib No.9 were selectively stored from the seven types of maritime experiment data. After transforming these data into scalograms, they were used to train a wavelet ViT classifier. The execution and transfer learning process of the ViT was conducted through MATLAB 2023b. The base model used for transfer learning was the “base-16-imagenet-384” from MATLAB, with 86.8 million learnable parameters. The adjustable parameters (hyperparameters) of the wavelet ViT classifier model utilized in this research are shown in Table 2.
The sample results of this classification are displayed in Figure 15 and Figure 16. Repeated results for each model are shown in Table 3.
The result demonstrated its capability of identifying minor faults, such as 7% breakage and thin rope entanglement, which were previously challenging to discern using other FDD USV-related research with different sensors and methodologies. This illustrates the classification methodology’s efficacy across various operational conditions.
The study aimed to classify maritime data but also incorporated more challenging on-ground experiment data into the training of the same model. The test settings and sample scalogram results are depicted in Appendix A, Figure A1. The sample results of this extended analysis, which includes both maritime and on-ground data, are presented in Figure 17 and Figure 18. Repeated results for each model are shown in Table 4.
The integration of on-ground and maritime data for fault diagnosis resulted in a noticeable decrease in overall accuracy. Given that the primary objective of this research is the diagnosis of faults within maritime data, a detailed examination of this phenomenon has been provided in Appendix A.
An interesting discussion point is that despite requiring significantly more training time under similar hyperparameter conditions, the ViT did not show a noticeable difference in thruster scalogram classifying accuracy compared to traditional CNN-based classifiers like ResNet50 or VGG counterparts. In an effort to explore this matter, repeated training of both CNN and ViT models was considered, but it was beyond the scope of this paper and challenging to discern whether the lack of significant performance difference was due to dataset size or the inherent complexity of the task.
During the training of the wavelet ViT model with maritime and on-ground datasets, it was observed that training accuracy tends to decrease when the number of epochs exceeds 14, likely due to dataset insufficiency and resultant overfitting issues. Similar to considerations regarding comparisons with CNN models for USV thruster diagnostics, further research seems necessary upon acquiring more data. Future studies will also aim to publish data not only from various vibration sensors but also include thruster-specific current consumption, voltage, noise, and USV states (position, velocity, and acceleration) after augmenting the dataset.
However, the study observed sensitive responses from the wavelet-based learning method to faults in marine platform thrusters and confirmed that ViT effectively distinguishes scalograms depicting the time-frequency distribution of vibrations transmitted inside the hull from underwater thrusters. Expanding this research into anomaly detection, which requires fewer additional datasets, could lead to more general and practical studies.

5. Conclusions

The underwater thrusters of USVs are crucial for mission execution yet, due to their direct engagement with the marine environment, rendering them susceptible to faults. A non-invasive and cost-effective vibration-based methodology that does not require altering existing systems is employed to diagnose thruster faults and to identify faults across all rotational speeds of the thrusters, not just at stationary speeds.
This research focused on diagnosing thruster faults due to environmental factors, explicitly targeting thruster blade breakage and debris entanglement. These issues lead to a shift in the center of mass, resulting in rotational imbalance and subsequent vibrations. The study highlighted the challenge of signal attenuation and noise introduction as vibrations traverse through varied materials to the hull. A data profile was employed to address the dynamic range of rotational speeds, underscoring the limitations of conventional Fourier-based vibration analysis methods in diagnosing faults within such fluctuating frequency data.
Therefore, this study applied wavelet transform to acquire insights into the distribution of frequency components aligned with variations in rotational speed and temporal changes. The CWT, a specific wavelet technique, was utilized to transfigure one-dimensional vibrational time-series data into scalogram representations. These scalogram images were then diagnosed for normal and various fault conditions using a ViT-based classifier. To empirically substantiate the efficacy of the proposed methodology, a custom USV (FDD USV) was fabricated for experimental trials at sea, conducting maritime experiments and facilitating the direct examination of vibrational data attributes and scalogram configurations under varied operational and fault-induced scenarios.
The FDD USV was engineered for both maritime navigation and wireless acquisition of large-scale data to a land-based control console. The research includes acquiring experimental data across seven maritime fault scenarios, including normal operations, by following a specific data profile of accelerating and decelerating the thruster from stationary to maximum rotation and back within 10 s. Data was captured at a 100 Hz sampling rate, and scalogram sample images for each fault condition were analyzed. The classification was performed using the proposed wavelet-ViT method on vibration data collected from two points inside the hull. Sensors affixed to the upper deck of the hull demonstrated an average accuracy of 0.9855, whereas those installed within a DAQ-equipped container achieved an average accuracy of 0.9905. Furthermore, classification efforts extended to eleven scenarios, incorporating four datasets from terrestrial tests, where sensors on the hull’s deck and within the container reported average accuracies of 0.9009 and 0.8761, respectively, illustrating the classification methodology’s efficacy across various operational conditions.

Author Contributions

Conceptualization, H.C. and K.-B.C.; methodology, H.C. and M.K.; software, H.C. and J.-H.P.; validation, H.C., K.-B.C. and M.K.; formal analysis, H.C.; investigation, H.C.; resources, H.C. and D.-H.J.; data curation, H.C. and J.-H.P.; writing—original draft preparation, H.C.; writing—review and editing, H.C.; visualization, H.C.; supervision, H.-S.C.; project administration, H.-S.C., D.-H.J. and H.C.; funding acquisition, H.-S.C. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the Unmanned Vehicles Core Technology Research and Development Program through the National Research Foundation of Korea (NRF) and the Unmanned Vehicle Advanced Research Center (UVARC), funded by the Ministry of Science and ICT, the Republic of Korea (NRF-2020M3C1C1A02086321).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Data are contained within the article.

Acknowledgments

The authors acknowledge the members of Korea Maritime & Ocean University Intelligent Robot & Automation Lab.

Conflicts of Interest

Author Myungjun Kim was employed by the company LIG NEX1 Co., Ltd. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Appendix A

Figure A1. On-ground collected fault case and its resulting CWT result: (a) Normal; (b) Breakage 21%; (c) Thin rope entanglement; (d) Net entanglement.
Figure A1. On-ground collected fault case and its resulting CWT result: (a) Normal; (b) Breakage 21%; (c) Thin rope entanglement; (d) Net entanglement.
Sensors 24 01697 g0a1
The comparative analysis of the CWT results from the on-ground and ocean experiments shows significant distinctions. In instances of imbalance-induced faults, there is a notable amplification of the primary component (first order, identical with rotor speed)’s impact. It is postulated that in on-ground experiments, this prominence of the primary component can be attributed to the absence of damping and interactions caused by fluids. This leads to the hypothesis that the observed effects are primarily mechanical in nature, occurring without the complicating influences of fluid dynamics. However, the CWT findings from the ocean experiment, while identifying changes in the first order vibration, also showed the presence of additional elements in other regions. This variation is hypothesized to stem from residual vibrations due to fluid interaction, especially pronounced in entanglement cases where the contact area with the fluid is enlarged, leading to the detection of multiple components beyond the first order.

References

  1. Martinez, M.J. Harbor and Coastal Structures: A Review of Mechanical Fatigue Under Random Wave Loading. Heliyon 2021, 7, e08241. [Google Scholar] [CrossRef]
  2. De Gracia, L.; Wang, H.; Mao, W.; Osawa, N.; Rychlik, I.; Storhaug, G. Comparison of Two Statistical Wave Models for Fatigue and Fracture Analysis of Ship Structures. Ocean Eng. 2019, 187, 106161. [Google Scholar] [CrossRef]
  3. Roche, M.; Bender, R.; Schütze, M. Corrosion Resistance of Steels, Nickel Alloys and Zinc in Aqueous Media; John Wiley & Sons: Hoboken, NJ, USA, 2016. [Google Scholar]
  4. Zhang, K.; Li, H.; Cheng, H.; Luo, B.; Liu, P. Combined Effects of Seawater Ageing and Fatigue Loading on the Bearing Performance and Failure Mechanism of CFRP/CFRP Single-Lap Bolted Joints. Compos. Struct. 2020, 234, 111677. [Google Scholar] [CrossRef]
  5. McIlgorm, A.; Campbell, H.F.; Rule, M.J. The Economic Cost and Control of Marine Debris Damage in the Asia-Pacific Region. Ocean Coast. Manag. 2011, 54, 643–651. [Google Scholar] [CrossRef]
  6. Choi, S.K.; Oh, H.J.; Yun, S.H.; Lee, H.J.; Lee, K.; Han, Y.S.; Kim, S.; Park, S.R. Population Dynamics of the “golden Tides” Seaweed, Sargassum Horneri, on the Southwestern Coast of Korea: The Extent and Formation of Golden Tides. Sustainability 2020, 12, 2903. [Google Scholar] [CrossRef]
  7. Qi, L.; Cheng, P.; Wang, M.; Hu, C.; Xie, Y.; Mao, K. Where Does Floating Sargassum in the East China Sea Come From? Harmful Algae 2023, 129, 102523. [Google Scholar] [CrossRef]
  8. Nuchturee, C.; Li, T.; Xia, H. Energy Efficiency of Integrated Electric Propulsion for Ships—A Review. Renew. Sustain. Energy Rev. 2020, 134, 110145. [Google Scholar] [CrossRef]
  9. Krell, E.; King, S.A.; Garcia Carrillo, L.R. Autonomous Surface Vehicle Energy-Efficient and Reward-Based Path Planning Using Particle Swarm Optimization and Visibility Graphs. Appl. Ocean Res. 2022, 122, 103125. [Google Scholar] [CrossRef]
  10. Johnston, P.; Marina, C.; Poole, M.; Marina, C. Wave-Propelled Unmanned Surface Vessel (USV). 2017, pp. 3–6. Available online: https://ieeexplore.ieee.org/abstract/document/8084782 (accessed on 27 February 2024).
  11. Astrov, I.; Udal, A. Neural Predictive Tracking Control of Catamaran Model Sailboat for Situation Awareness Applications. In Proceedings of the INES 2020—IEEE 24th International Conference on Intelligent Engineering Systems, Proceedings, Reykjavik, Iceland, 8–10 July 2020; pp. 153–158. [Google Scholar] [CrossRef]
  12. Wang, Z.; Carriveau, R.; Ting, D.S.K.; Xiong, W.; Wang, Z. A Review of Marine Renewable Energy Storage. Int. J. Energy Res. 2019, 43, 6108–6150. [Google Scholar] [CrossRef]
  13. Peša, T.; Krčum, M.; Kero, G.; Šoda, J. Retrofitting Vessel with Solar and Wind Renewable Energy Sources as an Example of the Croatia Study-Case. J. Mar. Sci. Eng. 2022, 10, 1471. [Google Scholar] [CrossRef]
  14. Yuan, Y.; Wang, J.; Yan, X.; Shen, B.; Long, T. A Review of Multi-Energy Hybrid Power System for Ships. Renew. Sustain. Energy Rev. 2020, 132, 110081. [Google Scholar] [CrossRef]
  15. Thombre, S.; Zhao, Z.; Ramm-Schmidt, H.; Vallet Garcia, J.M.; Malkamaki, T.; Nikolskiy, S.; Hammarberg, T.; Nuortie, H.; Bhuiyan, M.Z.H.; Särkkä, S.; et al. Sensors and AI Techniques for Situational Awareness in Autonomous Ships: A Review. IEEE Trans. Intell. Transp. Syst. 2022, 23, 64–83. [Google Scholar] [CrossRef]
  16. Zhuang, X.; Xu, Y.; Gao, Y.; Sun, G.; Lin, T.; Chan, C.C.K. Remote Data Transmission Technology Based on BeiDou Satellite Navigation Sensor System Onboard Ship. Sens. Mater. 2021, 33, 715–726. [Google Scholar] [CrossRef]
  17. Newaliya, N.; Singh, Y. A Review of Maritime Spatio-Temporal Data Analytics. In Proceedings of the 2021 International Conference on Computational Performance Evaluation, ComPE 2021, Virtual, 1–3 December 2021; Institute of Electrical and Electronics Engineers Inc.: New York City, NY, USA, 2021; pp. 219–226. [Google Scholar]
  18. Karatuğ, Ç.; Arslanoğlu, Y. Development of Condition-Based Maintenance Strategy for Fault Diagnosis for Ship Engine Systems. Ocean Eng. 2022, 256, 111515. [Google Scholar] [CrossRef]
  19. Xu, X.; Yan, X.; Yang, K.; Zhao, J.; Sheng, C.; Yuan, C. Review of Condition Monitoring and Fault Diagnosis for Marine Power Systems. Transp. Saf. Environ. 2021, 3, 85–102. [Google Scholar] [CrossRef]
  20. William O’connell, J. Shipboard Fault Detection, Marine Micro-Grid Power Diagnostics and Vessel Ventilation Monitoring. Doctoral Dissertation, Massachusetts Institute of Technology, Cambridge, MA, USA, 2015. [Google Scholar]
  21. Velasco-Gallego, C.; Navas De Maya, B.; Matutano Molina, C.; Lazakis, I.; Cubo Mateo, N. Recent Advancements in Data-Driven Methodologies for the Fault Diagnosis and Prognosis of Marine Systems: A Systematic Review. Ocean Eng. 2023, 284, 115277. [Google Scholar] [CrossRef]
  22. Pająk, M.; Kluczyk, M.; Muślewski, Ł.; Lisjak, D.; Kolar, D. Ship Diesel Engine Fault Diagnosis Using Data Science and Machine Learning. Electronics 2023, 12, 3860. [Google Scholar] [CrossRef]
  23. Capocci, R.; Omerdic, E.; Dooly, G.; Toal, D. Fault-Tolerant Control for ROVs Using Control Reallocation and Power Isolation. J. Mar. Sci. Eng. 2018, 6, 40. [Google Scholar] [CrossRef]
  24. Serra-Sogas, N.; O’Hara, P.D.; Pearce, K.; Smallshaw, L.; Canessa, R. Using Aerial Surveys to Fill Gaps in AIS Vessel Traffic Data to Inform Threat Assessments, Vessel Management and Planning. Mar. Policy 2021, 133, 104765. [Google Scholar] [CrossRef]
  25. Henriquez, P.; Alonso, J.B.; Ferrer, M.A.; Travieso, C.M. Review of Automatic Fault Diagnosis Systems Using Audio and Vibration Signals. IEEE Trans. Syst. Man. Cybern. Syst. 2014, 44, 642–652. [Google Scholar] [CrossRef]
  26. Tama, B.A.; Vania, M.; Lee, S.; Lim, S. Recent Advances in the Application of Deep Learning for Fault Diagnosis of Rotating Machinery Using Vibration Signals. Artif. Intell. Rev. 2023, 56, 4667–4709. [Google Scholar] [CrossRef]
  27. Genta, G. Dynamics of Rotating Systems; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2013; Volume 53, ISBN 9788578110796. [Google Scholar]
  28. Yucel, A.; Arpaci, A. Free and Forced Vibration Analyses of Ship Structures Using the Finite Element Method. J. Mar. Sci. Technol. 2013, 18, 324–338. [Google Scholar] [CrossRef]
  29. Varanis, M.; Silva, A.L.; Balthazar, J.M.; Pederiva, R. A Tutorial Review on Time-Frequency Analysis of Non-Stationary Vibration Signals with Nonlinear Dynamics Applications. Braz. J. Phys. 2021, 51, 859–877. [Google Scholar] [CrossRef]
  30. Addison, P.S. The Illustrated Wavelet Transform Handbook: Introductory Theory and Applications in Science, Engineering, Medicine and Finance; CRC Press: Boca Raton, FL, USA, 2017. [Google Scholar]
  31. Rioul, O.; Vetterli, M. Wavelets and signal processing. IEEE Signal Process. Mag. 1991, 8, 14–38. [Google Scholar] [CrossRef]
  32. Kemp, M.P. Underwater Thruster Fault Detection and Isolation. In Proceedings of the AIAA Scitech 2019 Forum, San Diego, CA, USA, 7–11 January 2019; pp. 1–8. [Google Scholar] [CrossRef]
  33. Tsai, C.M.; Wang, C.S.; Chung, Y.J.; Sun, Y.D.; Perng, J.W. Multi-Sensor Fault Diagnosis of Underwater Thruster Propeller Based on Deep Learning. Sensors 2021, 21, 7187. [Google Scholar] [CrossRef]
  34. Yuan, J.; Wan, J.; Zhang, W.; Liu, H.; Zhang, H. An Underwater Thruster Fault Diagnosis Simulator and Thrust Calculation Method Based on Fault Clustering. J. Robot. 2021, 2021, 1–10. [Google Scholar] [CrossRef]
  35. He, C.; Shi, H.; Si, J.; Li, J. Physics-Informed Interpretable Wavelet Weight Initialization and Balanced Dynamic Adaptive Threshold for Intelligent Fault Diagnosis of Rolling Bearings. J. Manuf. Syst. 2023, 70, 579–592. [Google Scholar] [CrossRef]
  36. Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; et al. An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale. arXiv 2020, arXiv:2010.11929. [Google Scholar]
  37. Yusoff, M.A.M.; Arshad, M.R. Active Fault Tolerant Control of a Remotely Operated Vehicle Propulsion System. Procedia Eng. 2012, 41, 622–628. [Google Scholar] [CrossRef]
  38. Liu, F.; Tang, H.; Qin, Y.; Duan, C.; Luo, J.; Pu, H. Review on Fault Diagnosis of Unmanned Underwater Vehicles. Ocean Eng. 2022, 243, 110290. [Google Scholar] [CrossRef]
  39. Antonelli, G.; Meridionale, L.; View, D.; Antonelli, G. A Survey of Fault Detection/Tolerance Strategies for AUVs and ROVs. In Fault Diagnosis and Fault Tolerance for Mechatronic Systems: Recent Advances; Springer: Berlin/Heidelberg, Germany, 2015; pp. 109–127. [Google Scholar]
  40. Falkenberg, T.; Gregersen, R.T.; Blanke, M. Navigation System Fault Diagnosis for Underwater Vehicle. IFAC Proc. Vol. (IFAC-PapersOnline) 2014, 19, 9654–9660. [Google Scholar] [CrossRef]
  41. Chen, X.; Bose, N.; Brito, M.; Khan, F.; Thanyamanta, B.; Zou, T. A Review of Risk Analysis Research for the Operations of Autonomous Underwater Vehicles. Reliab. Eng. Syst. Saf. 2021, 216, 108011. [Google Scholar] [CrossRef]
  42. Zakaria, W.N.W.; Mahmood, I.A.T.; Shamsudin, A.U.; Rahman, M.A.A.; Tomari, M.R.M. ROS-Based SLAM and Path Planning for Autonomous Unmanned Surface Vehicle Navigation System. In Proceedings of the 2022 IEEE 5th International Symposium in Robotics and Manufacturing Automation, ROMA 2022, Malacca, Malaysia, 6–8 August 2022; pp. 6–8. [Google Scholar] [CrossRef]
  43. Heiberg, A.; Larsen, T.N.; Meyer, E.; Rasheed, A.; San, O.; Varagnolo, D. Risk-Based Implementation of COLREGs for Autonomous Surface Vehicles Using Deep Reinforcement Learning. Neural Netw. 2022, 152, 17–33. [Google Scholar] [CrossRef]
  44. Nascimento, S.; Valdenegro-Toro, M. Modeling and Soft-Fault Diagnosis of Underwater Thrusters with Recurrent Neural Networks; Elsevier: Amsterdam, The Netherlands, 2018; Volume 51, pp. 80–85. [Google Scholar]
  45. Zhang, Y.; Geng, J.; Wang, N. Correlation analysis and fault detection of current and speed signals for underwater thruster entanglement. Adv. Control Appl. Eng. Ind. Syst. 2023, e145. [Google Scholar] [CrossRef]
  46. Cho, H.; Choi, H.S.; Kim, H.J.; Nam, K.S.; Ryu, J.D.; Ha, K.N. Feature Selection for Unmanned Surface Vehicle Fault Diagnosis Research and Experimental Verification. J. Inst. Control Robot. Syst. 2022, 28, 542–550. [Google Scholar] [CrossRef]
  47. Choo, K.-B.; Cho, H.; Park, J.-H.; Huang, J.; Jung, D.; Lee, J.; Jeong, S.-K.; Yoon, J.; Choo, J.; Choi, H.-S. A Research on Fault Diagnosis of a USV Thruster Based on PCA and Entropy. Appl. Sci. 2023, 13, 3344. [Google Scholar] [CrossRef]
  48. Yan, R.; Shang, Z.; Xu, H.; Wen, J.; Zhao, Z.; Chen, X.; Gao, R.X. Wavelet Transform for Rotary Machine Fault Diagnosis:10 Years Revisited. Mech. Syst. Signal Process 2023, 200, 110545. [Google Scholar] [CrossRef]
  49. Tuli, S.; Dasgupta, I.; Grant, E.; Griffiths, T.L. Are Convolutional Neural Networks or Transformers More like Human Vision? arXiv 2021, arXiv:2105.07197. [Google Scholar]
Figure 1. Types of unmanned marine platforms: (a) USV; (b) AUV; and (c) ROV.
Figure 1. Types of unmanned marine platforms: (a) USV; (b) AUV; and (c) ROV.
Sensors 24 01697 g001
Figure 2. Overview of FDD USV.
Figure 2. Overview of FDD USV.
Sensors 24 01697 g002
Figure 3. Imbalance on the underwater thruster blade.
Figure 3. Imbalance on the underwater thruster blade.
Sensors 24 01697 g003
Figure 4. Sample vibration data and its Fast Fourier Transform result (FFT): (a) Normal (healthy) thruster raw vibration and 1X component; (b) Faulty thruster (breakage; 21% of diameter) raw vibration and 1X component.
Figure 4. Sample vibration data and its Fast Fourier Transform result (FFT): (a) Normal (healthy) thruster raw vibration and 1X component; (b) Faulty thruster (breakage; 21% of diameter) raw vibration and 1X component.
Sensors 24 01697 g004
Figure 5. Sample scalogram results: (a) Normal (healthy) thruster scalogram; (b) Faulty thruster (breakage; 21% of diameter) scalogram.
Figure 5. Sample scalogram results: (a) Normal (healthy) thruster scalogram; (b) Faulty thruster (breakage; 21% of diameter) scalogram.
Sensors 24 01697 g005
Figure 6. The ViT Architecture.
Figure 6. The ViT Architecture.
Sensors 24 01697 g006
Figure 7. Sample data profile of FDD USV: (a) Analogue input, Rotational speed; (b) Acquired vibration data and FDD USV’s natural frequency (red dotted square).
Figure 7. Sample data profile of FDD USV: (a) Analogue input, Rotational speed; (b) Acquired vibration data and FDD USV’s natural frequency (red dotted square).
Sensors 24 01697 g007
Figure 8. Hardware configuration of FDD USV control systems.
Figure 8. Hardware configuration of FDD USV control systems.
Sensors 24 01697 g008
Figure 9. Configuration of FDD USV control and DAQ system.
Figure 9. Configuration of FDD USV control and DAQ system.
Sensors 24 01697 g009
Figure 10. Test cases: (a) Normal(healthy); (b) Breakage 7%; (c) Thin rope entanglement; (d) Breakage 14%; (e) Rope entanglement; (f) Breakage 21%; and (g) Net entanglement.
Figure 10. Test cases: (a) Normal(healthy); (b) Breakage 7%; (c) Thin rope entanglement; (d) Breakage 14%; (e) Rope entanglement; (f) Breakage 21%; and (g) Net entanglement.
Sensors 24 01697 g010
Figure 11. Sample result of CWT for test cases: (a) Normal (healthy); (b) Breakage 7%; (c) Thin rope entanglement; (d) Breakage 14%; (e) Rope entanglement; (f) Breakage 21%; (g) Net entanglement.
Figure 11. Sample result of CWT for test cases: (a) Normal (healthy); (b) Breakage 7%; (c) Thin rope entanglement; (d) Breakage 14%; (e) Rope entanglement; (f) Breakage 21%; (g) Net entanglement.
Sensors 24 01697 g011
Figure 12. Naturally-occurred thruster entanglement faults during the experiment.
Figure 12. Naturally-occurred thruster entanglement faults during the experiment.
Sensors 24 01697 g012
Figure 13. Overall fault diagnosis framework.
Figure 13. Overall fault diagnosis framework.
Sensors 24 01697 g013
Figure 14. Vibration sensor attachment position of both vib No.4 and vib No.9.
Figure 14. Vibration sensor attachment position of both vib No.4 and vib No.9.
Sensors 24 01697 g014
Figure 15. Vib No.4 maritime experiment dataset sample classification result.
Figure 15. Vib No.4 maritime experiment dataset sample classification result.
Sensors 24 01697 g015
Figure 16. Vib No.9 maritime experiment dataset sample classification result.
Figure 16. Vib No.9 maritime experiment dataset sample classification result.
Sensors 24 01697 g016
Figure 17. Vib No.4 maritime and on-ground experiment dataset classification result.
Figure 17. Vib No.4 maritime and on-ground experiment dataset classification result.
Sensors 24 01697 g017
Figure 18. Vib No.9 maritime and on-ground experiment dataset classification result.
Figure 18. Vib No.9 maritime and on-ground experiment dataset classification result.
Sensors 24 01697 g018
Table 1. Number of acquired datasets for all cases.
Table 1. Number of acquired datasets for all cases.
TypeNormalBreakage 7%Breakage 14%Breakage 21%
Number of data185200200220
TypeThin rope
entanglement
Rope
Entanglement
Net
entanglement
Number of data210218203
Type *Normal *Breakage 21% *Thin rope
entanglement *
Net
entanglement *
Number of data191194269150
* On-ground experiment datasets. Figures are shown in Appendix A, Figure A1.
Table 2. The adjustable parameters of the ViT classifier.
Table 2. The adjustable parameters of the ViT classifier.
ParametersValue
Input image size[384, 384, 3]
Number of Transformer blocks12
Number of attention heads12
Position encoding dimension2D
OptimizerAdam (learning rate: 1 × 10 3 )
Layer143
Patch size16
Batch size12
Max epoch12
Dropout probability0.1
Attention dropout probability0.1
Table 3. Accuracies of the maritime experiment classification result.
Table 3. Accuracies of the maritime experiment classification result.
Sensor NumberAccuracy
Vib No.40.9790
Vib No.40.9840
Vib No.40.9790
Vib No.41
Vib No.91
Vib No.91
Vib No.90.9790
Vib No.90.9840
Table 4. Accuracies of maritime and on-ground experiment classification result.
Table 4. Accuracies of maritime and on-ground experiment classification result.
Sensor NumberAccuracy
Vib No.40.9009
Vib No.40.9054
Vib No.40.9009
Vib No.40.8964
Vib No.90.8919
Vib No.90.8649
Vib No.90.8873
Vib No.90.8603
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Cho, H.; Park, J.-H.; Choo, K.-B.; Kim, M.; Ji, D.-H.; Choi, H.-S. Unmanned Surface Vehicle Thruster Fault Diagnosis via Vibration Signal Wavelet Transform and Vision Transformer under Varying Rotational Speed Conditions. Sensors 2024, 24, 1697. https://doi.org/10.3390/s24051697

AMA Style

Cho H, Park J-H, Choo K-B, Kim M, Ji D-H, Choi H-S. Unmanned Surface Vehicle Thruster Fault Diagnosis via Vibration Signal Wavelet Transform and Vision Transformer under Varying Rotational Speed Conditions. Sensors. 2024; 24(5):1697. https://doi.org/10.3390/s24051697

Chicago/Turabian Style

Cho, Hyunjoon, Jung-Hyeun Park, Ki-Beom Choo, Myungjun Kim, Dae-Hyeong Ji, and Hyeung-Sik Choi. 2024. "Unmanned Surface Vehicle Thruster Fault Diagnosis via Vibration Signal Wavelet Transform and Vision Transformer under Varying Rotational Speed Conditions" Sensors 24, no. 5: 1697. https://doi.org/10.3390/s24051697

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop