Search Results (17)

In industrial contexts, anomaly detection is crucial for ensuring quality control and maintaining operational efficiency in manufacturing processes. Leveraging high-level features extracted from ImageNet-trained networks and the robust capabilities of the Deep Support Vector Data Description (SVDD) model for anomaly detection, this paper proposes an improved Deep SVDD model, termed Feature-Patching SVDD (FPSVDD), designed for unsupervised anomaly detection in industrial applications. This model integrates a feature-patching technique with the Deep SVDD framework. Features are extracted from a pre-trained backbone network on ImageNet, and each extracted feature is split into multiple small patches of appropriate size. This approach effectively captures both macro-structural information and fine-grained local information from the extracted features, enhancing the model’s sensitivity to anomalies. The feature patches are then aggregated and concatenated for further training with the Deep SVDD model. Experimental results on both the MvTec AD and CIFAR-10 datasets demonstrate that our model outperforms current mainstream approaches and provides significant improvements in anomaly detection performance, which is vital for industrial quality assurance and defect detection in real-time manufacturing scenarios. Full article

In Nuclear Power Plants (NPPs), online condition monitoring and the fault detection of structures, systems and components (SSCs) can aid in guaranteeing safe operation. The use of data-driven methods for these tasks is limited by the requirement of physically consistent outcomes, particularly in safety-critical systems. Considering the importance of regulating valves (e.g., safety relief valves and main steam isolation valves), this work proposes a multistage Physics-Informed Neural Network (PINN) for fault detection in such components. Two stages of the PINN are built by developing the process model of the regulating valve, which integrates the basic valve sizing equation into the loss function to jointly train the two stages of the PINN. In the 1st stage, a shallow Neural Network (NN) with only one hidden layer is developed to estimate the equivalent flow coefficient (a key performance indicator of regulating valves) using the displacement of the valve as input. In the 2nd stage, a Deep Neural Network (DNN) is developed to estimate the flow rate expected in normal conditions using inputs such as the estimated flow coefficient from the 1st stage, the differential pressure, and the fluid temperature. Then, the residual, i.e., the difference between the estimated and measured flow rates, is fed into a Deep Support Vector Data Description (DeepSVDD) to detect the occurrence of faults. Moreover, the deviation between the estimated flow coefficients of normal and faulty conditions is used to interpret the consistency of the detection result with physics. The proposed method is, first, applied to a simulation case implemented to emulate the operating characteristics of regulating the valves of NPPs and then validated on a real-world case study based on the DAMADICS benchmark. Compared to state-of-the-art fault detection methods, the obtained results from the proposed method show effective fault detection performance and reasonable flow coefficient estimation, thus guaranteeing the physical consistency of the detection results. Full article

Because of the difficulty in fault detection for and diagnosing the adjustment hydraulic servomotor, this paper uses feature extraction technology to extract the time domain and frequency domain features of the pressure signal of the adjustment hydraulic servomotor and splice the features of multiple pressure signals through the Multi-source Information Fusion (MSIF) method. The comprehensive expression of device status information is obtained. After that, this paper proposes a fault detection Algorithm GA-SVDD-neg, which uses Genetic Algorithm (GA) to optimize Support Vector Data Description with negative examples (SVDD-neg). Through joint optimization with the Mutual Information (MI) feature selection algorithm, the features that are most sensitive to the state deterioration of the adjustment hydraulic servomotor are selected. Experiments show that the MI algorithm has a better performance than other feature dimensionality reduction algorithms in the field of the abnormal detection of adjustment hydraulic servomotors, and the GA-SVDD-neg algorithm has a stronger robustness and generality than other anomaly detection algorithms. In addition, to make full use of the advantages of deep learning in automatic feature extraction and classification, this paper realizes the fault diagnosis of the adjustment hydraulic servomotor based on 1D Convolutional Neural Network (1DCNN). The experimental results show that this algorithm has the same superior performance as the traditional algorithm in feature extraction and can accurately diagnose the known faults of the adjustment hydraulic servomotor. This research is of great significance for the intelligent transformation of adjustment hydraulic servomotors and can also provide a reference for the fault warning and diagnosis of the Electro-Hydraulic (EH) system of the same type of steam turbine. Full article

In the smart grid, malicious consumers can hack their smart meters to report false power consumption readings to steal electricity. Developing a machine-learning based detector for identifying these readings is a challenge due to the unavailability of malicious datasets. Most of the existing works in the literature assume attacks to compute malicious data. These detectors are trained to identify these attacks, but they cannot identify new attacks, which creates a vulnerability. Very few papers in the literature tried to address this problem by investigating anomaly detectors trained solely on benign data, but they suffer from these limitations: (1) low detection accuracy and high false alarm; (2) the need for knowledge on the malicious data to compute good detection thresholds; and (3) they cannot capture the temporal correlations of the readings and do not address the class overlapping issue caused by some deceptive attacks. To address these limitations, this paper presents a deep support vector data description (DSVDD) based unsupervised detector for false data in smart grid. Time-series readings are transformed into images, and the detector is exclusively trained on benign images. Our experimental results demonstrate the superior performance of our detectors compared to existing approaches in the literature. Specifically, our proposed DSVDD-based schemes have exhibited improvements of 0.5% to 3% in terms of recall and 3% to 9% in terms of the Area Under the Curve (AUC) when compared to existing state-of-the-art detectors. Full article

An open-set recognition scheme for tree leaves based on deep learning feature extraction is presented in this study. Deep learning algorithms are used to extract leaf features for different wood species, and the leaf set of a wood species is divided into two datasets: the leaf set of a known wood species and the leaf set of an unknown species. The deep learning network (CNN) is trained on the leaves of selected known wood species, and the features of the remaining known wood species and all unknown wood species are extracted using the trained CNN. Then, the single-class classification is performed using the weighted SVDD algorithm to recognize the leaves of known and unknown wood species. The features of leaves recognized as known wood species are fed back to the trained CNN to recognize the leaves of known wood species. The recognition results of a single-class classifier for known and unknown wood species are combined with the recognition results of a multi-class CNN to finally complete the open recognition of wood species. We tested the proposed method on the publicly available Swedish Leaf Dataset, which includes 15 wood species (5 species used as known and 10 species used as unknown). The test results showed that, with F1 scores of 0.7797 and 0.8644, mixed recognition rates of 95.15% and 93.14%, and Kappa coefficients of 0.7674 and 0.8644 under two different data distributions, the proposed method outperformed the state-of-the-art open-set recognition algorithms in all three aspects. And, the more wood species that are known, the better the recognition. This approach can extract effective features from tree leaf images for open-set recognition and achieve wood species recognition without compromising tree material. Full article

While the statistical properties of images are vital in forestry engineering, the usefulness of these properties in various forestry tasks may vary, and certain image properties might not be enough to adequately describe a particular tree species. To address this problem, we propose a novel method to comprehensively analyze the relationship between various image statistical properties and images of different tree species, and to determine the subset of features that best describe each individual tree species. In this study, we employed various image statistical properties to quantify images of five distinct tree species from diverse places. Multiple feature-filtering methods were used to find the feature subset with the greatest correlation with the tree species category variable. Support Vector Machines (SVM) were employed to determine the number of features with the greatest correlation with the tree species, and a grid search was used to optimize the model. For each type of tree species image, we obtained the important ranking of all features in this type of tree species, and the sensitive feature subset of various tree species according to the order of features was determined by adding them to the Deep Support Vector Data Description (Deep SVDD). Finally, the feasibility of using a sensitive subset of the tree species was confirmed. The experimental results revealed that by utilizing the filtering method in conjunction with SVM, a total of eight feature subsets with the highest correlation with tree species categories were identified. Additionally, the sensitive feature subsets of different tree species exhibited significant differences. Remarkably, employing the sensitive feature subset of each tree species resulted in F1-score higher than 0.7 for all tree species. These experimental results demonstrate that the sensitive feature subset of tree species based on image statistical properties can serve as a potential representation of a specific tree species, while features that are less strongly associated with tree species may be significant in related areas, such as forestry protection and other related fields. Full article

The tendon that is inserted into the duct is a crucial component of prestressed concrete (PSC) bridges and, when exposed to air, can quickly corrode, and cause structural collapse. It can interpret the signal measured by non-destructive testing (NDT) to determine the condition (normal or void) inside the duct. However, it requires the use of expensive NDT equipment such as ultrasonic waves or the hiring of experts. In this paper, we proposed an impact–echo (IE) method based on deep support vector data description (Deep SVDD) for economical void detection inside a duct. Because the pattern of IE changes for various reasons such as difference of specimen or bridge, supervised learning is not suitable. Deep SVDD is classified as normal and defective, which is a broad distribution as a hypersphere that encloses a multi-dimensional feature space for normal data represented by an autoencoder. Here, an autoencoder was developed based on the ELMo (embeddings from language model)-like structure to obtain an effective representation for IE. In the experiment, we evaluated the performance of the IE data measured in different specimens. Thus, our proposed model showed an accuracy of about 77.84% which is an improvement of up to about 47% compared to the supervised learning approach. Full article

Machine learning-based intrusion detection systems are an effective way to cope with the increasing security threats faced by industrial control systems. Considering that it is hard and expensive to obtain attack data, it is more reasonable to develop a model trained with only normal data. However, both high-dimensional data and the presence of outliers in the training set result in efficiency degradation. In this research, we present a hybrid intrusion detection method to overcome these two problems. First, we created an improved autoencoder that incorporates the deep support vector data description (Deep SVDD) loss into the training of the autoencoder. Under the combination of Deep SVDD loss and reconstruction loss, the novel autoencoder learns a more compact latent representation from high-dimensional data. The density-based spatial clustering of applications with noise algorithm is then used to remove potential outliers in the training data. Finally, a Bayesian Gaussian mixture model is used to identify anomalies. It learns the distribution of the filtered training data and uses the probabilities to classify normal and anomalous samples. We conducted a series of experiments on two intrusion detection datasets to assess performance. The proposed model performs better than other baseline methods when dealing with high-dimensional and contaminated data. Full article

Untargeted metabolomics (UM) is increasingly being deployed as a strategy for screening patients that are suspected of having an inborn error of metabolism (IEM). In this study, we examined the potential of existing outlier detection methods to detect IEM patient profiles. We benchmarked 30 different outlier detection methods when applied to three untargeted metabolomics datasets. Our results show great differences in IEM detection performances across the various methods. The methods DeepSVDD and R-graph performed most consistently across the three metabolomics datasets. For datasets with a more balanced number of samples-to-features ratio, we found that AE reconstruction error, Mahalanobis and PCA reconstruction error also performed well. Furthermore, we demonstrated the importance of a PCA transform prior to applying an outlier detection method since we observed that this increases the performance of several outlier detection methods. For only one of the three metabolomics datasets, we observed clinically satisfying performances for some outlier detection methods, where we were able to detect 90% of the IEM patient samples while detecting no false positives. These results suggest that outlier detection methods have the potential to aid the clinical investigator in routine screening for IEM using untargeted metabolomics data, but also show that further improvements are needed to ensure clinically satisfying performances. Full article

Deep neural network-based autoencoders can effectively extract high-level abstract features with outstanding generalization performance but suffer from sparsity of extracted features, insufficient robustness, greedy training of each layer, and a lack of global optimization. In this study, the broad learning system (BLS) is improved to obtain a new model for data reconstruction. Support Vector Domain Description (SVDD) is one of the best-known one-class-classification methods used to solve problems where the proportion of sample categories of data is extremely unbalanced. The SVDD is sensitive to penalty parameters C, which represents the trade-off between sphere volume and the number of target data outside the sphere. The training process only considers normal samples, which leads to a low recall rate and weak generalization performance. To address these issues, we propose a BLS-based weighted SVDD algorithm (BLSW_SVDD), which introduces reconstruction error weights and a small number of anomalous samples when training the SVDD model, thus improving the robustness of the model. To evaluate the performance of BLSW_SVDD model, comparison experiments were conducted on the UCI dataset, and the experimental results showed that in terms of accuracy and F1 values, the algorithm has better performance advantages than the traditional and improved SVDD algorithms. Full article

With the rapid development of network technologies, the network security of industrial control systems has aroused widespread concern. As a defense mechanism, an ideal intrusion detection system (IDS) can effectively detect abnormal behaviors in a system without affecting the performance of the industrial control system (ICS). Many deep learning methods are used to build an IDS, which rely on massive numbers of variously labeled samples for model training. However, network traffic is imbalanced, and it is difficult for researchers to obtain sufficient attack samples. In addition, the attack variants are rich, and constructing all possible attack types in advance is impossible. In order to overcome these challenges and improve the performance of an IDS, this paper presents a novel intrusion detection approach which integrates a one-dimensional convolutional autoencoder (1DCAE) and support vector data description (SVDD) for the first time. For the two-stage training process, 1DCAE fails to retain the key features of intrusion detection and SVDD has to add restrictions, so a joint optimization solution is introduced. A three-stage optimization process is proposed to obtain better performance. Experiments on the benchmark intrusion detection dataset NSL-KDD show that the proposed method can effectively detect various unknown attacks, learning with only normal traffic. Compared with the recent state-of-art intrusion detection baselines, the proposed method is improved in most metrics. Full article

Aiming at the online detection problem of rolling bearings, the limited amount of target bearing data leads to insufficient model in training and feature representation. It is difficult for the online detection model to construct an accurate decision boundary. To solve the problem, a multi-scale robust anomaly detection method based on data enhancement technology is proposed in this paper. Firstly, the training data are transformed into multiple subspaces through the data enhancement technology. Then, a prototype clustering method is introduced to enhance the robustness of features representation under the framework of the robust deep auto-encoding algorithm. Finally, the robust multi-scale Deep-SVDD hyper sphere model is constructed to achieve online detection of abnormal state data. Experiments are conducted on the IEEE PHM Challenge 2012 bearing data set and XJTU-TU data set. The proposed method shows much greater susceptibility to incipient faults, and it has fewer false alarms. The robust multi-scale Deep-SVDD hyper sphere model significantly improves the performance of incipient fault detection for rolling bearings. Full article

Advanced heart monitors, especially those enabled by the Internet of Health Things (IoHT), provide a great opportunity for continuous collection of the electrocardiogram (ECG), which contains rich information about underlying cardiac conditions. Realizing the full potential of IoHT-enabled cardiac monitoring hinges, to a great extent, on the detection of disease-induced anomalies from collected ECGs. However, challenges exist in the current literature for IoHT-based cardiac monitoring: (1) Most existing methods are based on supervised learning, which requires both normal and abnormal samples for training. This is impractical as it is generally unknown when and what kind of anomalies will occur during cardiac monitoring. (2) Furthermore, it is difficult to leverage advanced machine learning approaches for information processing of 1D ECG signals, as most of them are designed for 2D images and higher-dimensional data. To address these challenges, a new sensor-based unsupervised framework is developed for IoHT-based cardiac monitoring. First, a high-dimensional tensor is generated from the multi-channel ECG signals through the Gramian Angular Difference Field (GADF). Then, multi-linear principal component analysis (MPCA) is employed to unfold the ECG tensor and delineate the disease-altered patterns. Obtained principal components are used as features for anomaly detection using machine learning models (e.g., deep support vector data description (deep SVDD)) as well as statistical control charts (e.g., Hotelling $T^2$ chart). The developed framework is evaluated and validated using real-world ECG datasets. Comparing to the state-of-the-art approaches, the developed framework with deep SVDD achieves superior performances in detecting abnormal ECG patterns induced by various types of cardiac disease, e.g., an F-score of 0.9771 is achieved for detecting atrial fibrillation, 0.9986 for detecting right bundle branch block, and 0.9550 for detecting ST-depression. Additionally, the developed framework with the T² control chart facilitates personalized cycle-to-cycle monitoring with timely detected abnormal ECG patterns. The developed framework has a great potential to be implemented in IoHT-enabled cardiac monitoring and smart management of cardiac health. Full article

Support vector data description (SVDD) has been widely applied to batch process fault detection. However, it often performs poorly, especially when incipient faults occur, because it only considers the shallow data feature and omits the probabilistic information of features. In order to provide better monitoring performance on incipient faults in batch processes, an improved SVDD method, called deep probabilistic SVDD (DPSVDD), is proposed in this work by integrating the convolutional autoencoder and the probability-related monitoring indices. For mining the hidden data features effectively, a deep convolutional features extraction network is designed by a convolutional autoencoder, where the encoder outputs and the reconstruction errors are used as the monitor features. Furthermore, the probability distribution changes of these features are evaluated by the Kullback-Leibler (KL) divergence so that the probability-related monitoring indices are developed for indicating the process status. The applications to the benchmark penicillin fermentation process demonstrate that the proposed method has a better monitoring performance on the incipient faults in comparison to the traditional SVDD methods. Full article

The Varroa destructor mite is one of the most dangerous Honey Bee (Apis mellifera) parasites worldwide and the bee colonies have to be regularly monitored in order to control its spread. In this paper we present an object detector based method for health state monitoring of bee colonies. This method has the potential for online measurement and processing. In our experiment, we compare the YOLO and SSD object detectors along with the Deep SVDD anomaly detector. Based on the custom dataset with 600 ground-truth images of healthy and infected bees in various scenes, the detectors reached the highest F1 score up to 0.874 in the infected bee detection and up to 0.714 in the detection of the Varroa destructor mite itself. The results demonstrate the potential of this approach, which will be later used in the real-time computer vision based honey bee inspection system. To the best of our knowledge, this study is the first one using object detectors for the Varroa destructor mite detection on a honey bee. We expect that performance of those object detectors will enable us to inspect the health status of the honey bee colonies in real time. Full article