Out-of-Distribution (OOD) Detection Based on Deep Learning: A Review

Abstract

Out-of-Distribution (OOD) detection separates ID (In-Distribution) data and OOD data from input data through a model. This problem has attracted increasing attention in the area of machine learning. OOD detection has achieved good intrusion detection, fraud detection, system health monitoring, sensor network event detection, and ecosystem interference detection. The method based on deep learning is the most studied in OOD detection. In this paper, related basic information on OOD detection based on deep learning is described, and we categorize methods according to the training data. OOD detection is divided into supervised, semisupervised, and unsupervised. Where supervised data are used, the methods are categorized according to technical means: model-based, distance-based, and density-based. Each classification is introduced with background, examples, and applications. In addition, we present the latest applications of OOD detection based on deep learning and the problems and expectations in this field.

1. Introduction

Although machine learning based on a neural network has made great progress, even surpassing human beings under experimental conditions, the many failed examples reveal the vulnerability of the model when dealing with different distributed data [1]. Therefore, Out-of-Distribution (OOD) detection based on deep learning has received more and more attention in machine learning [2]. In order to separate OOD samples more effectively, various improvement methods have been proposed by researchers.

In general, a new research direction has to go from simple to complex, from point to surface. OOD detection is still in the initial stage. Most studies use specific data. Therefore, according to the number of labeled data, they are divided into supervised, semisupervised, and unsupervised. In the supervised methods, all the training data are labeled. In unsupervised methods, all the training data are not labeled. In the semisupervised method, part of the training data is labeled and the other part is not labeled. Among them, supervised methods are the most common. In these supervised methods, model-based OOD detection methods usually rely on the Softmax scoring function from the penultimate layer or output layer of the neural network [3]. For any given test time input, all existing solutions require a complete feedforward channel and use a fixed amount of calculation [4]. Distance-based methods usually define proximity metrics between objects. OOD data are far away from most other objects. When the data can be presented in two-dimensional or three-dimensional scatter diagrams, distance-based Out-of-Distribution points can be visually detected [5]. In addition, the density estimation of objects can be calculated relatively directly, especially when there is a proximity measure between objects. Objects in low-density areas are relatively far from their neighbors and may be considered out-of-distribution points. This type of method is called density-based.

OOD detection is similar to a binary classification problem in output form. ID samples are easy to obtain, but OOD samples are difficult to obtain, so the number of pieces is imbalanced. In special cases, there may be no OOD samples at all in the training stage, and only ID samples participate in model training. To solve this problem, we need to use semisupervised methods, of which the most common is the autoencoder [6,7,8]. In recent years, with the continuous development of OOD detection, researchers are no longer satisfied with detecting specific samples. They hope that the model has good generalization capability. Zhou et al. [9] proposed to solve the problem of data generalization by Out-of-Distribution Knowledge Distillation (OKD) and achieved good results. Rather than focusing on the model structure, most recent works [10,11,12,13,14] have targeted improvements in the objective function over ERM (empirical risk minimization).

In this paper, the background of OOD detection is briefly introduced in Section 2. Section 3 gives an overview of OOD detection based on deep learning. Section 4, Section 5 and Section 6 introduce the different classifications of OOD detection based on deep learning. Section 7 presents the areas where OOD detection based on deep learning has been applied. Section 8 introduces the challenges of OOD detection.

2. Background

In this section, we present background information related to OOD detection in this survey and the dataset and evaluation metrics used.

2.1. Development Background of OOD Detection

OOD detection is developed based on anomaly detection. This paper lists some representative research results by timeline. In the early 1980s, Hawkins et al. [15] summarized the latest achievements of anomaly detection at that time and proposed the possible future development direction in combination with the defects of anomaly detection in several areas. Svante et al. [16] proposed PCA (principal component analysis). PCA is a dimensionality reduction method that is often used to reduce the dimensionality of high-dimensional data sets. It converts a large set of variables into a smaller set of variables while retaining most of the information in the set, so the calculation method is simplified. Corts et al. [17] proposed the classic support vector machine, which solved the binary classification problem in supervised learning. Liu et al. [18] presented the Isolation Forest, solving the problem of slow speed and poor accuracy when processing big data. In 2012, Krizhevsky et al. [19] presented the AlexNet, solving the vanishing gradient problem of Sigmaid when the network is deep. Kim et al. [20] combined CNN and NLP for feature extraction, solving the problem of not being able to capture the key features formed by continuous data. Schlegl et al. [21] proposed an anomaly detection model based on generative adversarial networks (GAN), which provides negative training for unsupervised learning. In 2017, Hendrycks et al. [22] proposed a baseline system for OOD detection. Liang et al. [23] improved the baseline and proposed the ODIN model, and Devries [4], Shalev [24], Denouden [6], and Abdelzad et al. [5] improved the baseline in different directions, ultimately improving the detection effect. Yang et al. [25] expounded on the definition, method, evaluation, impact, and future direction of the OOD generalization problem in a review.

2.2. Datasets of Reference

2.2.1. MNIST

The MNIST dataset is a classic dataset in machine learning. It consists of 60,000 training samples and 10,000 test samples. Each sample is a 28 × 28 pixels grayscale handwritten digital picture, as shown in Figure 1. Download address: http://yann.lecun.com/exdb/mnist/ (accessed 19 on September 2022).

2.2.2. CIFAR-10

CIFAR-10 is a color image dataset closer to universal objects. CIFAR-10 is a small dataset compiled by Hinton students Alex Krizhevsky and Ilya Sutskever for identifying universal objects. It includes 10 categories of RGB color pictures: airplane, automobile, bird, cat, deer, dog, frog, horse, ship, and truck. The size of each print is 32 × 32, each category has 6000 images, and there are a total of 50,000 training pictures and 10,000 test pictures in the dataset, as shown in Figure 2. Download address: http://www.cs.toronto.edu/~kriz/cifar.html (accessed on 27 October 2022).

2.2.3. CIFAR-100

This dataset is like CIFAR-10, except it has 100 classes, and each class contains 600 images. Each category has 500 training images and 100 test images. The 100 classes in CIFAR-100 are divided into 20 superclasses. Each image has a label of its class and its superclass. Download address: http://www.cs.toronto.edu/~kriz/cifar.html (accessed on 27 October 2022).

2.3. Evaluation Metrics

There are several fixed detection indicators for OOD detection.

The true positive rate (TPR) is calculated as follows, where TP and FN represent true positives and false negatives, respectively:

$$TPR=\frac{TP}{TP+FN}$$

The false positive rate (FPR) is calculated as follows, where FP and TN indicate false positives and true negatives, respectively:

$$FPR=\frac{FP}{FP+TN}$$

The Area Under the Receiver Operating Characteristic curve (AUROC) represents the probability that the model ranks a random positive example more highly than a random negative example. Under ideal conditions, the expected score of AUROC is 100%. Area Under Precision–Recall (AUPR) is the area under the precision–recall curve reflecting the relationship between precision and recall, where precision is equal to TP/(TP + FP), and the recall rate is TP/(TP + FN). However, Hendrycks et al. [22] proposed that these are not objective and pointed out that the PR curve is related to identifying positive and negative samples. Therefore, they created two more modes to evaluate: AUPR In and AUPR Out. When the in-distribution samples are set to positive, the area under the precision-recall curve is called AUPR In. When the Out-of-Distribution samples are set to positive, the area under the precision-recall curve is called AUPR Out.

3. OOD Detection Based on Deep Learning

This section presents the definition of OOD detection based on deep learning and similar research. In addition, we present the categorization that we use to describe the methods.

3.1. Definition

The present popular deep learning methods include training data and test data. These data are ID (In Distribution) samples of IID (Independent Identical Distribution), but the application data re-input after model training is often uncontrollable in an actual application. In addition to the ID sample, the application data may be an OOD sample [26]. These OOD data may be caused by wrong data and new or unknown types of data. OOD detection’s main task is to focus on which data are different from all other data and make sure all such data are data-driven [27].

3.2. Related Works

3.2.1. Anomaly Detection

Anomaly detection (AD) aims to detect any anomalous samples that deviate from the predefined normality during testing [28]. The deviation can happen due to either a covariate shift or a semantic shift while assuming the other distribution shift does not exist. Atha et al. [29] used anomaly detection to evaluate metal surface corrosion. Patel et al. [30] addressed the problem of face spoof detection against print and replay attacks by anomaly detection. Other example applications include adversarial defense [31], image forensics [32], etc.

3.2.2. Novelty Detection

Novelty detection aims to detect any test samples that do not fall into any training category [33,34]. The rise in the use of police-operated surveillance cameras has outpaced the ability of humans to monitor them effectively. Idrees et al. [35] solved this problem with Novelty Detection. Kerner et al. [36] presented a system based on convolutional autoencoders for detecting novel features in multispectral images and applied the methodology to the detection of novel geological features in multispectral images of the Martian surface collected by the Mastcam imaging system on the Mars Science Laboratory Curiosity rover. Al-Behadili et al. [37] proposed an incremental Parzen window kernel density estimator (IncPKDE) that addresses the problems of data streaming using a model that is insensitive to the training set size and has the ability to detect novelties within multiclass recognition systems. Many conventional outlier detection tools are based on the assumption that the data are identically and independently distributed. Liu et al. [38] proposed an outlier-resistant data filter–cleaner.

3.2.3. Outlier Detection

Outlier detection aims to detect samples that are markedly different from the others in the given observation set, due to either covariate or semantic shift. Data closer in time are more correlated to each other than those farther apart. Basu et al. [39] proposed the problem of detecting unusual values or outliers from time series data. Machine learning-based strategies are examined by Xiao et al. [40] and are one type of Outlier Detection method.

3.2.4. Discussion

Anomaly detection is a technique for identifying abnormal situations and mining nonlogical data. Novelty detection finds novel data in the dataset that may belong to the classification but have not been seen. Outlier detection focuses on data points that are significantly different from other observations. Although the names of these terms are different, the core of their strategy is finding OOD samples. The difference between them is that the setting may be somewhat different [41]. Anomaly detection is generally a multicategory dataset (such as CIFAR10, which contains 10 categories). One or several are considered normal, and the other categories are considered abnormal. Out-of-Distribution detection usually uses a complete dataset as ID data and another complete dataset as OOD data (for example, CIFAR10 is ID data, while the SVHN dataset is OOD data).

3.3. Baseline Model

Background: In practical classification tasks, many highly reliable predictions are absurd and seriously wrong. So, if the classifier cannot accurately indicate when some errors occurred, which would cause serious problems, this system will be restricted in practical applications. To solve this problem, Hendrycks et al. [22] proposed an OOD detection baseline.

Application examples: Hendricks pointed out that the anomaly detection model gives misclassified samples and OOD samples a high softmax probability, so the softmax probability value cannot directly represent the confidence level of the model. The properly classified samples obtained a higher softmax probability value than the incorrectly classified and OOD samples.

Taking the image processing task as an example in the experimental process, three training sets were used: MINIST, CIFAR-10, and CIFAR-100. Naturally, the inputs consisting of the distribution of these three datasets were regarded as in-distribution. In order to construct test sets, Out-of-Distribution datasets for these were selected. The main contributions of the baseline can be summarized in three aspects:

(1)

Use softmax probability values of model predicted samples to detect OOD samples effectively;

(2)

OOD detection tasks and new evaluation indicators are developed;

(3)

A novel approach is proposed: determining whether a sample is abnormal by combining the output of a neural network with the quality of reconstructed samples.

Discussion: Experiments prove that the baseline can achieve a good recognition effect for different designated tasks, providing valuable ideas for end-to-end anomaly detection. In addition, it can be widely applied in many fields not limited to image processing and the natural language processing involved in the experiment.

3.4. Categorization

We refer to the classification method of Pang et al. [42] for anomaly detection. OOD detection can be classified by considering some criteria, such as:

• The machine learning paradigm used: supervised, semisupervised, or unsupervised;
• The different technical means: model, distance, or density.

The models used in the methods studied in this article are all based on deep learning. All methods are data-dependent because they have a training step. OOD detection is still in the initial experimental stage. There are few unsupervised methods in the existing research, so most of the methods discussed in this survey are supervised.

4. Supervised Methods

In this section, we present the supervised methods proposed in the literature for the problem of OOD detection according to the categorization adopted in this survey.

4.1. Model-Based Methods

4.1.1. Structure-Based Methods

Background: The probability output of the model does not directly represent the confidence level of the model. Therefore, the model can carry out OOD detection by learning the uncertain attributes of input samples. When testing data, if the data entered by the model are an ID sample, the uncertainty is low. Conversely, if the data entered by the model are an OOD sample, the uncertainty is high. This type of method needs to modify the network structure of the model to learn the uncertainty attribute [43].

Representative model: Devries et al. [4] proposed adding another branch to the original classification: a confidence branch to predict confidence $c$, where the input is $x$, the threshold value is $\theta$, and the predicted probability is $p$.

$$p,c=f(x,\theta )\hspace{1em}\hspace{1em}{p}_i,c\in \left[0,1\right],{\displaystyle \sum _{i=1}^M{p}_i=1}$$

During the training process, the network is indicated by interpolation, and the confidence is used to adjust the softmax prediction probability. In order to avoid some extreme situations in the network, for example, c always takes a value of 0, the confidence loss is added as a logarithmic penalty, and the confidence is always set to 1 (high confidence). After the training is completed, when the OOD sample is judged by loss estimation, the function is directly evaluated.

The model continuously improves classification performance through training and measures whether the input data are an ID sample based on confidence.

Application examples: Guénais et al. [44] proposed a Bayesian framework to obtain reliable uncertainty estimates for deep classifiers. Their approach consists of a plug-in “generator” used to augment the data with an additional class of points on the boundary of the training data, followed by Bayesian inference on top of features trained to distinguish these “out-of-distribution” points. A new nondistributed classifier based on policy entropy has been proposed by Andreas et al. [45]. This method uses policy entropy as the classification score of a class of classifiers, which can reliably detect states that are not encountered in deep reinforcement learning.

4.1.2. Threshold-Based Methods

Background: The OOD detection baseline uses the pretraining model’s maximum softmax probability of the output signal to conduct statistical analysis and judge the softmax probability distribution of OOD samples and ID samples. The distance between OOD samples and ID samples can be further expanded and then selects an appropriate threshold to evaluate the sample distribution.

Representative model: Liang et al. [23] proposed ODIN (Out-of-Distribution detector for Neural networks) based on the baseline, mainly using temperature scaling and input processing to improve the performance of OOD detection.

$f=\left(f_1,\cdots ,f_N\right)$, where N represents the classification number. $\epsilon$ is the perturbation magnitude.

Temperature scaling:

$$p_i(x;T)=\frac{\exp (f_i(x)/T)}{{\displaystyle \sum {}_{j-1}^N\exp (f_i(x)/T)}}$$

Input processing:

$$\overline{x}=x-\epsilon sign(-{\nabla }_x\log p_{\widehat{y}}(x;T))$$

ODIN uses temperature scaling and input processing to expand the softmax distribution difference between ID samples and OOD samples.

Application examples Hsu et al. [3] proposed a decomposition confidence score and an improved preprocessing method based on ODIN, making significant breakthroughs in semantic transfer and nonsemantic transfer. Zhou et al. [46] proposed contrast loss, which can improve the compactness of the representation so that OOD instances can be better distinguished from cases in the distribution. Xin et al. [47] proposed a method that introduced Channel Mean Deviation (CMD), a model-agnostic distance metric, to evaluate the statistics of the features extracted by the classification model. Single image detection is achieved by using a lightweight channel sensitivity adjustment model, which is an improvement on other statistical detection methods. A summary of similar methods since 2020 for the category is shown in

Table 1. Paper summary (2020–2021).

Number	Methodology	References
1	Generate OOD data by using ID data	[48,49]
2	Lightweight Detection of Out-of-Distribution and Adversarial Samples via Channel Mean Discrepancy	[50]
3	Learn the weights of training samples to eliminate the dependence between features and false correlations	[51]
4	The strong link between discovering the causal structure of the data and finding reliable features	[52,53]
5	Holochain-based security and privacy-preserving framework	[54]
6	Enhance robustness of Out-of-Distribution	[55,56,57,58]
7	The (OOD) detection problem in DNN as a statistical hypothesis testing problem	[59]
8	The linear classifier obtained by minimizing the cross-entropy loss after the graph convolution generalizes to out-of-distribution data	[45,60,61]
9	Invariant risk minimization (IRM) solves the prediction problem	[62]
10	The differences between scenarios and data sets will change the relative performance of the methods	[63,64]
11	pre-trains a model on OOD auxiliary outputs and fine-tunes this model with the pseudolabels	[65]
12	Nash equilibria of these games are closer to the ideal OOD solutions than the standard empirical risk minimization (ERM)	[66]
13	Interval bound propagation (IBP) is used to upper bound the maximal confidence in the l∞-ball and minimize this upper bound during training time	[67]
14	The density of states estimator is proposed	[68]
15	A new post-doc confidence calibration method is proposed, called CCAC (Confidence Calibration with an Auxiliary Class), for DNN classifiers on OOD datasets	[69]
16	The author proposes an easy-to-perform method both for group and point-wise anomaly detection via estimating the total correlation of representations in DGM	[70]
17	The author proposes OOD Analyzer, a visual analysis approach for interactively identifying OOD samples and explaining them in context	[71]

.

4.2. Distance-Based Methods

Background: This method is relatively straightforward, using a classifier to classify the extracted features to determine whether it is an OOD sample. Some methods modify the network structure to be a class classifier, which is the number of categories of the original classification task, and the first class is the OOD class. Some methods directly extract the features for classification without modifying the network structure. Although this method is straightforward, it has achieved good results.

Representative model: Abdelzad et al. [5] proposed the OODL (Out-of-Distribution discernment layer) method, which can distinguish OOD samples very easily by selecting specific and easily distinguishable layer output characteristics. Based on this, input and output data from different layers are extracted. The method uses a one-class SVM classifier, counts the classification error rate of this layer, and then selects the layer with the smallest error to detect OOD samples.

A preprocessing method to obtain $x^{\prime }$ is also proposed by adding a small perturbation to each input $x$. Then, we used the feature $x^{\prime }$ for detection.

Denote as $Q_i$ the output of network $Q$ for class $i$. $\epsilon$ is the perturbation magnitude.

$$x^{\prime }=x-\epsilon sign(-{\nabla }_x\log p({\max }_i{Q}_i\left(x\right)))$$

Application examples: Xu et al. [72] constructed a Latent Sequence Gaussian Mixture (LSGM) model to describe how the latent features in the distribution are generated across the representation space based on the traces of DNN inference. Chen et al. [73] proposed learning a shared latent space on a unit hypersphere. By using class centers and boundaries, invisible samples can be separated from visible samples.

4.3. Density-Based Methods

Background: The softmax confidence of any pretrained neural network could be replaced with an energy function. Compared with other anomaly detection methods that use pretrained models, this method does not need to adjust other model parameters due to the parameter-free feature of the energy measurement. This is different from the softmax confidence score. The probability density is aligned. Therefore, anomaly detection performance can be significantly improved.

Representative model: Li et al. [27] proposed an energy-based anomaly detection framework. OOD detection can be regarded as a binary classification problem. For the input sample model, a score value needs to be given to measure the degree of deviation of the current sample from the normal distribution. The intuitive method is to use density estimation. The energy function is used to build the density function of the model:

$E\left(x;f\right)$ represents the energy score of input $x$ and neural network $f$. $T$ is the temperature parameter.

$$p\left(x\right)=\frac{e^{-E\left(x;f\right)/T}}{{\displaystyle {\int }_x^{}{e}^{-E\left(x;f\right)/T}}}$$

Application examples: Zisselman et al. [74] introduced the residual flow, a novel flow architecture that learns the residual distribution from a base Gaussian distribution. Zong et al. [75] presented a Deep Autoencoding Gaussian Mixture Model (DAGMM) for unsupervised anomaly detection. The joint optimization, which balances autoencoding reconstruction, density estimation of latent representation, and regularization, helps the autoencoder escape from less attractive local optima and further reduces reconstruction errors, avoiding the need for pretraining. Ren et al. [76] proposed a likelihood ratio method for deep generative models, which effectively corrects for these confounding background statistics.

4.4. Performance Comparison

In this section, many representative models are based on the baseline system. In order to fully understand the performance, we used the same dataset for comparative experiments. Figure 3, Figure 4, Figure 5 and Figure 6 show the results of the representative methods of each classification under the CIFAR-10 and CIFAR-100 datasets. SMOOD is the OOD detection baseline, and ODIN, LC, and OODL are the representative models of Section 4.1.2, Section 4.1.1, and Section 4.2, respectively. The results show that the OODL effect of distinguishing the layer by finding the appropriate features is the best. Other methods with more classifications have a worse effect; the most they can reach is double the gap. OODL performance is very stable. The difference in the impact of the two datasets is about 10%, and OODL achieved a 100% correct effect on multiple projects. Two innovation modules increase the distance between ID data and OOD data, so the effect of ODIN is better in most cases (except Gaussian and Uniform). Although there is a gap with OODL, the structure is simpler and does not re-quire major changes to the original framework. LC focuses on the precise positioning of the decision boundary. The problem of the data near the classification boundary being difficult to distinguish has not been solved, so the effect of LC is only better than that of the baseline system. OODL inherits the advantages of ODIN and traverses all convolution layers to find the feature layer most suitable for ID and OOD data classification, so it has achieved the best results.

5. Semisupervised Methods

Background: This method mainly uses the reconstruction error of the autoencoder to determine whether it is an ID sample or an OOD sample. The latent space of the autoencoder can learn the obvious characteristics (silence vector) of ID data. Still, the OOD sample cannot, so the OOD sample will produce a higher reconstruction error. This type of method only focuses on OOD detection performance, without paying attention to the original task of ID data.

With the method based on VAE reconstruction, it is difficult to capture some specific abnormal samples. These samples are far from the known samples in the latent space, but they are very close to the hidden manifold.

The problem can be solved by increasing the dimension of the hidden space to capture more variation in the original data. However, this slowly deprives the model of the ability to distinguish between ID samples and OOD samples, because when the hidden space dimension is large enough, it can theoretically reconstruct any input.

Representative model: Denouden et al. [6] used the Mahalanobis distance to measure the distance between a sample $x$ and the ID training data in the manifold space:

$$D_M(x)=\sqrt{{(x-\stackrel{\wedge }{\mu })}^T{\displaystyle \sum _{}^{\wedge }{}^{-1}(x-\stackrel{\wedge }{\mu })}}$$

where $\widehat{\mu }$ and $\widehat{\mathsf{\Sigma }}$ are the mean and covariance matrices of the multivariate Gaussian distribution. The Mahalanobis distance is a constant scale and can consider the relationship between different dimensions. Finally, the reconstruction error and Mahalanobis distance can be used to detect OOD samples:

$D_m$ represents the Mahalanobis distance. $\alpha$ and $\beta$ are mixing parameters that were determined using a validation set of samples.

$$novelty(x)=\alpha \cdot D_M(E(x))+\beta \cdot \ell (x,D(E(x)))$$

Application examples: In addition to methods based on reconstruction and distance, another method is to generate some samples to surround the entire ID data manifold, train a classifier to get the dividing line of the package ID data manifold, and finally detect the OOD samples through the dividing line. Victor et al. [77], inspired by the success of variational autoencoders (VAEs) in machine learning, proposed iterative extensions of VAEs (iVAEs). Ran et al. [78] proposed an improved noise contrast prior (INCP) method to obtain reliable uncertainty estimates of standard VAE. By combining INCP with VAE, the differences between OOD and ID input can be captured and distinguished.

6. Generalization Detection

Background: The generalization problem based on OOD detection has been raised in recent years. Yang et al. [25] have already conducted a review, so we only give a simple example to illustrate.

Application examples: Zhang et al. [79] articulated and demonstrated the functional lottery ticket hypothesis: a full network contains a subnetwork that can achieve better OOD performance. They provided Modular Risk Minimization (MRM) to find these “tickets”.

The MRM algorithm is divided into four steps:

(1)

Determine the logits $\pi$ of the data, network, and subnetwork. Logit is a random distribution used to generate the mask. For example, if the network layer $l$ has $n_l$ parameters, then ${\pi }_l\in R^{n_i}$. The mask of this layer is obtained by sampling $sigmod\left({\pi }_l\right)$, and the mask $m$ transforms the complete network into a subnetwork;

(2)

Initialize the model and then use the ERM target to train $N_1$ steps;

(3)

Sample subnetworks from the entire network, combining cross-entropy and sparse regularization as a loss function to learn an effective subnetwork structure;

(4)

It is only necessary to use the weights in the obtained subnet to re-train and fix the other weights to zero.

Discussion: The major finding of the study is that MRM and the current mainstream research direction (modifying the objective function) are orthogonal. No matter the objective function, MRM can find such subnetworks with stronger generalization ability.

7. Already Applied Fields

7.1. Data Migration

With the emergence of more and more machine learning application scenarios and the existing better-performing supervised learning requiring a large amount of labeled data, labeling data is a tedious and costly task, so transfer learning is receiving more and more attention. Transfer learning refers to transferring a neural network originally used for a specific task to another new field to perform a new task [80]. The difficulty is finding the characteristic points of the new task. Xu et al. [81] introduced a simple, robust estimation criterion—transfer risk—specifically geared towards optimizing transfer to new environments. The criterion amounts to finding a representation that minimizes the risk of applying any optimal predictor trained on one environment to another. This method performs well in various Out-of-Distribution generalization tasks.

7.2. Fault Detection

When a machine is working, abnormal vibration can be used as an important feature to diagnose the working state of the machine. Normally working machines’ vibrations are smooth and regular. If a machine fails, there will be obvious abnormal vibrations. It is difficult for us to collect abnormal vibration samples in actual situations. The main reasons are as follows: (1) The industry that uses large machinery does not allow the machine to stop suddenly. (2) Most machine failures occur slowly. The uncertainty causes us to be unable to correctly estimate the time when the fault occurred, making it extremely difficult to collect data. (3) The complexity of the production environment and the influence of noise make the results produced by different environments different. Based on the Monte Carlo dropout method, Jin et al. [82] proposed a novel approach to augmenting the classification model with an additional unsupervised learning task.

7.3. Medical Image Processing

In medical imaging, the accurate diagnosis and evaluation of diseases depend on the collection and interpretation of medical images. Image acquisition has significantly improved in recent years, with equipment acquiring data at a faster rate and higher resolution. However, the image interpretation process has only recently benefited from computer technology. Doctors mostly perform the interpretation of medical images. However, medical image interpretation is limited by physician subjectivity, cognitive differences, and fatigue. Muhammad et al. [83] proposed an approach to robustly classify OOD samples in skin and malaria images without accessing labeled OOD samples during training. This method has reached its most advanced level in detecting skin cancer and malaria.

In addition to the application areas listed, with the continuous maturity of OOD detection technology based on deep learning, it is being applied in more and more fields. We summarize these applications in recent years in

Table 2. Application Summary (2020–2021).

Number	Application Field	References
1	Avian note classification	[84]
2	Natural language processing (NLP)	[85,86,87]
3	Autonomous Vehicle	[88,89]
4	Text and Image classification	[90,91,92,93] [94,95,96,97]
5	Pedestrian trajectory prediction	[98,99]
6	Digital Pathology	[100]
7	Medical imaging	[101,102,103]
8	Automated Diabetic Retinopathy Screening	[104]
9	Lung lesion segmentation	[105]
10	Autonomous robot platform	[106]
11	Drone performing vision-based obstacle avoidance	[107]
12	Particle physics collider events	[108]
13	Minimally invasive surgery (MIS)	[109]
14	Adversarial attacks (AA)	[110]
15	Automated skin disease classification	[111,112,113]
16	Machine sound monitoring system	[114,115]
17	Scene segmentation	[116]
18	AI assistance	[117,118]
19	Logical reasoning over symbols	[119]
20	Self-supervised learning (SSL)	[120]
21	Automotive perception	[121]

.

8. Challenges

OOD detection has made great progress. However, some challenges still need to be addressed to make OOD detection more effective and widely used. As a result, both the research community and society will benefit.

OOD detection based on deep learning has solved many practical problems but still faces many challenges, especially in the detection of unsupervised data. How to reduce false positives and enhance detection recall rates is one of the most important yet difficult challenges. Unsupervised methods do not have any prior knowledge of true anomalies. They rely heavily on the assumption of the distribution of anomalies. Moreover, there are unlabeled data affected by noise in unsupervised data. They may be accurately detected or incorrectly labeled, and noisy instances may be irregularly distributed in the data space. In addition, OOD detection based on deep learning works well in low-dimensional space, but OOD features are not obvious in high-dimensional space, and most methods are for point features. The detection of conditional features and group features are problems that need to be solved.

9. Conclusions

This work focuses on three categories of OOD detection based on supervised data—the model method, the distance method, and the density method—showing how each technique can distinguish OOD data from original data. In addition, we discuss their performance under the same data and model. In general, approaches are more concerned with improving the recognition effect, while failing to address space costs and efficiency issues. In this final section, we will present open issues and future research opportunities.

With the rise of deep learning, there has been a breakthrough in the performance of many tasks. OOD detection based on a deep model is of great significance to the development of AI, especially in the field of AI security. We are in an era of big data, and simply relying on manual processing speed does not meet the needs of society. Because not all data are supervised data and identically distributed, OOD detection can identify OOD samples in the input data in advance, thus helping the model to detect anomalies in the data faster, greatly reducing the model’s error rate and reducing the loss caused by practical applications. In addition, OOD detection plays an irreplaceable role in banking, transportation, medical treatment, network, and other fields. The emerging field of OOD detection is worthy of further research. However, in reality, there are not many researchers engaged in this field, and the number of related papers published is small, which leads to the slow development of this field. We hope through this review to comprehensively introduce OOD detection and attract more people to devote themselves to research in this field.