DSMBAD: Dual-Stream Memory Bank Framework for Unified Industrial Anomaly Detection

Abstract

Industrial image anomaly detection requires the simultaneous identification of local structural and global logical anomalies. Existing methods specialize in single-type anomalies due to divergent feature requirements: structural anomalies demand fine-grained local features, while logical anomalies need semantic features. Consequently, designing a unified network architecture that effectively captures both features without task conflicts remains a key challenge. To address this problem, we propose a Dual-Stream Memory Bank Anomaly Detection (DSMBAD) framework, which enables the collaborative detection of both structural and logical anomalies from complementary perspectives. The framework consists of two memory banks: one stores patch features for detecting structural anomalies through local feature discrepancies, while the other uses segmentation maps to model component relationships for logical anomaly identification. Additionally, a feature distillation mechanism aligns features from different backbone networks to enhance global semantic information. We also introduce a shape-based anomaly scoring method that quantifies differences in component relationships using spatial–morphological features. Experimental results on the MVTec LOCO AD dataset show that our method achieves 91.0% I-AUROC (logical) and 90.8% (structural), significantly outperforming single-type models. Ablation studies confirm the dual-stream design and module effectiveness, offering a novel unified solution.

1. Introduction

Industrial image anomaly detection [1,2,3,4,5,6] is one of the core tasks in quality control for smart manufacturing, aiming to precisely identify surface defects or assembly errors of products in complex industrial scenarios. With the rapid development of deep learning technologies [7,8], unsupervised learning-based anomaly detection methods, such as autoencoders [9] and generative adversarial networks (GANs) [10,11], have gradually become mainstream due to their ability to avoid the expensive and inefficient reliance on manual annotations. However, the complexity of anomaly types in industrial settings poses higher demands on detection models. In real-world production, anomalies encompass both local structural defects, such as those shown in the last two rows of Figure 1 (e.g., wrong label on juice bottle and notched line in splicing connectors), and global logical errors, as illustrated in the first two rows of Figure 1 (e.g., missing cereal and nuts in a breakfast box and pushpins that violates logical constraints). Current methods [12,13,14,15] are often tailored to address a single type of anomaly, struggling to cover both simultaneously, which leads to high rates of missed detections or false alarms in complex scenarios, thus limiting their universal applicability in industrial settings.

Existing approaches typically handle structural and logical anomalies separately. For structural anomalies, mainstream methods [16,17,18] rely on image reconstruction or local feature discrepancy detection. For instance, autoencoders [19] localize defects through reconstruction residuals, or multi-scale pyramid networks [20,21] extract fine-grained features. However, these methods lack modeling of global semantic information, making them ineffective at detecting logical anomalies. On the other hand, logical anomaly detection methods [14,22] predominantly depend on semantic segmentation maps, analyzing relationships between components or functional consistency. For example, attention mechanisms are used to capture constraint violations (e.g., positional rules or component combinations). Nevertheless, such methods exhibit insufficient sensitivity to local structural changes, resulting in subpar performance when detecting structural anomalies. Furthermore, most existing methods rely on a single anomaly score, which struggles to uniformly quantify the differences between structural damage and logical inconsistencies, exacerbating misjudgment risks in complex scenarios.

Figure 1. Some normal (first column) and anomalous (second column) images from the MVTec LOCO AD dataset. It includes training images, test images (anomalous regions highlighted in red), the predictions of PatchCore [23], and our method. It can be observed that the detection results of PatchCore include several false positives, mistakenly identifying some non-anomalous regions as anomalies. In contrast, our method accurately localizes all the true anomalous regions.

Figure 1. Some normal (first column) and anomalous (second column) images from the MVTec LOCO AD dataset. It includes training images, test images (anomalous regions highlighted in red), the predictions of PatchCore [23], and our method. It can be observed that the detection results of PatchCore include several false positives, mistakenly identifying some non-anomalous regions as anomalies. In contrast, our method accurately localizes all the true anomalous regions.

In industrial production, structural and logical anomalies often coexist, necessitating the construction of a unified detection framework to enhance detection accuracy. However, the feature spaces of two anomaly types differ significantly: structural anomalies rely on fine-grained local features, while logical anomalies require semantic features to model global relationships. Designing a network architecture that can accommodate both features while avoiding task conflicts remains a core challenge. Additionally, anomaly scores for both types need to be fused within a unified metric space to prevent decision biases caused by discrepancies in scale; failure to address this fusion challenge inherently limits the performance ceiling of a unified framework.

To address the aforementioned problems, we propose a dual-stream unified anomaly detection framework for industrial images, enabling the collaborative detection and quantification of structural and logical anomalies. Specifically, we detect these two types of anomalies by utilizing two memory banks based on segmentation maps and patch features, respectively. The segmentation map memory bank is responsible for capturing global features, while the patch feature memory bank handles local features. Furthermore, we design a feature distillation guidance mechanism to align features generated by different backbone networks, integrating them into a unified framework. We also propose an anomaly score calculation method based on spatial–morphological features such as centroid, area, and variance, modeling relationships between components from a morphology perspective to achieve superior detection performance. We evaluate the proposed method on the public MVTec LOCO AD dataset. Our approach achieves prior I-AUROC results not only in logical anomaly detection but also in structural anomaly detection. Our contributions are listed as follows:

• We propose a dual-stream memory bank framework that separately stores segmentation maps and fine-grained patch features. By analyzing discrepancies in both streams, the model can detect structural and logical anomalies simultaneously from complementary perspectives;
• We employ a feature distillation to guide the student model to learn from different backbone networks, enhancing its anomaly detection capability. We also introduce a new anomaly scoring method based on component shapes, offering a fresh perspective on anomaly computation;
• On the MVTec LOCO AD benchmark, our method achieves 91.0% I-AUROC for logical anomalies and 90.8% for structural anomalies, outperforming the latest approaches by 0.6% in terms of the average performance. These results also surpass those of single-type models. Ablation studies confirm that the dual-stream framework is critical to effective anomaly detection.

2. Related Works

Research on industrial image anomaly detection primarily revolves around two core issues: structural anomalies (e.g., surface defects and texture damage) and logical anomalies (e.g., misaligned components and assembly errors). Existing methods propose targeted solutions from the perspectives of feature modeling, detection paradigms, and task adaptability.

Structural Anomaly Detection Methods. Structural anomalies typically manifest as physical damage in localized regions (e.g., scratches and dents) with their detection relying on fine-grained feature modeling and local discrepancy analysis. (1) Image Reconstruction-Based Methods: Representative approaches include autoencoders (AE) [24] and variational autoencoders (VAE) [25]. These methods reconstruct input images and compute residual maps to locate anomalous regions. For instance, DRAEM [26]—a discriminatively trained reconstruction embedding for surface anomaly detection—integrates a segmentation network with an anomaly generator, detecting surface anomalies through the discriminative training of a reconstruction-based embedding model. However, such methods are susceptible to interference from complex backgrounds and struggle to distinguish normal texture variations from actual defects. (2) Generative Adversarial Network (GAN)-Based Methods: Examples include boosting fine-grained visual anomaly detection with coarse-knowledge-aware adversarial learning (CKAAD) [27], which leverages coarse knowledge-aware adversarial learning to enhance fine-grained visual anomaly detection. It employs an energy discriminator to differentiate between normal and abnormal features while guiding the autoencoder to transform anomalous samples into normal ones for anomaly detection and localization. (3) Feature Similarity-Matching Methods: These methods, represented by SPADE [28], PatchCore [23], and PaDiM [29], utilize pre-trained convolutional neural networks (CNNs) [30] to extract multi-scale features and construct normal sample feature distributions (e.g., Mahalanobis distance) to localize anomalies. While sensitive to local textures, these methods lack sufficient modeling of global semantic information. Meanwhile, current approaches for structural anomaly detection overly rely on local feature discrepancies, fail to adequately model global contextual relationships, and are prone to interference from background factors, leading to missed detections of logical anomalies.

Logical Anomaly Detection Methods. Logical anomalies involve violations of topological relationships or functional consistency among components, necessitating the modeling of global semantic associations. (1) Segmentation-Based Methods: For example, comAD [14] applies K-means clustering on pre-trained features to segment multiple components within an image. SAM-LAD [31] utilizes SAM [32] to decompose images into multiple components and analyzes component correspondences to detect logical anomalies. (2) Text Generation-Based Methods: Approaches like LogicAD [33] and LogSAD [34] leverage multi-modal vision language models [35,36,37] to generate text descriptions, fine-tuning them to model and analyze images from a textual perspective. By comparing textual differences, these methods aim to identify anomalies. However, methods targeting logical anomaly detection often neglect local details, resulting in suboptimal performance in detecting structural anomalies.

3. Method

Our method consists of two components: one is patch-feature stream, which stores patch features in the patch-feature memory bank, and the other is segmentation-map stream, which stores segmentation maps in a segmentation-map memory bank. As illustrated in Figure 2, during the training phase, the normal images are used to generate patch features and segmentation maps, which are then stored in their respective memory banks. Specifically, (1) the patch-feature memory bank stores patch-level features that capture fine-grained details of the image, and (2) the segmentation-map memory bank assists in determining the validity of various component combinations to identify logical anomalies. During the test phase, the differences between the patch feature and segmentation maps of the test image and those of the training images are computed separately. Finally, the distances are calculated to derive the anomaly score. The pseudo-code of the training and test procedure is shown in Algorithm 1.

3.1. Patch-Feature Stream

To accurately detect structural anomalies in images, more fine-grained local features are required—features that can precisely locate subtle structural defects within the image. Inspired by feature extraction methods [18,23], patch features are obtained by dividing the extracted image features into small patches. This operation enhances the granularity of the original features, allowing the model to focus on localized regions of the image and accurately pinpoint the locations of structural defects.

Algorithm 1 DSMBAD training and test pseudo-code

Require: Training set ${\mathcal{X}}_{train}$, test set ${\mathcal{X}}_{test}$, cluster number K 1: Training Phase: 2: for each $x_i\in {\mathcal{X}}_{train}$ do 3:  Get patch feature fpatch according to Equation (2) 4:  Get sematic feature fs according to Equation (4) 5:  Component ← SegModule(fs, K)   # SegModule: Segmentation Module 6:  ${\mathcal{M}}_{patch}$.add(fpatch) 7:  ${\mathcal{M}}_{map}$.add(Component) 8: end for 9: Testing Phase: 10: for each $x_t\in {\mathcal{X}}_{test}$ do 11:  Get patch-feature score Spatch according to Equation (7) 12:  Get segmentation-map score Smap according to Equation (8) 13:  Get shape score Sshape according to Equation (14) 14:  ${\tilde{S}}_{map}\leftarrow S_{map}+S_{shape}$ 15:  $S_{total}\leftarrow \mathrm{Norm}\left(S_{patch}\right)+\mathrm{Norm}\left({\tilde{S}}_{map}\right)$ 16: end for

Patchifying and Pooling. For a given image $x_i$ from the training set ${\mathcal{X}}_{train}$, the lth output of the feature extractor is $g_l\in {\mathbb{R}}^{H_l\times W_l\times C_l}$ (l = 1, 2, … , m), where $H_l$, $W_l$, and $C_l$ are the height, width, and channel size of the output. To generate fine-grained patch features based on the phases illustrated in Figure 2, our patch-feature stream can be described as follows:

For each entry $g_l^{(h,w)}\in {\mathbb{R}}^{C_l}$ at location ($h,w$), its patch feature set ${\mathcal{N}}_p^{(h,w)}$ with a patchsize p is constructed as:

$$\begin{array}{cc}\hfill {\mathcal{N}}_p^{(h,w)}=\left\{(h^{\prime },y^{\prime })\right|& h^{\prime }\in [h-\lfloor p/2\rfloor ,\dots ,h+\lfloor p/2\rfloor ],\hfill \\ & y^{\prime }\in [w-\lfloor p/2\rfloor ,\dots ,w+\lfloor p/2\rfloor ]\},\hfill \end{array}$$

(1)

where ${\mathcal{N}}_p^{(h,w)}$ defines a $p\times p$ neighborhood around ($h,w$), and $\lfloor p/2\rfloor$ ensures symmetric padding. An adaptive average pooling aggregation function $F_{agg}$ is applied to the patch features in ${\mathcal{N}}_p$ to obtain aggregated features $f_{patch}$ for each layer as:

$$f_{patch}=F_{agg}\left(\left\{g_l^{(h^{\prime },y^{\prime })}\mid \left(h^{\prime },y^{\prime }\right)\in {\mathcal{N}}_p^{(h,w)}\right\}\right),$$

(2)

and the patch features $f_{patch}$ are then stored in the patch-feature memory bank for use during the subsequent test phase.

3.2. Segmentation-Map Stream

To effectively detect logical anomalies in images, it is critical to explicitly model the intrinsic constraints among components, such as spatial layouts and topological dependencies. This is achieved by first segmenting the image into semantically meaningful component maps and then systematically analyzing pairwise relationships between components through statistical metrics. By quantifying deviations from normal constraints, the framework can pinpoint violations of logical consistency, such as misassembled parts or missing components, thereby enabling precise logical anomaly detection.

Feature Distillation Guidance. Our objective is to construct a unified framework for anomaly detection. The architecture of ResNet [30], comprising stacked convolutional layers, is more suited to extract fine-grained features. However, it is less effective at generating segmentation maps directly, as evidenced by our ablation study on different teacher models. Our work is inspired by the knowledge distillation approach [38], where a teacher model can effectively guide the student model to learn the features enriched with global context—a prerequisite for constructing discriminative segmentation maps to detect logical anomalies.

Specifically, we employ a self-supervised Vision Transformer DINO [39] as the teacher model, which excels in capturing holistic semantic relationships through its attention mechanism. Since the outputs of thes two models differ, the student model needs to align with the output of the teacher model. Inspired by RD [13], we design a multi-scale feature fusion module to align their feature representations. The specific steps are as follows: for an image $x_i$ from the training set ${\mathcal{X}}_{train}$, the multi-layer feature outputs $g_l$ (l = 1, 2, … , m) of the student model is extracted by feature extractor. We employ a transformation function to synchronize these various features with the corresponding features of the teacher:

$$\begin{array}{c}\hfill \varphi \left(g_l\right)=\left\{\begin{array}{cc}\mathrm{Downsample}\left(g_l\right)\hfill & \mathrm{if}{D}_s^l>D_t^l\hfill \\ \mathrm{Upsample}\left(g_l\right)\hfill & \mathrm{if}{D}_s^l<D_t^l\hfill \\ g_l\hfill & \mathrm{if}{D}_s^l=D_t^l\hfill \end{array}\right.,\end{array}$$

(3)

where $D_s^l$ and $D_t^l$ denote the feature dimensions of layer l in the student model and the teacher model, respectively.

Finally, we obtain m dimensionally consistent features, which are concatenated to form a new feature $f_s$ as follows:

$$\begin{array}{c}\hfill f_s=F_{cat}(\varphi \left(g_1\right),\varphi \left(g_2\right),\dots ,\varphi \left(g_m\right)).\end{array}$$

(4)

After aligning the feature $f_s$ with the feature $f_t$ extracted by the teacher network through two-dimensional convolution, we define the distillation loss $L_{log}$ between them. This alignment is achieved using a mean squared error (MSE) loss during training, and it is expressed as:

$$L_{log}={\displaystyle \frac{1}{N}}\sum _{i=1}^N{(f_t^{\left(i\right)}-f_s^{\left(i\right)})}^2,$$

(5)

where N denotes the total of training samples, while $f_t^{\left(i\right)}$ and $f_s^{\left(i\right)}$ represent the features $f_s$ and $f_t$ of the ith sample, respectively.

Segmentation Module. After the feature distillation process, the features extracted by ResNet can now be utilized to construct segmentation maps. As illustrated in the segmentation module in Figure 3, K-means clustering is applied to $f_s$ to obtain k clusters. The cosine similarity between each cluster and the original feature $f_s$ is then calculated to generate an initial segmentation map. Subsequently, the segmentation map is resized to match the original image dimensions through interpolation. Finally, a fully connected Gaussian Conditional Random Field (CRF) [40] is applied as a post-processing step to refine the segmentation results. To identify noise components, we further filter each refined map using an 11 × 11 mean filter, discarding regions with maximum values below 0.5 (normalized to [0, 1]) [14], thereby enhancing foreground–background separation and yielding the final segmentation map.

3.3. Anomaly Score Computation

The proposed method stores the patch features of normal images and their corresponding segmentation maps in separate memory banks. Patch feature-based methods have been shown to effectively detect structural anomalies by focusing on local features. In contrast, logical anomalies require modeling and analyzing the relationships between components in the segmentation maps. The core of our anomaly scoring mechanism relies on comparing the test sample against the representations of normal samples stored in the two memory banks to quantify deviations from normality. During testing, for each stream, we compute the discrepancy between the test sample and its nearest neighbors within the corresponding memory bank. This discrepancy serves as the foundational anomaly score for that stream. For outliers, we use KNN-based anomaly scores, which naturally smooth out minor outliers. The following describes how to calculate anomaly scores in two memory banks.

Patch-Feature Memory Bank. This memory bank ${\mathcal{M}}_{patch}$ is constructed by storing patch features to detect structural defects following established approaches [23,29]: For each training sample $x_i\in {\mathcal{X}}_{train}$, their patch features $f_{patch}^{\left(i\right)}$ are extracted and used to build a memory bank ${\mathcal{M}}_{patch}$:

$${\mathcal{M}}_{patch}=\bigcup _{i=1}^N\left\{f_{patch}^{\left(i\right)}\right\},$$

(6)

where $f_{patch}^{\left(i\right)}$ represents the patch features of the ith training sample. The anomaly score $S_{patch}$ for the test sample $x_t\in {\mathcal{X}}_{test}$ is predicted as:

$$S_{patch}=\underset{f\in \left\{f_{patch}^{\left(t\right)}\right\}}{max}\underset{\tilde{f}\in {\mathcal{M}}_{patch}}{min}{\parallel f-\tilde{f}\parallel }^2,$$

(7)

where $f_{patch}^{\left(t\right)}$ denotes the patch feature of the test sample.

Segmentation-Map Memory Bank. This memory bank ${\mathcal{M}}_{map}$ is built by storing segmentation maps containing four components to detect logical errors, following the calculation method used during the test phase. Specifically, the method includes the following steps. (1) Area Feature: The area feature is computed by summing the total number of pixels within the segmented regions. (2) Color Feature: The image is converted from RGB space to CIELAB space, which consists of three components: L (luminance), a (green–red), and b (blue–yellow). For each pixel, luminance is ignored, and the ratio $b/a$ is calculated. The average value over the entire region is then taken as the color feature. (3) Quantity Feature: It is derived by grouping regions using DBSCAN [41] and calculating their density. By combining these three features, the anomaly score $S_{map}$ is obtained by calculating the average $L_2$ distance between the test image $x_t$ and its five nearest neighbors ${\mathcal{N}}_k$$(k=5)$ and is defined as:

$$S_{map}={\displaystyle \frac{1}{k}}\sum _{x\in {\mathcal{N}}_k}{\parallel x_t-x\parallel }^2.$$

(8)

However, we find that the computation of distances between normal and test images based on the aforementioned features is relatively independent, leading to insufficient anomaly scoring and limited generalization across complex scenarios. To address this problem, based on anomalies that often exhibit specific spatial–morphological patterns, we additionally design a new distance calculation method that incorporates spatial–morphological features–centroid ($dx,dy$), area ratio $ar$ (the ratio of the number of pixels in the segmented region to the total image), and variance v to model inter-component relationships from a shape-based perspective. The centroid captures positional consistency, $ar$ quantifies size conformity relative to the entire image, and v measures shape regularity or distribution dispersion. The computing steps are as follows:

Step 1: Relationship Calculation: For each segmented component c in image $x_i$ in ${\mathcal{X}}_{train}$, we extract three key spatial–morphological features: centroid $(d{x}_c,d{y}_c)$, area ratio $a{r}_c$, and variance $v_c$. For every unique pair of components $(p,q)$ where $p<q$, we compute the absolute difference for each feature type between the two components:

$$\begin{array}{cc}\hfill \Delta d{x}_{pq}& =|d{x}_p-d{x}_q|,\hfill \\ \hfill \Delta d{y}_{pq}& =|d{y}_p-d{y}_q|,\hfill \\ \hfill \Delta a{r}_{pq}& =|a{r}_p-a{r}_q|,\hfill \\ \hfill \Delta v_{pq}& =|v_p-v_q|.\hfill \end{array}$$

(9)

Each pair $(p,q)$ thus contributes a 4-dimensional difference vector:

$${\Delta }_{pq}=(\Delta d{x}_{\left(pq\right)},\Delta d{y}_{\left(pq\right)},\Delta a{r}_{\left(pq\right)},\Delta v_{\left(pq\right)}).$$

(10)

The complete inter-component relationship representation for each image is the set of all pairwise difference vectors:

$$R=\left\{{\Delta }_{pq}\mid 1\le p<q\le K-1\right\},$$

(11)

where K denotes the number of clusters.

Step 2: Threshold Determination: Using the training set of normal images ${\mathcal{X}}_{train}$, we learn the acceptable range for each type of pairwise difference feature. For each difference feature type $k\in \{dx,dy,ar,v\}$ and for each possible component pair $(p,q)$, we collect all values $\Delta k_{pq}^{\left(i\right)}$ across all training images $x_i\in {\mathcal{X}}_{train}$, and the maximum and minimum values of $\Delta k_{pq}^{\left(i\right)}$ are obtained. The deviation threshold T is defined as the width of the normal range to define the normal range of deviations:

$$\begin{array}{cc}\hfill \Delta {\tilde{k}}_{pq}& =max(\Delta k_{pq}^{\left(i\right)})-min(\Delta k_{pq}^{\left(i\right)}),\hfill \\ \hfill T& =\left\{\Delta {\tilde{k}}_{pq}\mid 1\le p<q\le K-1\right\}.\hfill \end{array}$$

(12)

Step 3: Distance Calculation: As shown in Figure 4, for a test image $x_t$, its inter-component relationship representation $R^{\left(t\right)}=\left\{{\Delta }_{pq}^{\left(t\right)}\right\}$ is extracted. The nearest neighbors $x_r$ of the test image are retrieved from ${\mathcal{N}}_k$, and the relationships $R^{\left(r\right)}=\left\{{\Delta }_{pq}^{\left(r\right)}\right\}$ are identified as the reference relationships. For each feature type k within each pairwise difference vector ${\Delta }_{pq}^{\left(t\right)}$, the deviation between reference relationships and the relationships of the test image is computed as:

$$D_{k,pq}=|R_{k,pq}^{\left(t\right)}-R_{k,pq}^{\left(r\right)}|.$$

(13)

If the difference $D_{k,pq}$ exceeds the threshold $T_{k,pq}$, it is considered as an anomaly and the $L_2$-distance is calculated, i.e.:

$$S_{shape}=\sum _{1\le p<q\le K-1}\sum _{k\in \{dx,dy,ar,v\}}{\parallel R_{k,pq}^{\left(t\right)}-R_{k,pq}^{\left(r\right)}\parallel }^2,\mathrm{if}{D}_{k,pq}>T_{k,pq}.$$

(14)

$S_{shape}$ is added as part of the anomaly score for the segmentation map. The process is defined as:

$${\tilde{S}}_{map}=S_{shape}+S_{map}.$$

(15)

Aggregation of Anomaly Scores. The anomaly scores derived from the two memory banks originate from different feature spaces and calculation methods, leading to inherent differences in their numerical scales and distributions. Directly summing these uncalibrated scores would cause the score with the larger inherent scale to dominate the aggregated result, potentially masking the signal from the other anomaly type. This imbalance would compromise the accuracy of the final anomaly score. To ensure that both scores contribute proportionally to the final result based on their respective deviation magnitudes, we first normalize both scores, and the final aggregated anomaly score $S_{total}$ is obtained after normalization, i.e.,:

$$S_{total}=Norm\left(S_{patch}\right)+Norm\left({\tilde{S}}_{map}\right),$$

(16)

where $Norm(·)$ denotes the normalizing operation. Specifically, it is derived from $\left(S-\mu \right)/\sigma$, where S represents the original anomaly score, while $\mu$ and $\sigma$ denote the mean and standard deviation, respectively.

4. Experiment

4.1. Implementation Details

Dataset. (1) The MVTec LOCO AD dataset [42], which is widely recognized as a benchmark for detecting logical and structural anomalies simultaneously, is employed to evaluate all the compared methods discussed in this paper. This dataset includes five categories, with the training and validation sets consisting exclusively of normal images, while the test set contains both normal and abnormal images. Notably, the abnormal images in the test set encompass both structural and logical anomalies, making this dataset the only one that characterizes both types of anomalies. (2) Another dataset, the MVTec AD dataset [43], comprises 10 object categories and 5 texture categories. Each category consists of a training set with normal images and a test image with only structural anomalies.

Settings. We adopt a pre-trained WideResNet50 [44] network as our backbone network. In the patch-feature stream, the patchify and pooling operations follow the settings of PatchCore [23]. In the segmentation-map stream, the teacher model uses DINO ViT-S/8 [39] as the backbone network to guide the student model in generating features rich in global information. The student model extracts features with dimensions of 256, 512, and 1024 from the images corresponding to the first, second, and third layers, respectively, i.e., m set to 3, which are then aligned with the features generated by the teacher model through a multi-scale feature fusion process. The student model is trained using the Adam optimizer with a learning rate of 0.05, 500 iterations, and a batch size of 8. In the segmentation module, all images are resized to 224 × 224, and the number of cluster K is set to 5. In our comparative analysis, we benchmark our model against several state-of-the-art models including DSKD [21], LogicAD [33], SLSG [45], PatchCore [23], SINBAD [46], SimpleNet [18] and ComAD [14]. Following the previous research, we utilize the Area Under the Receiver Operating Characteristic (I-AUROC) as the principal quantitative metric to evaluate image-level anomaly detection performance on the MVTec LOCO AD dataset and MVTec AD dataset. We use the saturated Per-Region Overlap (sPRO) metric [42] to evaluate the anomaly localization performance with the per-pixel false positive rate of 5%, which is a generalized version of the PRO metric [43]. This metric saturates once the overlap with the ground truth reaches a predefined saturation threshold. All thresholds are provided by the MVTec LOCO AD dataset.

4.2. Quantitative and Qualitative Results

The anomaly detection results on the MVTec LOCO AD are shown in

Table 1. Comparison between our model and existing methods on the MVTec LOCO AD dataset in terms of both logical anomaly (LA) and structural anomaly (SA) with I-AUROC. The best and second-best performance scores are indicated in bold and underlined, respectively.

	Category/Method	DSKD  [21]	LogicAD  [33]	SLSG  [45]	PatchCore  [23]	SINBAD  [46]	SimpleNet  [18]	ComAD  [14]	Ours
LA	Breakfast Box	-	93.1	-	80.0	96.5	77.1	94.7	93.0
	Juice Bottle	-	81.6	-	92.3	96.6	87.8	90.9	95.6
	Pushpins	-	98.1	-	73.8	83.4	69.0	89.0	96.3
	Screw Bag	-	83.8	-	55.7	78.6	51.6	79.7	77.3
	Splicing Connectors	-	73.4	-	75.6	89.3	72.0	84.4	92.6
	Average (Logical)	81.2	86.0	89.6	75.5	88.9	71.5	87.8	91.0
SA	Breakfast Box	-	-	-	75.2	87.5	80.9	70.0	83.4
	Juice Bottle	-	-	-	97.8	93.1	90.4	80.5	97.2
	Pushpins	-	-	-	81.9	74.2	81.6	93.8	94.7
	Screw Bag	-	-	-	88.6	92.2	83.3	65.0	86.6
	Splicing Connectors	-	-	-	94.9	76.7	82.6	63.8	92.0
	Average (Structural)	86.9	-	91.4	87.7	84.7	83.8	74.6	90.8
Average		84.0	-	90.3	81.6	86.8	77.6	81.2	90.9

, where the image-level anomaly scores are calculated using Equation (16). As shown in the Table 1, our method achieves the highest overall performance when detecting both types of anomalies simultaneously.

Models specifically designed to detect logical anomalies, such as LogicAD [33] and ComAD [14], tend to perform superior on logical anomaly detection than on structural anomaly detection. Compared with them, our method not only outperforms these models in structural anomaly detection but also surpasses them in detecting logical anomalies. Similarly, compared to models like SimpleNet [18] and PatchCore [23] tailored for structural anomaly detection, our approach achieves superior performance in both structural and logical anomaly detection. Moreover, when compared with models like DSKD [21], SINBAD [46] and SLSG [45] explicitly designed to detect both types of anomalies simultaneously, our method achieves better performance in terms of the average performance, with 91.0% and 90.8% I-AUROC for logical and structural anomalies, surpassing the second-best method in average scores by 0.6%. For the image anomaly localization metric sPRO, as shown in

Table 2. Comparison between our model and existing methods on the MVTec LOCO AD dataset in terms of the average of both logical and structural anomaly with sPRO. The best and second-best performance scores are indicated in bold and underlined, respectively.

Category/Method	DSKD  [21]	GCAD  [42]	THFR  [47]	GLCF  [15]	SLSG  [45]	Ours
Breakfast Box	56.8	50.2	58.3	52.8	-	72.1
Juice Bottle	86.5	91.0	89.6	91.3	-	91.5
Pushpins	82.5	73.9	76.3	61.5	-	75.9
Screw Bag	62.7	55.8	61.5	61.5	-	70.2
Splicing Connectors	76.7	79.8	84.8	78.5	-	86.6
Average	73.0	70.1	74.1	70.3	67.3	79.3

, our method outperforms the second-best comparative approach by 5.2%, demonstrating the accuracy of our anomaly localization capability. The results from both metrics exhibit that our dual-stream framework effectively resolves the performance trade-off between different anomaly types, offering a unified solution.

We also compared the efficiency of our method with other models, as shown in

Table 3. Efficiency comparison of different methods. Performance is evaluated by I-AUROC/sPRO metrics. The best and second-best performance scores are indicated in bold and underlined, respectively.

Method	Params	FLOPs	Latency (ms)	Performance
PatchCore [23]	-	-	47.1	81.6/-
DADF [48]	298.5 M	149.3 G	54.5	83.7/67.4
SAM-LAD [31]	-	34.6 G	526.3	90.7/83.2
Ours	90.9 M	48.4 G	12.2	90.9/79.3

. Compared to existing approaches, our model employs fewer parameters and FLOPs, achieving an inference speed of 12.2 ms. Although our method slightly lags behind SAM-LAD in terms of the sPRO metric, we achieve significantly faster inference—over 40 times quicker. A possible reason is that SAM-LAD employs multiple backbone networks along with the SAM (Segment Anything Model) [32], leading to a larger number of parameters during inference, which slows down the process. This further demonstrates that our method strikes a better balance between performance and efficiency.

The detection results on the MVTec-AD dataset are shown in

Table 4. Anomaly detection results in terms of the I-AUROC on the MVTec-AD dataset. The best and second-best performance scores are indicated in bold and underlined, respectively.

Category/Method	HGAD [49]	MoEAD [50]	PatchCore [23]	RD [13]	PaDiM [29]	Ours
Carpet	100	99.7	98.7	98.9	99.8	98.9
Grid	99.6	99.2	98.2	100	96.7	98.5
Leather	100	100	100	100	100	100
Tile	100	99.5	98.7	99.3	98.1	98.9
Wood	99.5	98.6	99.2	99.2	99.2	99.1
Average (Texture)	99.8	99.4	99.0	99.5	98.8	99.1
Bottle	100	100	100	100	99.9	100
Cable	97.3	97.6	99.5	95.0	92.7	99.7
Capsule	99.0	92.5	98.1	96.3	91.3	98.3
Hazelhut	99.9	100	100	99.9	92.0	100
Metal Nut	100	99.4	100	100	98.7	100
Pill	96.3	94.7	96.6	96.6	93.3	97.1
Screw	95.5	92.7	98.1	97.0	85.8	98.5
Toothbrush	91.2	94.2	100	99.5	96.1	100
Transistor	97.7	99.8	100	96.7	97.4	99.9
Zipper	100	98.3	99.4	98.5	90.3	99.5
Average (Object)	97.7	96.9	99.2	98.0	93.8	99.3
Average	98.4	97.7	99.1	98.5	95.4	99.2

. On the MVTec-AD dataset, which contains only structural anomalies, our model demonstrates commendable performance with 99.2% compared to models [13,23,29] specifically designed to address structural anomalies, surpassing PatchCore [23] by 0.1% in I-AUROC. Furthermore, our method also exhibits notable advantages over HGAD [49] and MoEAD [50], which are recently new methods for detecting structural anomalies. Our method surpasses theirs by 0.8% and 1.5%, respectively.

Representative samples and the corresponding anomaly localization results are visualized in Figure 5 and Figure 6.

As shown in Figure 5, our method demonstrates superior performance in detecting logical anomalies compared to PatchCore. For instance, in the “breakfast box” category, PatchCore misclassifies certain non-anomalous regions—the fruit in the left half of the third column—as anomalies, resulting in false positives. Similar issues are observed across other categories. In the “juice bottle” category, where the color of the juice is highly similar to that of the label, making them difficult to distinguish, our model accurately identifies the location of the missing label. Moreover, our method successfully detects the absence of the liquid that should be present inside the bottle and verifies whether the liquid and label correspond correctly. In the “pushpins” category, the grid containing the pushpins is easily confused with the background. Nevertheless, our model effectively detects violations of the logical constraints—where each grid should contain exactly one pushpin. In the “splicing connectors” category, our approach also successfully identifies cases where two connectors are incorrectly linked.

As illustrated in Figure 6, in the case of structural anomalies, PatchCore performs poorly in certain categories such as “splicing connectors” and “screw bag”, where defects are often small and easily obscured by background noise, making them difficult to detect. Despite these challenges, our model successfully detects and precisely localizes such subtle and localized anomalies, including incomplete oranges in the “breakfast box”, damaged packaging and broken washers in the “screw bag”, and fractured connector lines and contaminated connectors in the “splicing connectors”. These results demonstrate the effectiveness of our method in identifying a wide variety of structural defects.

4.3. Ablation Studies

Ablation Study on Different Memory Banks. The experimental results in

Table 5. Performance comparison on MVTec LOCO AD with different memory banks (measured in I-AUROC). ${\mathcal{M}}_{patch}$ and ${\mathcal{M}}_{map}$ represent the patch-feature and segmentation-map memory banks, respectively. The best and second-best performance scores are indicated in bold and underlined, respectively. The “✓” denotes the scenario in which the module is used.

${\mathcal{M}}_{\mathit{patch}}$	${\mathcal{M}}_{\mathit{map}}$		Logical Anomaly	Structural Anomaly	Average
${\mathit{S}}_{\mathit{patch}}$	${\mathit{S}}_{\mathit{map}}$	${\mathit{S}}_{\mathit{shape}}$
✓			75.5	87.7	81.6
	✓		87.5	74.0	80.8
		✓	80.7	75.1	77.9
✓	✓		90.8	90.7	90.8
✓		✓	85.3	88.3	86.8
	✓	✓	88.1	74.5	81.3
✓	✓	✓	91.0	90.8	90.9

validate the effectiveness of different memory banks of our method in handling various types of anomalies. As shown in the first three rows of the table, when using only a single memory bank—such as the patch-feature memory bank—the model achieves strong structural anomaly detection (87.7% I-AUROC), surpassing the only use of the segmentation-map memory bank by 13.2%. This performance gap confirms that patch features excel at capturing fine-grained local details for structural defects. Similarly, in the segmentation-map memory bank, we model the relationships between components, and the logical anomaly detection reaches 87.5% I-AUROC higher than the only use of the patch-feature memory bank by 12.6%. This demonstrates targeted effectiveness in detecting logical anomalies. When combining the patch feature score $S_{patch}$ with either part ($S_{map}$ or $S_{shape}$) of the segmentation map score, the overall performance improves. For instance, combining $S_{patch}$ with $S_{shape}$ (based on spatial–morphological features) elevates the average I-AUROC to 86.8%, which is driven by a notable rise in logical anomaly detection from 75.5% to 85.3%. Similarly, integrating $S_{patch}$ with $S_{map}$ (which models component relationships via color and quantity features), the average I-AUROC increases from 81.6% (patch-only) to 90.8% with structural anomaly detection improving from 87.7% to 90.7%. Moreover, only when both $S_{map}$ and $S_{shape}$ are jointly fused with $S_{patch}$ does the model achieve optimal performance (91.0% for logical and 90.8% for structural anomalies). The experiment in the last row integrates the strengths of both memory banks—leveraging their respective advantages in structural and logical anomaly detection—achieving the best overall performance. This result confirms the effectiveness of the proposed dual-stream memory bank framework.

Ablation Study on Different Modules. The experimental results presented in

Table 6. Performance comparison on MVTec LOCO AD with different modules and different teacher models (measured in I-AUROC). The experiments in the table are based on the results obtained under the dual-stream architecture. The “w/o” denotes the scenario without feature distillation. The best and second-best performance scores are indicated in bold and underlined, respectively. The “✓” denotes the scenario in which the module is used.

Teacher	Feature Distillation	Shape-Based Anomaly Score	Logical Anomaly	Structural Anomaly	Average
w/o			80.3	86.9	83.6
w/o		✓	82.5	87.8	85.2
WideResNet50	✓	✓	82.1	89.2	85.7
DINO	✓		90.8	90.7	90.8
DINO	✓	✓	91.0	90.8	90.9

demonstrate the impact of different modules on the overall model performance. The experiments in the second and third rows of the table verify the effectiveness of the two proposed modules. Specifically, the results in the second row indicate that the knowledge learned by the student model through knowledge distillation is effective for anomaly detection. The results in the third row confirm the importance of the shape-based anomaly score in capturing the shape relationships between components. Furthermore, by comparing the results of the first and the last rows, it is evident that the integration of both modules contributes to a further improvement in the overall performance of the model.

Ablation Study on Different Teacher Models. To further validate the impact of the teacher model on the performance of the dual-stream framework, we compare three configurations: “w/o” (without feature distillation), “WideResNet50” (WideResNet50 as the teacher model), and “DINO” (DINO as the teacher model), as shown in Table 6. The experimental results demonstrate that removing the feature distillation mechanism (w/o) leads to I-AUROC scores of 82.5% for logical anomalies, 87.8% for structural anomalies and 85.2% for the average. This indicates that while the student model alone can effectively capture local structural anomalies through independently extracted features, it lacks sufficient capability in modeling global logical anomalies. When WideResNet50 is employed as the teacher model, the performance on structural anomaly detection reaches 89.2%, but the score for logical anomalies slightly drops to 82.1%. This reflects that the multi-scale convolutional framework of WideResNet50 enhances local feature alignment but lacks global semantic guidance, which is critical for logical anomaly detection. In contrast, when DINO is used as the teacher model and under the dual-stream architecture, both logical and structural anomaly detection performance improves significantly, achieving 91.0% and 90.8% I-AUROC, respectively, with an average of 90.9%. DINO captures global semantic relationships in images through self-supervised pre-training, guiding the student model to generate highly discriminative segmentation maps. Additionally, its multi-scale feature fusion mechanism optimizes local feature alignment, leading to balanced improvements in the detection of both anomaly types. These experiments confirm that the choice of teacher model is crucial for the success of the unified framework.

Ablation Study on Different Cluster numbers. This experiment investigates the impact of the number of cluster (K) in the segmentation module on anomaly detection performance, as shown in

Table 7. Performance comparison on MVTec LOCO AD with different cluster numbers (measured in I-AUROC). The best and second-best performance scores are indicated in bold and underlined, respectively.

K	Logical Anomaly	Structural Anomaly	Average
3	86.0	89.4	87.7
4	89.3	90.7	90.0
5	91.0	90.8	90.9
6	90.2	90.3	90.3
7	90.1	90.3	90.2

. The experimental results show that when $K=3$, the performance is the worst in both cases, in which the I-AUROC of logical anomalies and structural anomalies are 86.0% and 89.4%, respectively, with an average of 87.7%. A small number of clusters proves insufficient for accurately segmenting complex components, thereby limiting the performance of logical anomaly detection. As K increases to 5, the I-AUROC of logical anomaly detection has improved significantly from 86.0% to 91.0%, while structural anomaly detection remains stable at 90.8%, resulting in an average of 90.9%. This indicates that a moderate number of clusters can effectively capture semantic relationships among key components for identifying logical anomalies while preserving local details for the precise localization of structural defects. However, further increasing K to 6 or 7 leads to a slight decline in logical anomaly detection performance to 90.2% and 90.1%, respectively, with the average of both dropping to 90.3% and 90.2%. This may be attributed to excessive clustering causing redundant segmentation maps and introducing noise interference. These experiments confirm that $K=5$ is the optimal choice on the selected dataset, achieving fine-grained component segmentation, thus validating the effectiveness of the default setting used in this study.

Ablation Study on Different Scales of Memory Bank. We conduct experiments to investigate the impact of memory bank scale on anomaly detection performance, as presented in

Table 8. Performance comparison on MVTec LOCO AD with different scales of memory bank (measured in I-AUROC). The memory bank scale represents the scale of training samples used from the memory bank. The best and second-best performance scores are indicated in bold and underlined, respectively.

Memory Bank Scale	Logical Anomaly	Structural Anomaly	Average
1%	88.1 ± 2.5	87.9 ± 2.8	88.0 ± 2.7
25%	89.6 ± 1.4	88.7 ± 2.1	89.2 ± 1.7
50%	90.0 ± 1.0	89.2 ± 1.6	89.6 ± 1.3
75%	90.5 ± 0.5	89.9 ± 0.9	90.2 ± 0.7
100%	91.0	90.8	90.9

. The results show that as the memory bank scale increases from 1% to 100%, the I-AUROC of logical and structural anomalies improves significantly from 88.1% and 87.9% to 91.0% and 90.8%, respectively, with the average performance increasing from 88.0% to 90.9%. When the memory bank scale is small (e.g., 1%), the limited coverage of stored patch features and segmentation maps restricts the model’s ability to fully capture the diversity of normal samples. This leads to notable fluctuations in logical anomaly detection (STD ± 2.5) and limited generalization capability for local details in structural anomaly detection (STD ± 2.8). As the memory bank scale increases to 75%, the STDs for logical and structural anomalies drop to ±0.5 and ±0.9, respectively, indicating that a larger memory bank enhances the model’s robustness to the normal data distribution and reduces variance caused by random sampling. At a 100% memory bank scale, the model achieves the highest performance by comprehensively learning both patch-level features and component relationships from all training samples, resulting in I-AUROC scores of 91.0% for logical anomalies and 90.8% for structural anomalies. These experiments validate the positive correlation between memory bank scale and detection performance—that is, a larger memory bank scale lead to better detection accuracy. They also highlight the need to balance computational cost and accuracy in practical applications.

5. Conclusions

In this paper, we propose a novel framework for image anomaly detection, DSMBAD, which unifies the tasks of structural and logical anomaly detection through a dual-stream memory bank discrepancy analysis. The patch-feature memory bank localizes structural defects by comparing stored fine-grained patch features, while the segmentation-map memory bank enhances the detection of logical anomalies by modeling and contrasting inter-component relationships. The designed feature distillation guidance, along with the shape-based anomaly score mechanism, further contributes to improving the overall performance of the model. Experimental results demonstrate competitive improvements in accuracy for both structural and logical anomaly detection tasks. Despite superior performance, DSMBAD has the limitation that the dual-stream design increases computational and memory costs, which could limit deployment. Future work will explore methods for compressing anomaly detection networks, such as network pruning techniques [51], to improve deployment efficiency.