Interpretable Deep Prototype-Based Neural Networks: Can a 1 Look like a 0?

Abstract

Prototype-Based Networks (PBNs) are inherently interpretable architectures that facilitate understanding of model outputs by analyzing the activation of specific neurons—referred to as prototypes—during the forward pass. The learned prototypes serve as transformations of the input space into a latent representation that more effectively encapsulates the main characteristics shared across data samples, thereby enhancing classification performance. Crucially, these prototypes can be decoded and projected back into the original input space, providing direct interpretability of the features learned by the network. While this characteristic marks a meaningful advancement toward the realization of fully interpretable artificial intelligence systems, our findings reveal that prototype representations can be deliberately or inadvertently manipulated without compromising the superficial appearance of explainability. In this study, we conduct a series of empirical investigations that demonstrate this phenomenon, framing it as a structural paradox potentially intrinsic to the architecture or its design, which may represent a significant robustness challenge for explainable AI methodologies.

1. Introduction

Model interpretability is a critical requirement in the development of machine learning models intended to represent aspects of real-world phenomena. Numerous techniques and studies have sought to enhance interpretability, primarily through post hoc analyses or by employing surrogate models that are inherently more interpretable. Despite these efforts, a fundamental challenge persists: establishing a clear correspondence between the knowledge acquired by the model and observable elements of reality [1]. In an effort to address this issue, a Prototype Deep Neural Network (PDNN) was introduced in [2]. Prototype learning is a subclass of intrinsically interpretable methods, derived from the classical paradigm of case-based reasoning known as prototype classification. In this context, a prototype is not constrained to a single observation from the training set, but rather may constitute a generalized representation formed from a combination of multiple samples in the learned latent space [3]. The prototype learning framework typically incorporates an autoencoder in conjunction with a prototype classifier layer. The autoencoder generates a compressed latent representation of the input data, while the prototype layer identifies representative points within this space that both resemble specific inputs and are indicative of particular classes. Each learned prototype can subsequently be visualized through the decoder, enabling interpretability of the learned features, and can be traced back to the output softmax layer, thereby providing insight into class associations. Moreover, this architectural design supports a form of case-based reasoning by identifying the training sample(s) most similar to a given prototype, further enhancing the transparency and accountability of the model’s decision-making process.

In this context, while the interpretability of the method has received considerable attention, the reliability and validity of the learned prototypes have often been overlooked. For relatively simple datasets, such as handwritten digits [4], prototype validation is typically performed by reconstructing and visualizing the prototypes in the input space, followed by a qualitative assessment of whether the prototypes visually resemble the intended classes. However, this raises a critical question: can we confidently assert that a prototype which appears to represent, for instance, the digit “1”, is indeed interpreted as such by the Deep Neural Network (DNN)? In this study, we address this question through a series of experiments, including the development of a novel Deep Prototype-Based Network. Our results demonstrate that visual similarity does not necessarily align with the model’s internal representation, highlighting a potential gap between human interpretability and model-driven classification.

2. State of the Art

Deep networks can reason through learned prototypes while remaining end-to-end trainable [2]. Prototype-based neural networks offer a hybrid approach that combines the power of deep learning with the interpretability of case-based reasoning. Its architecture integrates a convolutional autoencoder with a prototype-based classification layer, allowing the model to explain its predictions through comparisons with prototypical training examples. Building upon this foundational work, ProtoPNet was proposed in [5], that model popularized the “this-looks-like-that” paradigm through patch-to-prototype visualizations, which has also been extended in Semi-ProtoPNet [6] and Ps-ProtoPNet [7] to include rejection capabilities and the combination of models in Comb-ProtoPNet [8]. The training methodology employed techniques such as prototype projection and class-specific weight regularization, achieving competitive performance across fine-grained classification tasks involving bird, car, and dermatological image datasets, while preserving interpretable, image-level justifications.

Two primary strategies have been proposed to overcome the representational limitations inherent in single-layer prototype-based models. First, ProtoTree was introduced in [9], a differentiable decision tree architecture in which the internal nodes correspond to learned prototypes. This structure enables an input to traverse only $O\left(logK\right)$ nodes, producing compact and interpretable rule chains. Second, explicit prototype hierarchies [10] organize low-level visual parts into increasingly abstract semantic concepts, facilitating multi-granular reasoning and enhancing robustness to out-of-distribution inputs.

Hybrid approaches have also been developed to enable reasoning beyond raw pixel representations. Concept Bottleneck Models (CBMs) [11] structure the prediction pipeline around human-interpretable attributes, allowing users to intervene and correct misclassified concepts at inference time. In a complementary direction, the Deep k-Nearest Neighbours (DkNN) algorithm [12] enhances arbitrary classifiers by incorporating layer-wise nearest neighbor retrieval, thereby quantifying model uncertainty through non-conformity scores. Both methods underscore the utility of leveraging intermediate representations, particularly similarity structures within hidden layers, without compromising predictive accuracy, a principle that underpins the multilayer prototype architecture proposed in this work. Additionally, Capsule Networks [13] implement dynamic routing mechanisms to capture part-to-whole relationships, while part-based R-CNNs [14] localize discriminative object regions.

The Deep Prototype Network (DPN) introduced in this work adopts a compositional reasoning framework akin to prior hierarchical or part-based models, but implements this structure through a deliberately layered architecture rather than relying on post hoc clustering of prototypes. This design choice is motivated by the principle that successive prototype layers should progressively capture increasingly abstract semantic concepts or larger spatial extents, thereby enabling richer part-to-whole reasoning directly within the network’s forward pass.

3. Deep Prototype-Based Network Architecture

Figure 1 illustrates the proposed Deep Prototype-Based Network (DPBN) architecture, which extends [2] notation, cost terms, and design philosophy, but stacks several prototype layers in a feedforward chain rather than using a single one. Therefore, each layer has similar architectural complexity to [2] (including the convolutional neural network). The proposed DPBN performs a back-and-forth data flow that couples representational capacity with prototype-based interpretability.

Let $\mathcal{D}={\left\{(x_i,y_i)\right\}}_{i=1}^N\subset \mathcal{X}\times \{1,\dots ,K\}$, $\mathcal{X}\subset {\mathbb{R}}^p$ denote a finite training set sampled from an underlying distribution ${\mathbb{P}}_{XY}$. All spaces are Euclidean and, therefore, equipped with the canonical inner product $\langle ·,·\rangle$ and the induced norm ${\parallel ·\parallel }_2$. The uppercase calligraphic symbols note sets or spaces, and the superscripts in parentheses index the depth $\ell \in \{1,\dots ,L\}$. The set of all network parameters is denoted as $\Theta =\{{\theta }_{f^{\left(1\right)}},{\{{\theta }_{e^{(\ell )}},{\theta }_{d^{(\ell )}}\}}_{\ell =2}^L,\left\{P_j^{(\ell )}\right\},W\}$.

3.1. The Forward Encoding Flow

Figure 1 shows that the forward flow begins in the input space, followed by the first encoder, which maps the input data to a latent space. Then, the transformation network f comprises L-1 stacked triples consisting of a prototype layer, the calculation of a similarity vector, and an encoder for the following layer. Finally, the prototype classifier network h takes as input the last prototype layer (L), calculates the similarity vector, and passes the results through a linear mapping with a subsequent softmax layer that produces the estimated classification distribution of an input.

The input space domain $\mathcal{X}$ is a subset of a p-dimensional real vector space. A sample is denoted by x $\in \mathcal{X}$. No further structure is assumed, except for the differentiability of all subsequent mappings defined on $\mathcal{X}$.

The first layer of the DPBN is the first encoder, which projects input data onto a lower-dimensional manifold:

$$\begin{array}{l}{f}^{\left(1\right)}:\mathcal{X}⟶{\mathcal{Z}}_1,{\mathcal{Z}}_1={\mathbb{R}}^{q_1},q_1\ll p,\\ f^{\left(1\right)}\left(x\right)=z^{\left(1\right)}.\end{array}$$

The mapping $f^{\left(1\right)}$ is usually implemented using a convolutional neural network that ends with a fully connected layer of width $q_1$. The parameters associated with this mapping are collectively denoted by ${\theta }_{f^{\left(1\right)}}$ and form part of the general parameter set.

The next layer is the first prototype layer ${\mathcal{P}}^{\left(1\right)}$,

$${\mathcal{P}}^{\left(1\right)}={\left\{P_j^{\left(1\right)}\right\}}_{j=1}^{M_1}\subset {\mathcal{Z}}_1,$$

and comprises a finite set of prototypes. The value $M_1\in \mathbb{N}$ is a fixed hyperparameter a priori, while prototypes are learned during the training process. Given a latent representation $z^{\left(1\right)}$, its similarity vector is calculated using the squared Euclidean distance to all prototypes. It is collected in the similarity vector

$$p^{\left(1\right)}\left(z^{\left(1\right)}\right)={\left(d_1^{\left(1\right)}\left(z^{\left(1\right)}\right),\dots ,d_{M_1}^{\left(1\right)}\left(z^{\left(1\right)}\right)\right)}^{\top }\in {\mathbb{R}}^{M_1},d_j^{\left(1\right)}\left(z^{\left(1\right)}\right)={∥z^{\left(1\right)}-P_j^{\left(1\right)}∥}_2^2,$$

which defines $M_1$ differentiable functions $d_j^{\left(1\right)}:{\mathcal{Z}}_1\to {\mathbb{R}}^{+}$, thereby enabling the application of gradient-based optimization methods without requiring any modifications.

The next layer is the second encoder. The resulting similarity vector $p^{\left(1\right)}\left(z^{\left(1\right)}\right)$ computed in the previous layer, which does not contain any positional ordering, is compressed by a learned distance encoder to obtain the second layer representation.

$$e^{\left(2\right)}:{\mathbb{R}}^{M_1}⟶{\mathcal{Z}}_2,{\mathcal{Z}}_2={\mathbb{R}}^{q_2},$$

$$z^{\left(2\right)}=e^{\left(2\right)}\left(p^{\left(1\right)}\left(z^{\left(1\right)}\right)\right).$$

The function $e^{\left(2\right)}$ is implemented as a multilayer perceptron, with its parameters collectively denoted by ${\theta }_{e^{\left(2\right)}}$. The representation $z^{\left(2\right)}$ can be expressed as $z^{\left(2\right)}=e^{\left(2\right)}\circ p^{\left(1\right)}\circ f^{\left(1\right)}\left(x\right)$ using the composition of functions.

Similarly to the first layer, for every subsequent layer $\ell \in \{2,\dots ,L\}$, there is an encoder that generates the latent space ${\mathcal{Z}}_{\ell }$, followed by a prototype set ${\mathcal{P}}^{(\ell )}={\left\{P_j^{(\ell )}\right\}}_{j=1}^{M_{\ell }}\subset {\mathcal{Z}}_{\ell }$, then a similarity vector $p^{(\ell )}\left(z^{(\ell )}\right)$, and finally an encoder for the next layer $\ell +1$:

$$p^{(\ell )}\left(z^{(\ell )}\right)={\left(d_1^{(\ell )}\left(z^{(\ell )}\right),\dots ,d_{M_{\ell }}^{(\ell )}\left(z^{(\ell )}\right)\right)}^{\top }\in {\mathbb{R}}^{M_{\ell }},$$

$$e^{(\ell +1)}:{\mathbb{R}}^{M_{\ell }}⟶{\mathcal{Z}}_{\ell +1},z^{(\ell +1)}=e^{(\ell +1)}\left(p^{(\ell )}\left(z^{(\ell )}\right)\right).$$

We denote as ${\Phi }^{(\ell )}\left(x\right)=e^{(\ell )}\circ p^{(\ell -1)}\circ e^{(\ell -1)}\circ \dots \circ p^{\left(1\right)}\circ f^{\left(1\right)}\left(x\right)=z^{(\ell )}$ the composite mapping that transforms x into $z^{(\ell )}$.

To provide the final classification decision after passing through all layers of the DPBN, the final similarity vector $p^{\left(L\right)}\left(z^{\left(L\right)}\right)\in {\mathbb{R}}^{M_L}$ is fed into a linear map $W\in {\mathbb{R}}^{K\times M_L}$ to obtain

$$\eta \left(x\right)=W{p}^{\left(L\right)}\left(z^{\left(L\right)}\right)\in {\mathbb{R}}^K.$$

The subsequent softmax activation function produces the estimated class distribution $\widehat{y}\left(x\right)$, where $\widehat{c}\left(x\right)=argmax\widehat{y}\left(x\right)$ is the predicted class.

A summary of the forward flow from the input vector x to the predicted class distribution is as follows:

$$x\stackrel{f^{\left(1\right)}}{\to }{z}^{\left(1\right)}\stackrel{{\mathcal{P}}^{\left(1\right)}}{\to }{p}^{\left(1\right)}\left(z^{\left(1\right)}\right)\stackrel{e^{\left(2\right)}}{\to }{z}^{\left(2\right)}\stackrel{{\mathcal{P}}^{\left(2\right)}}{\to }\dots \stackrel{e^{\left(L\right)}}{\to }{z}^{\left(L\right)}\stackrel{{\mathcal{P}}^{\left(L\right)}}{\to }$$

$$p^{\left(L\right)}\left(z^{\left(L\right)}\right)\stackrel{W+Softmax}{\to }\widehat{y}.$$

3.2. The Backward Decoding Flow

What distinguishes Prototype-Based Networks, and in particular the proposed DPBN, from regular neural networks is the presence of prototypes that, through their reconstruction back into the input space, offer visual explanations for their predictions. However, this visualization is not straightforward in the case of the DPBN. The procedure to visualize deeper prototypes introduces a novel approach called the hypersphere intersection (HS-Int) algorithm, which transforms a reconstructed similarity vector at deep ℓ, ${RS}^{(\ell )}$ in Figure 1 or ${\tilde{p}}^{(\ell )}$, into a reconstructed code ${\tilde{z}}^{(\ell )}$.

Figure 1 also illustrates the backward decoding flow, which reconstructs the deepest code $z^{\left(L\right)}$ back into the input space. This reconstruction process performs L-1 decoding and hypersphere intersection pairs until the first reconstructed code ${\tilde{z}}^{\left(1\right)}$ is obtained. Finally, ${\tilde{z}}^{\left(1\right)}$ is decoded back into $\tilde{x}$.

The hypersphere intersection algorithm is based on previous results from algebraic and geodesy/GPS positioning [15,16,17], as well as nonlinear least-squares optimization (Levenberg–Marquardt) [18,19] and the Moore–Penrose generalized inverse matrix theory [20,21].

For any depth $\ell \ge 2$, the decoder produces the radius vector

$${\widehat{p}}^{(\ell -1)}={\left({\widehat{\rho }}_1^{(\ell -1)},\dots ,{\widehat{\rho }}_{M_{\ell -1}}^{(\ell -1)}\right)}^{\top },{\widehat{\rho }}_j^{(\ell -1)}\ge 0,$$

which, together with the prototype set ${\mathcal{P}}^{(\ell -1)}={\left\{P_j^{(\ell -1)}\right\}}_{j=1}^{M_{\ell -1}}\subset {\mathcal{Z}}_{\ell -1}$, defines $M_{\ell -1}$ hyperspheres in the ambient space ${\mathcal{Z}}_{\ell -1}$ as follows:

$${\mathcal{S}}_j=\left\{z\in {\mathcal{Z}}_{\ell -1}:{\parallel z-P_j\parallel }_2^2={\widehat{\rho }}_j^2\right\},$$

where $P_j=P_j^{(\ell -1)}$ and ${\widehat{\rho }}_j^2={\left({\widehat{\rho }}_j^{(\ell -1)}\right)}^2$ to simplify the notation. Thus, the objective becomes to recover a point $z^{★}\in {\bigcap }_{j=1}^{M_{\ell -1}}{\mathcal{S}}_j,$ or, if the intersection is empty, the solution to the minimization problem:

$$z^{★}=mi{n}_{z\in {\mathcal{Z}}_{\ell -1}}\left\{F\left(z\right)\right\},F\left(z\right)=\sum _{j=1}^{M_{\ell -1}}\left(\parallel z-P_j{\parallel }_2^2-{\widehat{\rho }}_j^2\right).$$

(1)

We draw on previous studies in this field to design the algorithm in three steps. The first is to perform linear reduction by applying Bancroft’s algebraic difference trick [15] to obtain the linear system:

$$Az=b,A\in {\mathbb{R}}^{(M_{\ell -1}-1)\times q_{\ell -1}},b\in {\mathbb{R}}^{M_{\ell -1}-1}.$$

Its least-squares solution is $z_p=A^{+}b$, where $=A^{+}$ is the Moore–Penrose pseudoinverse. Every other solution differs from $z_p$ by an element of the null space of A:

$$z=z_p+N\alpha ,N\in {\mathbb{R}}^{q_{\ell -1}\times d},\alpha \in {\mathbb{R}}^d,d=q_{\ell -1}-rank\left(A\right),$$

where $rank\left(A\right)$ represents the rank of matrix A, N stacks an orthonormal basis of $kerA$, with $kerA$ being the kernel of matrix A, so d equals the number of remaining degrees of freedom and $\alpha$ are their coordinates. If $d=0$, the linear part has isolated at most one candidate; otherwise, the quadratic constraint of the next step must fix $\alpha$ (or show that there is no exact intersection).

The second step inserts the above-mentioned parameterization $z=z_p+N\alpha$ into the first sphere equation $\parallel z-P_1{\parallel }_2^2={\widehat{\rho }}_1^2$, which yields

$$q\left(\alpha \right)=\parallel z_p+N\alpha -P_1{\parallel }_2^2-{\widehat{\rho }}_1^2=0.$$

If $d=0$, no free parameters remain, and q reduces to a constant; the unique linear candidate $z_p$ is an exact intersection; otherwise, no actual solution exists.

If $d=1$, then $q\left(\alpha \right)$ is an ordinary quadratic in one variable. Its roots have closed forms tabulated in [17] producing up to two exact points, which we collect in ${\mathcal{Z}}_{\mathrm{exact}}$.

Lastly, if $d\ge 2$, then it means that the same equation defines at most a $(d-1)$-dimensional quadric in the $\alpha$ space. Therefore, we launch a Levenberg–Marquardt search [19] from $\alpha =0$ to find a feasible $\alpha$ (or certify that none exists), following the least-squares recipe in [17].

Finally, any $\alpha$ that satisfies $q\left(\alpha \right)=0$ is mapped through $z=z_p+N\alpha$ and added to ${\mathcal{Z}}_{\mathrm{exact}}$ in the third step. However, if no exact intersection is found, (${\mathcal{Z}}_{\mathrm{exact}}=⌀$), we minimize the nonlinear least-squares cost

$$F\left(z\right)=\frac{1}{2}\sum _{j=1}^{M_{\ell -1}}{\left(\parallel z-P_j{\parallel }_2^2-{\widehat{\rho }}_j^2\right)}^2.$$

The optimization process uses the Levenberg–Marquardt-damped Gauss–Newton scheme [18,19], which smoothly interpolates between gradient descent and the Gauss–Newton step via an adaptive damping factor. After convergence, we obtain $z^{★}$. The local uniqueness is verified with the Jacobian rank criterion of Abel-Chaffee [16]. The algorithm finally returns ${\tilde{z}}^{(\ell -1)}=z^{★}$ together with the uniqueness flag.

Once the HS-Int algorithm is described, we can define the decoding cascade, which starts from $z^{\left(L\right)}$, applies a decoding step

$${\widehat{p}}^{(\ell -1)}=d^{(\ell )}\left(z^{(\ell )}\right),$$

and then applies the HS-Int algorithm

$${\tilde{z}}^{(\ell -1)}=HS-Int\left({\mathcal{P}}^{(\ell -1)},{\widehat{p}}^{(\ell -1)}\right)$$

to $\ell =L,L-1,\dots ,2$. Note that the decoding step transforms a code from a latent space into a reconstructed similarity vector. Then, the HS-Int algorithm takes the reconstructed similarity vector and the set of prototypes from the corresponding prototype layer to generate the reconstructed code for the previous layer. Finally, $\tilde{x}=d^{\left(1\right)}\left(z^{\left(1\right)}\right)\in \mathcal{X}$ is the global reconstruction against which the reconstruction term in (2) is measured. This cascade enforces explicit geometric consistency between successive latent spaces, endowing every latent point with a deterministic pre-image in the input domain.

The reconstruction pathway can be summarized as

$$z^{\left(L\right)}\stackrel{d^{\left(L\right)}}{\to }{\widehat{p}}^{(L-1)}\stackrel{HSInt}{\to }{\tilde{z}}^{(L-1)}\stackrel{d^{(L-1)}}{\to }\dots {\tilde{z}}^{\left(1\right)}\stackrel{d^{\left(1\right)}}{\to }\tilde{x}.$$

The proposed DPBN provides expressive capacity and interpretability. Every depth $\ell \ge 2$ applies the composition $e^{(\ell )}\circ p^{(\ell -1)}$, which is Lipschitz continuous when $e^{(\ell )}$ employs bounded activation functions, endowing the network with a hierarchical piecewise-smooth warping of the latent metric. Yet, interpretability is preserved: each prototype $P_j^{(\ell )}$ is deterministically mapped to the input domain by the reconstruction chain, which yields a visual or otherwise perceptible exemplar representing that prototype at its innate semantic granularity.

3.3. Guided Prototype Learning

Together with the pipelines for classification and decoding, it is crucial to define the joint learning objective of the network architecture.

Let ${\mathcal{L}}_{\mathrm{CE}}(\widehat{y},y)=-ylog\widehat{y}$ be the cross-entropy between the prediction $\widehat{y}$ and the ground truth y. With ${\left\{z_i^{(\ell )}\right\}}_{i=1}^B$ referring to the latent codes of a mini-batch of size B, we define the layer-wise prototype coverages as

$${R_1}^{(\ell )}={\displaystyle \frac{1}{M_{\ell }}}\sum _{j=1}^{M_{\ell }}\underset{i=1,\dots ,B}{min}{∥P_j^{(\ell )}-z_i^{(\ell )}∥}_2^2,{R_2}^{(\ell )}={\displaystyle \frac{1}{B}}\sum _{i=1}^B\underset{j=1,\dots ,M_{\ell }}{min}{∥z_i^{(\ell )}-P_j^{(\ell )}∥}_2^2.$$

Similarly, with a mini-batch of size B, we establish the loss functions of the autoencoders for the image and the distances as

$${\mathcal{L}}_{\mathrm{img}}^{\mathrm{AE}}={\displaystyle \frac{1}{B}}\sum _{i=1}^B{∥{\tilde{x}}_i-x_i∥}_2^2{\mathcal{L}}_{\mathrm{dist}}^{\mathrm{AE}}={\displaystyle \frac{1}{B}}\sum _{i=1}^B\sum _{\ell =2}^L{∥{\tilde{d}}_i^{(\ell )}-d_i^{(\ell )}∥}_2^2.$$

As such, for positive scalars acting as regularization terms ${\lambda }_{{\mathrm{R}}_{\mathrm{I}}}$, ${\lambda }_{{\mathrm{R}}_{\mathrm{D}}}$, ${\lambda }_{{\mathrm{R}}_1}^{(\ell )}$, and ${\lambda }_{{\mathrm{R}}_2}^{(\ell )}$, the total loss reads

$$\mathcal{L}={\displaystyle \frac{1}{B}}\sum _{i=1}^B{\mathcal{L}}_{\mathrm{CE}}\left(\widehat{y}\left(x_i\right),y_i\right)+{\lambda }_{{\mathrm{R}}_{\mathrm{I}}}{\mathcal{L}}_{\mathrm{img}}^{\mathrm{AE}}+{\lambda }_{{\mathrm{R}}_{\mathrm{D}}}{\mathcal{L}}_{\mathrm{dist}}^{\mathrm{AE}}+\sum _{\ell =1}^L\left({\lambda }_{{\mathrm{R}}_1}^{(\ell )}{R}_1^{(\ell )}+{\lambda }_{{\mathrm{R}}_2}^{(\ell )}{R}_2^{(\ell )}\right).$$

(2)

All model parameters $\Theta$ are trained by stochastic gradient descent on (2), with gradients obtained via automatic differentiation through the analytic Jacobian of HS−Int whenever the exact intersection is unique; otherwise, the sub-gradient of the best-fit residual is used.

4. Experiments and Results

In this section, we conduct a systematic evaluation of the proposed Deep Prototype-Based Network, complemented by a series of targeted architectural variants. This analysis aims to demonstrate the model’s capabilities while also addressing a key challenge: the tension between visually interpretable prototype reasoning and the internal mechanisms of class assignment within the network.

The first is a baseline experiment to assess performance on the MNIST dataset [4] and to examine the prototype representations obtained from the proposed DPBN architecture (see Figure 1). The second experiment is a model distillation study in which we investigate whether the prototypes learned by the deepest layer of a four-layer DPBN can be effectively transferred to a significantly shallower model, without compromising either predictive performance or interpretability. The third experiment addresses a binary classification task designed to demonstrate that the network can produce accurate predictions and visualizations consistent with the class “0”, while its internal explanatory mechanisms align more closely with class “1”.

For each experimental configuration, we report both classification accuracy and a novel validation metric introduced in this study—the Normalized Negative Entropy (NNE) score. Quantitative findings are further supported by qualitative analyses, comprising visualizations of reconstructed prototypes and learned weight matrices. Collectively, the results elucidate the impact of each architectural choice on the reliability, interpretability, and overall performance of DPBNs.

All the experiments are conducted using the MNIST [4] benchmark dataset. The original training set, comprising 60,000 images, is randonmly partitioned into $90\%$ for training and $10\%$ for validation. The official test set of 10,000 images is held out for evaluation. All images are resized to $28\times 28$ pixels and normalized to the $[0,1]$ range using min-max scaling.

4.1. NNE Score

Prototype-Based Networks typically conclude with a linear classifier, where the weight matrix $W\in {\mathbb{R}}^{K\times M_L}$ maps prototype–sample distances to class logits. To assess the interpretability of this final classification layer, we introduce the NNE score. This metric interprets the prototype-to-class weight matrix W as an information-theoretic channel. After negating W, a row-wise softmax is applied to obtain a class probability distribution for each prototype. The Shannon entropy of each distribution is then computed, inverted, normalized by $logK$, and averaged across all prototypes. The resulting NNE score lies within the interval $[0,1]$, attaining a maximum value of 1 only when each prototype exhibits a one-hot (i.e., perfectly class-specific) voting pattern.

In contrast to geometric alignment measures—such as the B-Cos transform [22], which enhances the fidelity of saliency maps by maximizing cosine similarity between inputs and classifier weights, or the support-vector-based alignment strategy employed by WASUP [23]—the NNE metric exclusively captures the distributional sharpness of the classification head, providing a complementary perspective on model interpretability.

Because large negative distances should yield a high vote, we first negate the weight matrix and apply a row-wise softmax:

$$P=\sigma (-W^{\top })\in {[0,1]}^{M_L\times K},{\sum }_{k=1}^K{P}_{ik}=1\forall i.$$

Each row $p_i$ can now be read as a categorical distribution over classes for prototype i. As such, for a row $p_i$, we can compute its Shannon entropy

$$H\left(p_i\right)=-\sum _{k=1}^K{p}_{ik}log{p}_{ik},$$

(3)

and normalize it by the maximum possible entropy $logK$ to obtain a confidence score

$$\tilde{H}\left(p_i\right)=1-{\displaystyle \frac{H\left(p_i\right)}{logK}},\tilde{H}\left(p_i\right)\in [0,1].$$

(4)

$\tilde{H}\left(p_i\right)=1$ when $p_i$ is a one-hot vector (prototype decisively represents one class) and $\tilde{H}\left(p_i\right)=0$ when $p_i$ is uniform (prototype conveys no class information).

Finally, we average over all P prototypes:

$$NNE\left(W\right)={\displaystyle \frac{1}{P}}\sum _{i=1}^P\tilde{H}\left(p_i\right).$$

(5)

$NNE\in [0,1]$ is therefore higher for more interpretable weight matrices. We validated the metric with two extreme cases as shown at

Table 1. Extreme matrices and their corresponding normalized negative entropy (NNE) values.

Matrix	NNE
Identity ${\mathbf{I}}_K$ (perfectly sharp)	$1.000$
Uniform $\left(\frac{1}{K}\mathbf{1}\right)$ (fully diffuse)	$0.000$

.

It complements structural sparsity metrics such as global/local size and Hoyer–Square norms in ProtoS-ViT [24] and the “transparent-head complexity’’ of Shallow-ProtoPNet [25] by quantifying decisiveness even when the head is dense. Where neuron-level Class-Selectivity Indices normalize entropy over activations [26], NNE is the first to apply an entropy normalization directly to prototype voting weights, making it agnostic to input distribution and layer depth. Recent analyses of classifier weight spectra, enabled by bilinear multilayer perceptrons (MLPs) [27], underscore the value of interpreting the classification head as an information-theoretic channel. In this context, the NNE score provides a concise, task-size-invariant metric that aligns with and supports this perspective.

The proposed metric will be employed throughout this work to evaluate the quality of the classification weight matrix across all experiments, serving as a means to assess and compare the reliability and class specificity of the learned prototypes.

4.2. Experiment 1: Baseline Architecture

The objective of this experiment is twofold: to evaluate the performance of the proposed architecture on the MNIST dataset and to analyze the structure of the resulting prototype representations. As an initial step, we consider the canonical DPBN, which closely follows the configuration introduced by [2], with minor modifications to facilitate subsequent ablation studies. The input pipeline preserves the original $28\times 28$ luminance pixel values of the MNIST digits, thereby avoiding information loss during preprocessing.

Within the network, layer 1 comprises compact convolutional encoder–decoder module that projects each input image into a 40-dimensional latent vector and reconstructs it back into pixel space. The encoder employs a fixed stride pattern of $(2,2,2,2)$ and a sequence of convolutional filters $(32,32,32,10)$, progressively reducing the spatial resolution from $28\times 28$ to $2\times 2$ over four downsampling stages. This architectural design ensures that each spatial location in the final feature map corresponds to a roughly square receptive field in the input image, a property leveraged in later experiments to interpret prototype activations in a spatially meaningful manner.

The flattened feature map is subsequently projected into a latent representation of dimension $q_1=15$ via a fully connected layer. The choice of $q_1<40$ is deliberate, as it constrains the image autoencoder to learn a non-trivial, low-dimensional manifold rather than approximating the identity mapping. This serves as an implicit regularization mechanism for the initial prototype search space. Within this space, $M_1=20$ prototypes are allocated, resulting in a prototype-to-latent ratio of $M_1/q_1=1.33$. Empirical observations from preliminary experiments indicated that this ratio is sufficient to support a near one-to-one correspondence between digit classes and prototypes, while avoiding the introduction of redundant representations.

The subsequent layers (layers 2 through 4) share an identical structural design, differing from the initial layer primarily in that their encoders are implemented as MLPs operating on similarity vectors rather than raw image data. Specifically, each distance vector produced by layer l is projected into a latent space of dimension $q_{l+1}=10$, then decoded back to its original dimensionality $M_l$, and subsequently passed to the next prototype layer of equal size. In each layer, the prototypes $p_j^{\left(l\right)}$ are initialized from a uniform distribution $\mathcal{U}(0,1)$ and jointly optimized alongside the corresponding autoencoder weights. Importantly, we enforce the condition $M_l\ge q_l$ at every depth to ensure that the resulting distance matrices are overcomplete—a necessary property for the hypersphere intersection solver used to reconstruct prototype representations across layers.

The coefficients we set to ${\lambda }_{{\mathrm{R}}_{\mathrm{I}}}={\lambda }_{{\mathrm{R}}_{\mathrm{D}}}={\lambda }_{R_1}={\lambda }_{R_2}=0.5$ based on a coarse log-grid search aimed at balancing classification accuracy with reconstruction fidelity. Model training was performed using the Adam optimizer with a fixed learning rate of 1 × 10⁻⁴. An early stopping criterion with a patience of 100 epochs was employed, terminating training when validation accuracy failed to improve over 100 consecutive epochs. On the MNIST dataset, the model converged to a test accuracy of 98.05% after approximately 850 epochs, thereby establishing a strong performance upper bound for the ablation studies that follow.

Interpretability, however, diminishes with increasing network depth. As illustrated in Figure 2, reconstructed prototypes from all four layers are visually compared. Prototypes in the first layer exhibit clearly defined glyphs, each distinctly associated with a specific digit class. This observation confirms that the autoencoder, in conjunction with the $R_1/R_2$ regularization terms, effectively aligns prototypes with the underlying data manifold. By layer 2, the prototypes lose fine-grained details and become more abstract, although they still retain rough class-specific contours. In layers 3 and 4, interpretability degrades substantially: the reconstructed shapes converge into nearly indistinguishable forms, differentiated only by subtle intensity variations and with weakly class-informative interpretable patterns. Despite this loss in human interpretability, the softmax-normalized weight matrix yields a NNE score of 0.26, indicating that the classifier maintains a coherent and class-specific internal representation, even as the prototypes themselves become less visually discernible. This observation is particularly significant, as it indicates that the final-layer prototypes retain sufficient information to support accurate classification while simultaneously integrating multiple visual patterns. This behavior is a consequence of the decreasing dimensionality across successive layers, which compels the network to fuse lower-level features. Accordingly, the process can be interpreted as a hierarchical compositional mechanism that preserves discriminative attributes across layers.

4.3. Experiment 2: Model Distillation

In this experiment, we investigate whether the prototypes learned by the deepest layer of a four-layer DPBN could be effectively transferred to a significantly shallower model without compromising either predictive accuracy or interpretability. Specifically, the prototype layer extracted from the fourth layer of the reference DBPN was copied into a single-layer DPBN, where it remained fixed during training. Only the autoencoder and the classification head were subsequently trained. Qualitative comparisons between the original layer 4 prototypes and those reconstructed by the distilled model, along with the associated weight matrices W₁ and W₂, are shown in Figure 3 and Figure 4.

Despite the substantial reduction in network depth, the distilled model achieved an accuracy statistically indistinguishable from that of the baseline. Moreover, its weight matrix maintained coherence, with an NNE of 0.18. To assess whether this performance could be attributed to the information encoded in the transferred prototypes, we repeated the experiment using randomly initialized prototype vectors, which were likewise frozen during training. Contrary to our initial hypothesis, the network again attained comparable accuracy and produced sharply defined reconstructed prototypes. This suggests that the raw numerical values of the prototypes, in isolation, contribute little semantic content. Rather, interpretability appears to arise from the dynamics of the autoencoder and the spatial configuration of the prototypes within the latent space. Notably, the geometry of this space retains semantic structure: prototypes corresponding to visually similar digits, such as 8 and 9, are positioned closely, whereas those representing semantically distant digits, such as 2 and 9, are more widely separated. It is also worth noting that incorrect associations appear in the weight matrix (Figure 4), where prototype 12 (row 11) is assigned to class ‘0’, whereas it can be visually interpreted as an ‘8’ or ‘9’. We postulate that this latent space organization underlies the richer structure observed in the weight matrix when training begins with learned or trainable prototypes as opposed to random ones. These findings motivated the follow-up experiment described in the subsequent section, in which the network was trained end-to-end using randomly initialized, frozen prototypes. This design aimed to isolate the contribution of latent space geometry from the influence of prototype initialization.

4.4. Experiment 3: Cana 1 Look like a 0?

In this experiment we investigate whether the conclusions drawn in the previous multiclass setting generalize to the simpler binary task of distinguishing the digit “0” from all other handwritten digits. To construct the training corpus, all images corresponding to class “0” from the MNIST training split were retained, while samples from the remaining classes were randomly downsampled to ensure that the alternative class (denoted other) contained an equal number of instances. The resulting dataset was thus balanced, with 80% allocated for training, 10% for validation, and the remaining 10% for testing. All target labels were transformed into binary form using the indicator function $y\mapsto 1[y\ne 0]$.

We retained the deep prototype architecture of the previous experiments, but specified a four-layer prototype hierarchy $(15,20)\to (10,15)\to (10,15)\to (10,4),$ where each pair denotes the latent dimensionality, and the number of prototypes. The network was trained for 370 epochs with Adam (learning rate = ${10}^{-4}$) and early stopping after 100 epochs without validation improvement. The highest validation accuracy achieved was 99.49%, with a final test accuracy of 99.48%. Despite this strong predictive performance, the interpretability metrics were less encouraging. Specifically, the softmax-normalized weight matrix of the final classification layer exhibited a NNE of $0.14$, indicating a relatively flat association between prototypes and class labels. Furthermore, two of the four prototypes reconstructed from the deepest layer resembled noisy, amorphous “clouds” (see Figure 5) rather than well-defined digit patterns, and the weight matrix (see Figure 6) deviated substantially from a one-hot configuration. Regarding class purity or faithfulness, Figure 6 shows a faithfulness of 1/4, where all prototypes are assigned to class 1, but prototype 2 is correctly assigned to 0. This example is representative of the results achieved with other instances, which have been tested under human supervision.

To examine whether the latent geometry captured by the deep model could be more explicit, we distilled the last-layer prototypes into a shallow DPBN comprising a single layer, following the same procedure as in Experiment 2. The prototypes were held fixed during training, while only the autoencoder and classification weights were updated. Training converged after 210 epochs, achieving an accuracy of 99.83%, marginally surpassing that of the original deep model. These results support the hypothesis that a well-structured set of prototypes can facilitate more efficient representation learning, even when the prototypes themselves lack of clear visual interpretability.

A detailed examination of the distilled model revealed an noteworthy behavior. Prototype #4, whose reconstruction clearly depicts a digit “0” (see Figure 7), is associated by the weight matrix with the ’others’ class with a probability of 66 %. An analysis of the twelve test images with latent encodings closest to this prototype (Figure 8) confirms that all correspond to well-formed instances of the digit “0”, each correctly classified as class “0”. Consequently, the prototype provides clear visual evidence of the digit “0” while functionally contributing to the decision for the opposing class. Because the mapping between prototypes and classes is learned independently of the visual characteristics of the prototypes, users lacking access to the weight matrix may be misled into believing that the model’s predictions for the “others” class are based on archetypal non-zero digits, when in fact, the decisive signal is anchored in a visually canonical zero.

The binary classification setting thus amplifies a concern previously suggested by the multiclass experiments: prototype visualizations alone are insufficient to establish model trustworthiness. A prototype may function as a decoy, simultaneously presenting a compelling, human-interpretable exemplar while driving predictions for an unexpected class. Given that the distance vectors input to the linear classification head are high-dimensional, manual verification of class–prototype associations is impractical, and summary metrics such as NNE reflect only global sharpness rather than individual inconsistencies.

5. Conclusions

In this study, we introduced a novel prototype-based neural network architecture incorporating multiple prototype layers. We evaluated its performance and interpretability using the MNIST dataset, demonstrating both its representational capabilities and highlighting critical limitations concerning interpretability. Results demonstrate the network’s capacity to integrate and represent patterns across successive prototype layers. However, we observed a progressive decline in the visual interpretability of prototypes at deeper layers, complicating human understanding of the learned representations. Importantly, the decline in visual clarity at deeper prototype layers did not compromise the model’s predictive performance, which remained consistently high—indicating that the final prototype layer retained strong discriminative capacity despite reduced interpretability. A distilled shallow model, constructed by reusing the final prototype layer from the original DPBN as a fixed component, achieved a slight improvement in classification accuracy (99.83%). However, it exposed a critical flaw—prototypes visually representing “0” contributed internally to decisions for the “1” class. Hence, visualizations of prototypes alone are insufficient for model transparency and reliable interpretability, and it is required to align visual and functional behaviors. Accordingly, we argue that prototype-based explanations within the “this-looks-like-that” paradigm necessitate additional safeguards—either through training constraints that enforce alignment between visual and functional semantics, adjusting design parameters (such as number of prototypes), or via post hoc auditing tools capable of identifying contradictory associations. A comprehensive investigation into such reliability guarantees is reserved for future work.